-- Logs begin at Wed 2022-08-31 08:58:38 UTC, end at Wed 2022-08-31 10:15:39 UTC. -- Aug 31 09:32:33.524120 dv-cinder-844961-2-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Aug 31 09:32:33.534387 dv-cinder-844961-2-iscsi devstack@keystone.service[88096]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Aug 31 09:32:33.536455 dv-cinder-844961-2-iscsi devstack@keystone.service[88096]: *** Starting uWSGI 2.0.18-debian (64bit) on [Wed Aug 31 09:32:33 2022] *** Aug 31 09:32:33.536455 dv-cinder-844961-2-iscsi devstack@keystone.service[88096]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Aug 31 09:32:33.536455 dv-cinder-844961-2-iscsi devstack@keystone.service[88096]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Aug 31 09:32:33.536455 dv-cinder-844961-2-iscsi devstack@keystone.service[88096]: nodename: dv-cinder-844961-2-iscsi Aug 31 09:32:33.536455 dv-cinder-844961-2-iscsi devstack@keystone.service[88096]: machine: x86_64 Aug 31 09:32:33.536455 dv-cinder-844961-2-iscsi devstack@keystone.service[88096]: clock source: unix Aug 31 09:32:33.536455 dv-cinder-844961-2-iscsi devstack@keystone.service[88096]: pcre jit disabled Aug 31 09:32:33.536455 dv-cinder-844961-2-iscsi devstack@keystone.service[88096]: detected number of CPU cores: 4 Aug 31 09:32:33.536455 dv-cinder-844961-2-iscsi devstack@keystone.service[88096]: current working directory: / Aug 31 09:32:33.536455 dv-cinder-844961-2-iscsi devstack@keystone.service[88096]: detected binary path: /usr/bin/uwsgi-core Aug 31 09:32:33.536455 dv-cinder-844961-2-iscsi devstack@keystone.service[88096]: your processes number limit is 47680 Aug 31 09:32:33.536455 dv-cinder-844961-2-iscsi devstack@keystone.service[88096]: your memory page size is 4096 bytes Aug 31 09:32:33.537078 dv-cinder-844961-2-iscsi devstack@keystone.service[88096]: detected max file descriptor number: 2048 Aug 31 09:32:33.537078 dv-cinder-844961-2-iscsi devstack@keystone.service[88096]: lock engine: pthread robust mutexes Aug 31 09:32:33.537078 dv-cinder-844961-2-iscsi devstack@keystone.service[88096]: thunder lock: enabled Aug 31 09:32:33.537078 dv-cinder-844961-2-iscsi devstack@keystone.service[88096]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Aug 31 09:32:33.537078 dv-cinder-844961-2-iscsi devstack@keystone.service[88096]: Python version: 3.8.10 (default, Jun 22 2022, 20:18:18) [GCC 9.4.0] Aug 31 09:32:33.580023 dv-cinder-844961-2-iscsi devstack@keystone.service[88096]: Python main interpreter initialized at 0x55c3727c09c0 Aug 31 09:32:33.580023 dv-cinder-844961-2-iscsi devstack@keystone.service[88096]: python threads support enabled Aug 31 09:32:33.580023 dv-cinder-844961-2-iscsi devstack@keystone.service[88096]: your server socket listen backlog is limited to 100 connections Aug 31 09:32:33.580023 dv-cinder-844961-2-iscsi devstack@keystone.service[88096]: your mercy for graceful operations on workers is 90 seconds Aug 31 09:32:33.580430 dv-cinder-844961-2-iscsi devstack@keystone.service[88096]: mapped 403077 bytes (393 KB) for 2 cores Aug 31 09:32:33.580430 dv-cinder-844961-2-iscsi devstack@keystone.service[88096]: *** Operational MODE: preforking *** Aug 31 09:32:33.580812 dv-cinder-844961-2-iscsi devstack@keystone.service[88096]: *** uWSGI is running in multiple interpreter mode *** Aug 31 09:32:33.580812 dv-cinder-844961-2-iscsi devstack@keystone.service[88096]: spawned uWSGI master process (pid: 88096) Aug 31 09:32:33.581083 dv-cinder-844961-2-iscsi systemd[1]: Started Devstack devstack@keystone.service. Aug 31 09:32:33.584712 dv-cinder-844961-2-iscsi devstack@keystone.service[88096]: spawned uWSGI worker 1 (pid: 88113, cores: 1) Aug 31 09:32:33.584712 dv-cinder-844961-2-iscsi devstack@keystone.service[88096]: spawned uWSGI worker 2 (pid: 88114, cores: 1) Aug 31 09:32:33.584712 dv-cinder-844961-2-iscsi devstack@keystone.service[88096]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Aug 31 09:32:38.012293 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 31 09:32:38.012293 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: from cryptography.utils import int_from_bytes Aug 31 09:32:38.013175 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 31 09:32:38.013175 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: from cryptography.utils import int_from_bytes Aug 31 09:32:38.013296 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 31 09:32:38.013296 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: from cryptography.utils import int_from_bytes Aug 31 09:32:38.013296 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 31 09:32:38.013296 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: from cryptography.utils import int_from_bytes Aug 31 09:32:38.065757 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=88113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Aug 31 09:32:38.066005 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=88114) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Aug 31 09:32:38.066144 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=88113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Aug 31 09:32:38.066301 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=88114) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Aug 31 09:32:38.066415 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG uwsgi [-] command line args: [] {{(pid=88113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Aug 31 09:32:38.066536 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG uwsgi [-] command line args: [] {{(pid=88114) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Aug 31 09:32:38.066663 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=88113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Aug 31 09:32:38.066787 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=88114) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Aug 31 09:32:38.066919 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG uwsgi [-] ================================================================================ {{(pid=88113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Aug 31 09:32:38.067041 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG uwsgi [-] ================================================================================ {{(pid=88114) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Aug 31 09:32:38.067156 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG uwsgi [-] admin_token = **** {{(pid=88113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 09:32:38.067274 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG uwsgi [-] admin_token = **** {{(pid=88114) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 09:32:38.067400 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG uwsgi [-] config_dir = [] {{(pid=88113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 09:32:38.067523 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG uwsgi [-] config_dir = [] {{(pid=88114) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 09:32:38.067650 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=88113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 09:32:38.067778 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=88114) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 09:32:38.068143 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG uwsgi [-] config_source = [] {{(pid=88113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 09:32:38.068311 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG uwsgi [-] config_source = [] {{(pid=88114) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 09:32:38.068460 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG uwsgi [-] debug = True {{(pid=88113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 09:32:38.068632 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG uwsgi [-] debug = True {{(pid=88114) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 09:32:38.068793 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=88113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 09:32:38.070275 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=88114) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 09:32:38.070275 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=88114) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 09:32:38.070607 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=88113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 09:32:38.070727 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG uwsgi [-] insecure_debug = False {{(pid=88114) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 09:32:38.070899 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG uwsgi [-] insecure_debug = False {{(pid=88113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 09:32:38.071017 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=88114) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 09:32:38.071151 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=88113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 09:32:38.071151 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=88113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 09:32:38.071434 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=88114) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 09:32:38.071550 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG uwsgi [-] list_limit = None {{(pid=88113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 09:32:38.071721 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG uwsgi [-] list_limit = None {{(pid=88114) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 09:32:38.071829 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG uwsgi [-] log_config_append = None {{(pid=88113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 09:32:38.072004 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG uwsgi [-] log_config_append = None {{(pid=88114) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 09:32:38.072121 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=88113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 09:32:38.072303 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=88114) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 09:32:38.072411 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG uwsgi [-] log_dir = None {{(pid=88113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 09:32:38.072577 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG uwsgi [-] log_dir = None {{(pid=88114) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 09:32:38.072699 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG uwsgi [-] log_file = None {{(pid=88113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 09:32:38.072820 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG uwsgi [-] log_file = None {{(pid=88114) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 09:32:38.072941 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=88113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 09:32:38.073080 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=88114) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 09:32:38.073197 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=88113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 09:32:38.073330 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=88114) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 09:32:38.073496 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=88113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 09:32:38.073618 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=88114) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 09:32:38.073740 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=88113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 09:32:38.073915 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=88114) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 09:32:38.074024 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=88113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 09:32:38.074207 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=88114) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 09:32:38.074323 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=88113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 09:32:38.074512 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=88114) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 09:32:38.074684 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=88113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 09:32:38.074793 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=88114) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 09:32:38.075058 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=88113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 09:32:38.075168 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=88114) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 09:32:38.075286 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=88113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 09:32:38.075420 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=88114) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 09:32:38.075546 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=88113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 09:32:38.075686 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=88114) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 09:32:38.075847 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=88113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 09:32:38.075962 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=88114) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 09:32:38.076071 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=88113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 09:32:38.076197 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=88114) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 09:32:38.076197 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=88114) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 09:32:38.076438 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=88113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 09:32:38.076555 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG uwsgi [-] notification_format = cadf {{(pid=88114) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 09:32:38.076664 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG uwsgi [-] notification_format = cadf {{(pid=88113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 09:32:38.076786 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=88114) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 09:32:38.076898 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=88113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 09:32:38.077035 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.13/identity {{(pid=88114) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 09:32:38.077145 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.13/identity {{(pid=88113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 09:32:38.077268 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG uwsgi [-] publish_errors = False {{(pid=88114) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 09:32:38.077416 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG uwsgi [-] publish_errors = False {{(pid=88113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 09:32:38.077543 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=88114) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 09:32:38.077650 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=88113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 09:32:38.077767 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=88114) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 09:32:38.077889 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=88113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 09:32:38.078009 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=88114) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 09:32:38.078118 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=88113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 09:32:38.078229 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=88114) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 09:32:38.078382 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=88113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 09:32:38.078501 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=88114) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 09:32:38.078619 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=88113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 09:32:38.078727 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG uwsgi [-] standard_threads = False {{(pid=88114) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 09:32:38.078839 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG uwsgi [-] standard_threads = False {{(pid=88113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 09:32:38.078949 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG uwsgi [-] strict_password_check = False {{(pid=88114) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 09:32:38.079062 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG uwsgi [-] strict_password_check = False {{(pid=88113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 09:32:38.079238 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=88114) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 09:32:38.079356 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=88113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 09:32:38.079356 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG uwsgi [-] use_eventlog = False {{(pid=88113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 09:32:38.079573 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG uwsgi [-] use_eventlog = False {{(pid=88114) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 09:32:38.079682 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG uwsgi [-] use_journal = False {{(pid=88113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 09:32:38.079793 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG uwsgi [-] use_journal = False {{(pid=88114) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 09:32:38.079793 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG uwsgi [-] use_json = False {{(pid=88114) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 09:32:38.080012 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG uwsgi [-] use_json = False {{(pid=88113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 09:32:38.080123 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG uwsgi [-] use_stderr = False {{(pid=88114) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 09:32:38.080243 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG uwsgi [-] use_stderr = False {{(pid=88113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 09:32:38.080352 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG uwsgi [-] use_syslog = False {{(pid=88114) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 09:32:38.080976 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG uwsgi [-] use_syslog = False {{(pid=88113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 09:32:38.081117 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG uwsgi [-] watch_log_file = False {{(pid=88114) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 09:32:38.081255 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG uwsgi [-] watch_log_file = False {{(pid=88113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 09:32:38.081375 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=88114) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:38.081534 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=88113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:38.081644 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=88114) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:38.081758 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=88113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:38.081889 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=88114) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:38.082193 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=88113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:38.082305 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=88114) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:38.082431 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=88113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:38.082551 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=88114) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:38.082670 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=88113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:38.082796 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=88114) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:38.082912 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=88113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:38.083021 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=88114) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:38.083133 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=88113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:38.083254 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=88114) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:38.083369 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=88113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:38.083369 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=88113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:38.083591 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=88114) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:38.083704 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=88113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:38.083816 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=88114) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:38.083984 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=88113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:38.084092 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=88114) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:38.084092 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=88114) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:38.084323 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=88113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:38.084443 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=88114) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:38.084563 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=88113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:38.084693 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=88114) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:38.084815 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=88113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:38.084941 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=88114) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:38.085050 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=88113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:38.085160 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=88114) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:38.085272 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=88113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:38.085422 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=88114) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:38.085535 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=88113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:38.085737 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=88114) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:38.085845 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=88113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:38.085961 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=88114) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:38.086074 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=88113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:38.086187 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=88114) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:38.086187 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=88114) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:38.086420 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=88113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:38.086420 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=88113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:38.086639 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=88114) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:38.086747 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=88113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:38.086867 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=88114) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:38.086975 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=88113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:38.087098 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=88114) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:38.087206 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=88113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:38.087327 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=88114) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:38.087435 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=88113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:38.087557 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=88114) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:38.087664 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=88113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:38.087789 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=88114) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:38.087897 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=88113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:38.088009 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=88114) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:38.088125 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=88113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:38.088246 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=88114) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:38.088365 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=88113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:38.088473 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=88114) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:38.088582 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=88113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:38.088693 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=88114) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:38.088803 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=88113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:38.088914 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=88114) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:38.089036 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=88113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:38.089144 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=88114) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:38.089255 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=88113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:38.089365 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=88114) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:38.089562 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=88113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:38.089684 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=88114) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:38.089823 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=88113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:38.089935 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=88114) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:38.090050 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=88113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:38.090166 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=88114) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:38.090290 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=88113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:38.090403 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=88114) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:38.090532 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=88113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:38.090644 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=88114) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:38.090760 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=88113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:38.090884 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=88114) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:38.090996 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=88113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:38.091111 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=88114) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:38.091320 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=88113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:38.091431 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=88114) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:38.091549 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=88113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:38.091664 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=88114) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:38.091778 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=88113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:38.091894 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=88114) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:38.092024 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=88113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:38.092134 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=88114) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:38.092249 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=88113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:38.092365 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=88114) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:38.092479 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=88113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:38.092594 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=88114) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:38.092720 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=88113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:38.092832 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=88114) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:38.092984 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=88113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:38.093133 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=88114) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:38.093287 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=88113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:38.093485 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=88114) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:38.093671 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=88113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:38.093836 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=88114) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:38.093986 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=88113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:38.094150 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=88114) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:38.094317 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=88113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:38.094466 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=88114) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:38.094639 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=88113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:38.094809 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=88114) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:38.094975 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=88113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:38.095170 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=88114) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:38.095345 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=88113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:38.095514 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=88114) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:38.095656 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=88113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:38.095656 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=88113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:38.095934 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=88114) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:38.096068 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=88113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:38.096762 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=88114) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:38.096891 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=88113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:38.097032 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=88114) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:38.097148 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=88113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:38.097272 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=88114) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:38.097420 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=88113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:38.097540 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=88114) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:38.097679 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=88113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:38.097796 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=88114) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:38.097930 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=88113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:38.098057 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=88114) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:38.098186 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=88113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:38.098308 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=88114) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:38.098425 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=88113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:38.098589 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=88114) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:38.098702 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=88113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:38.098841 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=88114) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:38.099023 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=88113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:38.099194 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=88114) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:38.099309 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=88113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:38.099514 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=88114) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:38.099679 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=88113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:38.099802 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=88114) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:38.099973 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=88113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:38.100084 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=88114) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:38.100257 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=88113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:38.100367 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=88114) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:38.100552 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=88113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:38.100662 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=88114) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:38.100835 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=88113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:38.100954 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=88114) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:38.101086 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=88113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:38.101207 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=88114) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:38.101351 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=88113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:38.101558 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=88114) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:38.101673 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG uwsgi [-] cache.enabled = True {{(pid=88113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:38.101785 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG uwsgi [-] cache.enabled = True {{(pid=88114) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:38.101911 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=88113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:38.102023 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=88114) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:38.102152 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=88113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:38.102262 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=88114) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:38.102374 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=88113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:38.102497 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=88114) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:38.102653 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=88113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:38.102761 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=88114) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:38.102888 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG uwsgi [-] cache.memcache_password = {{(pid=88113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:38.102999 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG uwsgi [-] cache.memcache_password = {{(pid=88114) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:38.103110 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=88113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:38.103239 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=88114) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:38.103349 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=88113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:38.103462 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=88114) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:38.103587 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=88113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:38.103706 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=88114) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:38.103822 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=88113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:38.103934 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=88114) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:38.104060 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG uwsgi [-] cache.memcache_sasl_enabled = False {{(pid=88113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:38.104181 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG uwsgi [-] cache.memcache_sasl_enabled = False {{(pid=88114) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:38.104308 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=88113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:38.104418 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=88114) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:38.104584 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=88113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:38.104698 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=88114) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:38.104862 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG uwsgi [-] cache.memcache_username = {{(pid=88113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:38.104970 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG uwsgi [-] cache.memcache_username = {{(pid=88114) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:38.105165 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=88113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:38.105276 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=88114) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:38.105405 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=88113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:38.105519 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=88114) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:38.105632 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=88113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:38.105743 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=88114) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:38.105873 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=88113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:38.105994 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=88114) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:38.106122 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=88113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:38.106232 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=88114) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:38.106345 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=88113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:38.106345 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=88113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:38.106571 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=88114) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:38.106736 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=88113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:38.106862 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=88114) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:38.106972 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=88113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:38.107085 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=88114) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:38.107198 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=88113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:38.107310 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=88114) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:38.107424 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=88113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:38.107549 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=88114) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:38.107659 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=88113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:38.107773 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=88114) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:38.107885 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=88113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:38.107997 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=88114) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:38.108110 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=88113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:38.108239 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=88114) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:38.108349 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=88113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:38.108462 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=88114) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:38.108575 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=88113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:38.108688 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=88114) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:38.108813 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=88113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:38.108922 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=88114) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:38.109034 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=88113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:38.109146 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=88114) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:38.109292 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG uwsgi [-] auth.external = None {{(pid=88113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:38.109462 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=88114) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:38.109575 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG uwsgi [-] auth.mapped = None {{(pid=88113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:38.109707 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG uwsgi [-] auth.external = None {{(pid=88114) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:38.109829 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=88113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:38.109958 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG uwsgi [-] auth.mapped = None {{(pid=88114) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:38.110073 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=88113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:38.110187 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=88114) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:38.110302 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG uwsgi [-] auth.password = None {{(pid=88113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:38.110415 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=88114) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:38.110540 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG uwsgi [-] auth.token = None {{(pid=88113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:38.110654 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG uwsgi [-] auth.password = None {{(pid=88114) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:38.110769 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=88113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:38.110882 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG uwsgi [-] auth.token = None {{(pid=88114) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:38.111007 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG uwsgi [-] catalog.caching = True {{(pid=88113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:38.111210 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=88114) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:38.111378 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=88113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:38.111489 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG uwsgi [-] catalog.caching = True {{(pid=88114) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:38.111621 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=88113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:38.111732 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=88114) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:38.111848 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=88113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:38.111963 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=88114) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:38.112090 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=88113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:38.112202 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=88114) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:38.112315 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=88113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:38.112429 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=88114) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:38.112542 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG uwsgi [-] credential.caching = True {{(pid=88113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:38.112655 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=88114) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:38.112788 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG uwsgi [-] credential.driver = sql {{(pid=88113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:38.112899 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG uwsgi [-] credential.caching = True {{(pid=88114) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:38.113014 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=88113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:38.113133 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG uwsgi [-] credential.driver = sql {{(pid=88114) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:38.113246 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=88113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:38.113359 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=88114) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:38.113547 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=88113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:38.113547 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=88113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:38.113759 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=88114) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:38.113872 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=88113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:38.113985 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=88114) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:38.114098 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=88113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:38.114225 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=88114) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:38.114336 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=88113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:38.114450 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=88114) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:38.114575 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=88113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:38.114688 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=88114) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:38.114807 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=88113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:38.114976 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=88114) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:38.115084 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=88113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:38.115201 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=88114) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:38.115315 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=88113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:38.115429 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=88114) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:38.115541 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=88113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:38.115714 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=88114) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:38.115824 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=88113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:38.115940 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=88114) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:38.116054 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=88113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:38.116167 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=88114) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:38.116278 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG uwsgi [-] federation.caching = True {{(pid=88113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:38.116405 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=88114) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:38.116519 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=88113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:38.116633 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=88114) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:38.116745 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG uwsgi [-] federation.driver = sql {{(pid=88113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:38.116860 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG uwsgi [-] federation.caching = True {{(pid=88114) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:38.116973 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=88113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:38.117098 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=88114) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:38.117216 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=88113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:38.117329 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG uwsgi [-] federation.driver = sql {{(pid=88114) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:38.117516 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=88113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:38.117627 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=88114) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:38.117741 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=88113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:38.117873 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=88114) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:38.117986 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=88113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:38.118113 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=88114) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:38.118226 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=88113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:38.118340 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=88114) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:38.118453 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=88113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:38.118579 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=88114) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:38.118691 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=88113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:38.118805 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=88114) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:38.118918 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=88113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:38.119031 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=88114) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:38.119145 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG uwsgi [-] identity.caching = True {{(pid=88113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:38.119344 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=88114) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:38.119456 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=88113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:38.119572 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=88114) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:38.119685 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=88113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:38.119807 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG uwsgi [-] identity.caching = True {{(pid=88114) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:38.119933 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=88113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:38.120044 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=88114) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:38.120044 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=88114) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:38.120259 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=88113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:38.120372 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=88114) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:38.120487 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG uwsgi [-] identity.driver = sql {{(pid=88113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:38.120610 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=88114) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:38.120722 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=88113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:38.120722 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=88113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:38.120945 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG uwsgi [-] identity.driver = sql {{(pid=88114) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:38.120945 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=88114) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:38.121164 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=88113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:38.121292 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=88114) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:38.121424 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=88113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:38.121546 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=88114) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:38.121656 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=88113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:38.121783 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=88114) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:38.121897 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=88113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:38.122029 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=88114) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:38.122141 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=88113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:38.122256 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=88114) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:38.122381 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=88113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:38.122495 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=88114) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:38.122611 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=88113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:38.122739 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=88114) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:38.122852 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=88113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:38.122852 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=88113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:38.123087 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=88114) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:38.123209 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=88113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:38.123322 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=88114) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:38.123451 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=88113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:38.123563 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=88114) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:38.123679 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=88113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:38.123793 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=88114) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:38.123908 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=88113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:38.123908 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=88113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:38.124134 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=88114) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:38.124246 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=88113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:38.124359 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=88114) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:38.124481 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=88113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:38.124595 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=88114) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:38.124711 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=88113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:38.124838 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=88114) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:38.125010 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=88113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:38.125121 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=88114) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:38.125241 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=88113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:38.125368 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=88114) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:38.125565 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=88113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:38.125703 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=88114) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:38.125816 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=88113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:38.125930 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=88114) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:38.126045 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=88113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:38.126159 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=88114) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:38.126278 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=88113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:38.126405 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=88114) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:38.126517 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=88113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:38.126632 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=88114) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:38.126757 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=88113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:38.126873 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=88114) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:38.126991 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=88113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:38.127120 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=88114) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:38.127234 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=88113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:38.127350 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=88114) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:38.127465 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=88113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:38.127585 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=88114) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:38.127702 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG uwsgi [-] ldap.password = **** {{(pid=88113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:38.127836 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=88114) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:38.127949 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=88113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:38.128065 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=88114) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:38.128183 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=88113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:38.128300 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=88114) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:38.128416 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=88113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:38.128544 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG uwsgi [-] ldap.password = **** {{(pid=88114) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:38.128658 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=88113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:38.128777 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=88114) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:38.128908 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=88113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:38.129021 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=88114) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:38.129138 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=88113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:38.129267 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=88114) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:38.129400 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=88113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:38.129516 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=88114) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:38.129638 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=88113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:38.129758 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=88114) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:38.129879 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=88113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:38.130016 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=88114) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:38.130132 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=88113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:38.130254 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=88114) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:38.130374 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=88113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:38.130495 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=88114) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:38.130615 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=88113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:38.130748 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=88114) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:38.130866 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=88113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:38.130986 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=88114) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:38.131113 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=88113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:38.131237 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=88114) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:38.131357 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG uwsgi [-] ldap.user = None {{(pid=88113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:38.131491 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=88114) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:38.131609 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=88113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:38.131729 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=88114) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:38.131849 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=88113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:38.131969 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=88114) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:38.132103 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=88113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:38.132220 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG uwsgi [-] ldap.user = None {{(pid=88114) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:38.132341 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=88113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:38.132461 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=88114) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:38.132582 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=88113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:38.132712 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=88114) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:38.132847 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=88113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:38.132963 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=88114) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:38.133084 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=88113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:38.133204 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=88114) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:38.133324 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=88113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:38.133480 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=88114) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:38.133611 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=88113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:38.133725 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=88114) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:38.133842 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=88113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:38.133958 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=88114) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:38.134076 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=88113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:38.134192 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=88114) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:38.134321 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=88113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:38.134435 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=88114) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:38.134598 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=88113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:38.134711 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=88114) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:38.134829 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=88113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:38.134947 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=88114) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:38.135072 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=88113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:38.135187 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=88114) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:38.135304 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=88113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:38.135430 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=88114) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:38.135545 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=88113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:38.135662 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=88114) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:38.135793 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=88113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:38.135906 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=88114) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:38.136023 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=88113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:38.136139 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=88114) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:38.136263 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=88113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:38.136379 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=88114) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:38.136508 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=88113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:38.136623 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=88114) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:38.136739 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=88113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:38.136893 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=88114) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:38.137003 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=88113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:38.137123 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=88114) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:38.137253 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=88113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:38.137368 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=88114) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:38.137515 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=88113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:38.137638 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=88114) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:38.137758 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=88113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:38.137883 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=88114) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:38.138019 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG uwsgi [-] policy.driver = sql {{(pid=88113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:38.138135 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=88114) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:38.138257 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=88113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:38.138379 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=88114) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:38.138497 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=88113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:38.138619 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=88114) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:38.138750 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=88113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:38.138868 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG uwsgi [-] policy.driver = sql {{(pid=88114) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:38.138988 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG uwsgi [-] receipt.caching = True {{(pid=88113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:38.139108 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=88114) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:38.139230 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=88113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:38.139349 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=88114) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:38.139483 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=88113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:38.139603 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=88114) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:38.139723 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=88113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:38.139844 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG uwsgi [-] receipt.caching = True {{(pid=88114) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:38.139964 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=88113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:38.140093 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=88114) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:38.140223 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=88113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:38.140223 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG uwsgi [-] resource.caching = True {{(pid=88113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:38.140446 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=88114) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:38.140564 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=88113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:38.140685 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=88114) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:38.140805 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG uwsgi [-] resource.driver = sql {{(pid=88113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:38.140945 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=88114) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:38.141065 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=88113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:38.141185 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=88114) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:38.141310 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=88113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:38.141486 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG uwsgi [-] resource.caching = True {{(pid=88114) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:38.141599 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=88113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:38.141736 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=88114) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:38.141848 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG uwsgi [-] revoke.caching = True {{(pid=88113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:38.141965 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG uwsgi [-] resource.driver = sql {{(pid=88114) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:38.142082 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=88113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:38.142200 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=88114) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:38.142316 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=88113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:38.142520 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=88114) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:38.142632 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG uwsgi [-] role.cache_time = None {{(pid=88113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:38.142751 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=88114) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:38.142869 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG uwsgi [-] role.caching = True {{(pid=88113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:38.143491 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG uwsgi [-] revoke.caching = True {{(pid=88114) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:38.143609 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG uwsgi [-] role.driver = sql {{(pid=88113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:38.143739 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=88114) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:38.143852 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG uwsgi [-] role.list_limit = None {{(pid=88113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:38.143968 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=88114) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:38.144084 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=88113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:38.144199 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG uwsgi [-] role.cache_time = None {{(pid=88114) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:38.144316 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=88113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:38.144446 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG uwsgi [-] role.caching = True {{(pid=88114) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:38.144559 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=88113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:38.144675 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG uwsgi [-] role.driver = sql {{(pid=88114) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:38.144792 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=88113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:38.144908 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG uwsgi [-] role.list_limit = None {{(pid=88114) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:38.145025 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=88113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:38.145150 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=88114) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:38.145274 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=88113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:38.145657 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=88114) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:38.145788 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=88113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:38.145910 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=88114) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:38.146048 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=88113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:38.146164 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=88114) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:38.146286 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=88113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:38.146286 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=88113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:38.146502 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=88114) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:38.146620 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=88113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:38.146755 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=88114) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:38.146872 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=88113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:38.146999 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=88114) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:38.147126 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=88113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:38.147238 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=88114) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:38.147238 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=88114) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:38.147469 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=88113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:38.147595 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=88114) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:38.147717 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=88113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:38.147839 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=88114) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:38.147962 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=88113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:38.148085 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=88114) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:38.148225 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=88113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:38.148358 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=88114) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:38.148487 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=88113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:38.148487 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=88113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:38.148710 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=88114) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:38.148829 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=88113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:38.148964 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=88114) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:38.149084 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=88113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:38.149205 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=88114) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:38.149326 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=88113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:38.149536 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=88114) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:38.149664 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=88113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:38.149804 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=88114) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:38.149917 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=88113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:38.150037 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=88114) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:38.150155 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=88113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:38.150272 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=88114) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:38.150396 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=88113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:38.150540 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=88114) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:38.150661 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=88113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:38.150780 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=88114) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:38.150897 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=88113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:38.151021 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=88114) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:38.151150 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=88113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:38.151281 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=88114) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:38.151403 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=88113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:38.151535 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=88114) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:38.151662 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=88113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:38.151779 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=88114) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:38.151924 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG uwsgi [-] token.cache_time = None {{(pid=88113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:38.152050 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=88114) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:38.152166 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG uwsgi [-] token.caching = True {{(pid=88113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:38.152283 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=88114) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:38.152399 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=88113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:38.152516 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=88114) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:38.152644 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG uwsgi [-] token.provider = fernet {{(pid=88113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:38.152758 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=88114) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:38.152876 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=88113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:38.152993 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=88114) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:38.153110 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=88113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:38.153227 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG uwsgi [-] token.cache_time = None {{(pid=88114) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:38.153365 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=88113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:38.153518 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG uwsgi [-] token.caching = True {{(pid=88114) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:38.153638 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=88113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:38.153762 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=88114) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:38.153879 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=88113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:38.153999 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG uwsgi [-] token.provider = fernet {{(pid=88114) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:38.154136 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=88113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:38.154250 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=88114) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:38.154370 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG uwsgi [-] trust.driver = sql {{(pid=88113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:38.154370 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=88113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:38.154640 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=88114) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:38.154748 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=88113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:38.154880 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=88114) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:38.155012 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=88113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:38.155138 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=88114) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:38.155251 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=88113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:38.155380 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=88114) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:38.155380 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=88114) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:38.155597 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=88113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:38.155709 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG uwsgi [-] trust.driver = sql {{(pid=88114) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:38.155833 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=88113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:38.155977 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=88114) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:38.156118 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=88113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:38.156235 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=88114) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:38.156357 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=88113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:38.156488 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=88114) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:38.156612 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG uwsgi [-] database.connection = **** {{(pid=88113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:38.156725 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=88114) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:38.156850 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=88113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:38.156972 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=88114) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:38.156972 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=88114) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:38.157215 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=88113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:38.157361 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=88114) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:38.157563 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=88113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:38.157676 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=88114) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:38.157799 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=88113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:38.157930 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG uwsgi [-] database.connection = **** {{(pid=88114) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:38.158050 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=88113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:38.158168 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=88114) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:38.158313 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=88113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:38.158440 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=88114) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:38.158560 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=88113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:38.158722 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=88114) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:38.159347 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=88113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:38.159466 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=88114) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:38.159599 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=88113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:38.159717 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=88114) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:38.159832 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=88113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:38.159945 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=88114) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:38.160057 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=88113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:38.160184 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=88114) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:38.160184 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=88114) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:38.160424 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=88113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:38.160548 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=88114) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:38.160670 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=88113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:38.160792 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=88114) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:38.160913 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=88113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:38.161051 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=88114) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:38.161172 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=88113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:38.161298 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=88114) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:38.161456 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=88113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:38.161456 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=88113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:38.161664 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=88114) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:38.161795 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=88113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:38.161795 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=88113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Aug 31 09:32:38.162012 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=88114) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:38.162012 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=88114) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:38.162012 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=88114) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:38.162012 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=88114) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:38.162012 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=88114) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 09:32:38.162012 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=88114) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Aug 31 09:32:38.186208 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=88113) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 09:32:38.186441 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=88113) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 09:32:38.186723 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=88113) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 09:32:38.186969 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=88114) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 09:32:38.186969 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=88114) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 09:32:38.187229 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=88114) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 09:32:38.196267 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=88113) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 09:32:38.197241 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=88114) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 09:32:38.226143 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=88113) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 09:32:38.226436 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=88113) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 09:32:38.226595 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=88114) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 09:32:38.226788 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=88114) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 09:32:38.281038 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=88113) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 09:32:38.281477 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=88114) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 09:32:38.331798 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=88113) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 09:32:38.332134 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=88113) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 09:32:38.332826 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=88114) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 09:32:38.333065 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=88114) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 09:32:38.492374 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=88113) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 09:32:38.492696 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=88114) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 09:32:38.492901 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=88113) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 09:32:38.493068 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=88114) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 09:32:38.493228 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=88113) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 09:32:38.493418 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=88114) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 09:32:38.493592 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=88113) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 09:32:38.493750 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=88114) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 09:32:38.493904 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=88113) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 09:32:38.494056 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=88114) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 09:32:38.494208 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=88113) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 09:32:38.494371 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=88114) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 09:32:38.517004 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=88113) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 09:32:38.517292 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=88113) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 09:32:38.517494 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=88114) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 09:32:38.517658 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=88113) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 09:32:38.517815 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=88114) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 09:32:38.518007 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=88113) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 09:32:38.518160 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=88114) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 09:32:38.518994 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=88113) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 09:32:38.519227 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=88114) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 09:32:38.519227 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=88114) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 09:32:38.519535 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=88113) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 09:32:38.519718 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=88114) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 09:32:38.519871 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=88113) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 09:32:38.520021 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=88114) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 09:32:38.530517 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=88113) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 09:32:38.530870 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=88114) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 09:32:38.544373 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=88113) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 09:32:38.544638 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=88114) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 09:32:38.544796 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=88113) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 09:32:38.544949 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=88114) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 09:32:38.545102 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=88113) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 09:32:38.545271 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=88114) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 09:32:38.545467 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=88113) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 09:32:38.545626 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=88114) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 09:32:38.545778 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=88113) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 09:32:38.545930 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=88114) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 09:32:38.546096 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=88113) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 09:32:38.546247 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=88114) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 09:32:38.546397 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=88113) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 09:32:38.546548 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=88114) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 09:32:38.563648 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=88113) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 09:32:38.563965 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=88113) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 31 09:32:38.564189 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=88113) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 09:32:38.564341 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=88114) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 09:32:38.564494 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=88113) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 31 09:32:38.564644 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=88114) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 31 09:32:38.564811 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=88113) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 09:32:38.564973 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=88114) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 09:32:38.565316 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=88113) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 09:32:38.565579 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=88114) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 31 09:32:38.565579 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=88114) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 09:32:38.565799 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=88113) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 09:32:38.565934 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=88114) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 09:32:38.566077 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=88113) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 09:32:38.566165 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=88114) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 09:32:38.566165 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=88114) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 09:32:38.575356 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=88113) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 09:32:38.575639 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=88113) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 09:32:38.575840 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=88114) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 09:32:38.576005 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=88113) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 09:32:38.576160 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=88114) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 09:32:38.576312 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=88113) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 09:32:38.576466 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=88114) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 09:32:38.576627 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=88114) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 09:32:38.583734 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=88113) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 31 09:32:38.584257 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=88114) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 31 09:32:38.586637 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=88113) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 31 09:32:38.586926 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=88113) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 09:32:38.587183 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=88114) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 31 09:32:38.587337 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=88113) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 09:32:38.587488 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=88114) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 09:32:38.587637 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=88113) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 09:32:38.587749 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=88113) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 09:32:38.587961 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=88114) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 09:32:38.588120 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=88113) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 09:32:38.588267 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=88114) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 09:32:38.588424 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=88113) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 09:32:38.588573 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=88114) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 09:32:38.588720 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=88113) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 09:32:38.588875 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=88114) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 09:32:38.588875 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=88114) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 09:32:38.589157 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=88113) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 09:32:38.589319 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=88114) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 09:32:38.589452 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=88113) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 09:32:38.589597 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=88114) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 09:32:38.589710 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=88113) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 09:32:38.589830 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=88114) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 09:32:38.589830 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=88114) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 09:32:38.612560 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=88113) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 09:32:38.613767 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=88114) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 09:32:38.614053 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=88113) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 31 09:32:38.614335 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=88113) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 09:32:38.615256 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=88114) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 31 09:32:38.615545 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=88114) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 09:32:38.618976 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=88113) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 31 09:32:38.619261 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=88113) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 09:32:38.619540 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=88113) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 09:32:38.620089 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=88114) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 31 09:32:38.620378 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=88114) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 09:32:38.620661 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=88114) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 09:32:38.625840 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=88113) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 31 09:32:38.626126 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=88113) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 09:32:38.627467 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=88114) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 31 09:32:38.627867 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=88114) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 09:32:38.630256 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=88113) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 31 09:32:38.630547 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=88113) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 09:32:38.630829 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=88113) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 09:32:38.631108 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=88113) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 09:32:38.631380 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=88113) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 09:32:38.631648 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=88113) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 09:32:38.631912 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=88113) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 09:32:38.632040 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=88114) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 31 09:32:38.632161 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=88113) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 09:32:38.632284 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=88114) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 09:32:38.632484 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=88114) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 09:32:38.632770 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=88114) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 09:32:38.633052 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=88114) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 09:32:38.635283 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=88114) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 09:32:38.635573 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=88114) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 09:32:38.635853 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=88114) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 09:32:38.650927 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=88113) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 09:32:38.651219 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=88113) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 09:32:38.652520 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=88114) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 09:32:38.652827 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=88114) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 09:32:38.655173 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=88113) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 31 09:32:38.656815 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=88114) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 31 09:32:38.658471 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=88113) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 09:32:38.659457 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=88113) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 09:32:38.660539 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=88114) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 09:32:38.660838 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=88114) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 09:32:38.663743 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=88113) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 31 09:32:38.665202 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=88114) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 31 09:32:38.667010 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=88113) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 31 09:32:38.668365 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=88114) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 31 09:32:38.670292 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=88113) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 09:32:38.670602 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=88113) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 09:32:38.670886 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=88113) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 09:32:38.671168 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=88113) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 09:32:38.671452 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=88113) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 09:32:38.671567 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=88114) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 09:32:38.671727 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=88113) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 09:32:38.671851 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=88114) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 09:32:38.672053 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=88114) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 09:32:38.672344 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=88114) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 09:32:38.672631 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=88114) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 09:32:38.673022 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=88114) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 09:32:38.687698 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=88113) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 31 09:32:38.688029 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=88113) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 09:32:38.688300 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=88113) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 09:32:38.688562 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=88113) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 09:32:38.691989 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=88114) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 31 09:32:38.692436 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=88114) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 09:32:38.692797 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=88114) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 09:32:38.693158 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=88114) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 09:32:38.695692 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH2: ['/token' {}] {{(pid=88113) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 09:32:38.697273 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=88113) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 09:32:38.698830 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=88113) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 09:32:38.699107 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=88113) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 09:32:38.701802 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=88113) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 31 09:32:38.702094 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=88113) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 09:32:38.702505 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH2: ['/token' {}] {{(pid=88114) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 09:32:38.702629 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=88113) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 09:32:38.702786 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=88113) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 09:32:38.703154 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=88113) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 09:32:38.704210 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=88114) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 09:32:38.706363 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=88114) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 09:32:38.706645 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=88114) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 09:32:38.709427 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=88114) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 31 09:32:38.709760 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=88114) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 09:32:38.710056 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=88114) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 09:32:38.710344 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=88114) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 09:32:38.710639 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=88114) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 09:32:38.715319 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=88113) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 31 09:32:38.715614 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=88113) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 09:32:38.716003 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=88113) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 09:32:38.716285 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=88113) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 09:32:38.716567 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=88113) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 09:32:38.716848 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=88113) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 09:32:38.717143 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=88113) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 09:32:38.723039 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=88114) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 31 09:32:38.723340 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=88114) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 09:32:38.723629 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=88114) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 09:32:38.723915 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=88114) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 09:32:38.724197 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=88114) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 09:32:38.724479 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=88114) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 09:32:38.724783 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=88114) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 09:32:38.733277 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=88113) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 31 09:32:38.736209 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=88113) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 31 09:32:38.739864 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=88113) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 09:32:38.741142 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=88114) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 31 09:32:38.741318 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=88113) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 09:32:38.742973 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=88113) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 31 09:32:38.743208 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=88113) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 09:32:38.743496 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=88113) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 09:32:38.744284 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=88114) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 31 09:32:38.747211 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=88114) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 09:32:38.748649 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=88114) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 09:32:38.749970 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=88113) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 09:32:38.750155 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=88114) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 31 09:32:38.750440 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=88114) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 09:32:38.751083 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=88114) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 09:32:38.751441 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=88113) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 31 09:32:38.755285 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=88113) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 09:32:38.755606 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=88113) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 09:32:38.755935 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=88113) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 09:32:38.756213 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=88113) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 09:32:38.758284 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=88114) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 09:32:38.759796 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=88114) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 31 09:32:38.762743 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=88114) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 09:32:38.763067 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=88114) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 09:32:38.763339 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=88114) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 09:32:38.763620 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=88114) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 09:32:38.764308 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=88113) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 31 09:32:38.764609 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=88113) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 09:32:38.764886 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=88113) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 09:32:38.772060 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=88114) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 31 09:32:38.772268 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=88113) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 31 09:32:38.772428 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=88114) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 09:32:38.772583 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=88113) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 09:32:38.772736 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=88114) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 09:32:38.772888 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=88113) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 09:32:38.773052 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=88113) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 09:32:38.773204 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=88113) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 09:32:38.773474 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=88113) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 09:32:38.773742 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=88113) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 09:32:38.774014 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=88113) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 09:32:38.774290 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=88113) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 09:32:38.774563 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=88113) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 09:32:38.774828 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=88113) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 09:32:38.775085 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=88113) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 09:32:38.775346 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=88113) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 09:32:38.775622 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=88113) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 09:32:38.779657 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=88114) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 31 09:32:38.779955 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=88114) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 09:32:38.780241 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=88114) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 09:32:38.780523 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=88114) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 09:32:38.780799 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=88114) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 09:32:38.781081 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=88114) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 09:32:38.781357 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=88114) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 09:32:38.781661 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=88114) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 09:32:38.781944 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=88114) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 09:32:38.782225 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=88114) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 09:32:38.782518 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=88114) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 09:32:38.782760 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=88114) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 09:32:38.783037 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=88114) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 09:32:38.783319 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=88114) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 09:32:38.810731 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x55c3727c09c0 pid: 88113 (default app) Aug 31 09:32:38.814251 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec52a193-f503-44f1-ba8f-52db76c9ba1a None None] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:32:38.814534 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec52a193-f503-44f1-ba8f-52db76c9ba1a None None] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:32:38.814972 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec52a193-f503-44f1-ba8f-52db76c9ba1a None None] PATH_INFO: `/v3` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:32:38.819376 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x55c3727c09c0 pid: 88114 (default app) Aug 31 09:32:38.819911 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 1/1] 10.222.0.13 () {50 vars in 791 bytes} [Wed Aug 31 09:32:38 2022] GET /identity/v3/ => generated 255 bytes in 9 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:32:45.601896 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f702e3fc-c791-480a-a73e-9bce7191fde2 None None] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:32:45.602546 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f702e3fc-c791-480a-a73e-9bce7191fde2 None None] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:32:45.602546 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f702e3fc-c791-480a-a73e-9bce7191fde2 None None] PATH_INFO: `/` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:32:45.607078 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 1/2] 10.222.0.13 () {54 vars in 927 bytes} [Wed Aug 31 09:32:45 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 31 09:32:45.613342 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a02c94d-2df8-4976-9ee3-92061d70e545 None None] REQUEST_METHOD: `POST` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:32:45.614087 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a02c94d-2df8-4976-9ee3-92061d70e545 None None] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:32:45.614087 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a02c94d-2df8-4976-9ee3-92061d70e545 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:32:45.689175 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: INFO dbcounter [None req-7a02c94d-2df8-4976-9ee3-92061d70e545 None None] Registered counter for database keystone Aug 31 09:32:45.713231 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG oslo_db.sqlalchemy.engines [None req-7a02c94d-2df8-4976-9ee3-92061d70e545 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=88113) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Aug 31 09:32:45.720783 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writer thread running {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Aug 31 09:32:45.913992 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG passlib.handlers.bcrypt [None req-7a02c94d-2df8-4976-9ee3-92061d70e545 None None] detected 'bcrypt' backend, version '4.0.0' {{(pid=88113) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Aug 31 09:32:45.914306 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG passlib.handlers.bcrypt [None req-7a02c94d-2df8-4976-9ee3-92061d70e545 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=88113) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Aug 31 09:32:45.976225 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.auth.core [None req-7a02c94d-2df8-4976-9ee3-92061d70e545 None None] MFA Rules not processed for user `fd836cf590a145df9acf19cf3ae2ac56`. Rule list: `[]` (Enabled: `True`). {{(pid=88113) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:32:45.978605 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.fernet_utils [None req-7a02c94d-2df8-4976-9ee3-92061d70e545 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:32:46.121558 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 2/3] 10.222.0.13 () {58 vars in 1024 bytes} [Wed Aug 31 09:32:45 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 509 msecs (HTTP/1.1 201) 6 headers in 384 bytes (2 switches on core 0) Aug 31 09:32:46.556246 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f702e3fc-c791-480a-a73e-9bce7191fde2 None None] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:32:46.626628 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: INFO dbcounter [None req-f702e3fc-c791-480a-a73e-9bce7191fde2 None None] Registered counter for database keystone Aug 31 09:32:46.648833 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG oslo_db.sqlalchemy.engines [None req-f702e3fc-c791-480a-a73e-9bce7191fde2 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=88114) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Aug 31 09:32:46.655342 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writer thread running {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Aug 31 09:32:46.786497 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f702e3fc-c791-480a-a73e-9bce7191fde2 None None] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:32:46.790816 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a16d0bff-5bc7-49fc-adbf-e8ddc6a4cb49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:32:46.791451 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16d0bff-5bc7-49fc-adbf-e8ddc6a4cb49 admin admin] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:32:46.791725 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16d0bff-5bc7-49fc-adbf-e8ddc6a4cb49 admin admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:32:46.791997 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16d0bff-5bc7-49fc-adbf-e8ddc6a4cb49 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:32:46.802064 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a16d0bff-5bc7-49fc-adbf-e8ddc6a4cb49 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:32:46.855451 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: WARNING keystone.common.rbac_enforcer.enforcer [None req-a16d0bff-5bc7-49fc-adbf-e8ddc6a4cb49 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Aug 31 09:32:46.855718 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a16d0bff-5bc7-49fc-adbf-e8ddc6a4cb49 admin admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:32:46.866998 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 2/4] 10.222.0.13 () {58 vars in 1324 bytes} [Wed Aug 31 09:32:46 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 31 09:32:46.872947 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a02c94d-2df8-4976-9ee3-92061d70e545 None None] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:32:46.888215 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a02c94d-2df8-4976-9ee3-92061d70e545 None None] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:32:46.892641 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc42ad28-03ef-46b8-837a-fdc44f5c7acd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:32:46.894288 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc42ad28-03ef-46b8-837a-fdc44f5c7acd admin admin] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:32:46.894566 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc42ad28-03ef-46b8-837a-fdc44f5c7acd admin admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:32:46.894836 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc42ad28-03ef-46b8-837a-fdc44f5c7acd admin admin] PATH_INFO: `/v3/projects/3f187a2ac46b4c229b4eeead01522f05` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:32:46.896672 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc42ad28-03ef-46b8-837a-fdc44f5c7acd admin admin] RBAC: Authorizing `identity:get_project(project_id=3f187a2ac46b4c229b4eeead01522f05)` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:32:46.949211 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: WARNING keystone.common.rbac_enforcer.enforcer [None req-cc42ad28-03ef-46b8-837a-fdc44f5c7acd admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Aug 31 09:32:46.949576 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc42ad28-03ef-46b8-837a-fdc44f5c7acd admin admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:32:46.954298 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 3/5] 10.222.0.13 () {56 vars in 1207 bytes} [Wed Aug 31 09:32:46 2022] GET /identity/v3/projects/3f187a2ac46b4c229b4eeead01522f05 => generated 344 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:32:48.882571 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6f7ca0-712d-4bad-918c-b8e8e947e357 None None] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:32:48.883150 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6f7ca0-712d-4bad-918c-b8e8e947e357 None None] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:32:48.883150 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6f7ca0-712d-4bad-918c-b8e8e947e357 None None] PATH_INFO: `/` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:32:48.887406 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 3/6] 10.222.0.13 () {54 vars in 927 bytes} [Wed Aug 31 09:32:48 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:32:48.894462 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf467ba-8a5b-4e29-8719-c494eced6356 None None] REQUEST_METHOD: `POST` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:32:48.895272 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf467ba-8a5b-4e29-8719-c494eced6356 None None] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:32:48.895679 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf467ba-8a5b-4e29-8719-c494eced6356 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:32:48.917341 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.auth.core [None req-0bf467ba-8a5b-4e29-8719-c494eced6356 None None] MFA Rules not processed for user `fd836cf590a145df9acf19cf3ae2ac56`. Rule list: `[]` (Enabled: `True`). {{(pid=88113) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:32:48.919015 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.fernet_utils [None req-0bf467ba-8a5b-4e29-8719-c494eced6356 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:32:48.927503 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 4/7] 10.222.0.13 () {58 vars in 1024 bytes} [Wed Aug 31 09:32:48 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Aug 31 09:32:49.198752 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c6f7ca0-712d-4bad-918c-b8e8e947e357 None None] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:32:49.212721 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c6f7ca0-712d-4bad-918c-b8e8e947e357 None None] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:32:49.216943 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-212e8483-5dca-44ee-bfd9-67be46bb1ec0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:32:49.217599 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212e8483-5dca-44ee-bfd9-67be46bb1ec0 admin admin] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:32:49.217876 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212e8483-5dca-44ee-bfd9-67be46bb1ec0 admin admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:32:49.218144 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212e8483-5dca-44ee-bfd9-67be46bb1ec0 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:32:49.226638 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-212e8483-5dca-44ee-bfd9-67be46bb1ec0 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:32:49.228412 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-212e8483-5dca-44ee-bfd9-67be46bb1ec0 admin admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:32:49.237897 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 4/8] 10.222.0.13 () {58 vars in 1324 bytes} [Wed Aug 31 09:32:49 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 31 09:32:49.243592 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf467ba-8a5b-4e29-8719-c494eced6356 None None] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:32:49.253429 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf467ba-8a5b-4e29-8719-c494eced6356 None None] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:32:49.257926 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bdc48bc-32f8-470e-be1b-90a6cdb60084 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:32:49.258566 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bdc48bc-32f8-470e-be1b-90a6cdb60084 admin admin] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:32:49.258843 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bdc48bc-32f8-470e-be1b-90a6cdb60084 admin admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:32:49.259113 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bdc48bc-32f8-470e-be1b-90a6cdb60084 admin admin] PATH_INFO: `/v3/users/fd836cf590a145df9acf19cf3ae2ac56` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:32:49.260439 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bdc48bc-32f8-470e-be1b-90a6cdb60084 admin admin] RBAC: Authorizing `identity:get_user(user_id=fd836cf590a145df9acf19cf3ae2ac56)` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:32:49.262330 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bdc48bc-32f8-470e-be1b-90a6cdb60084 admin admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:32:49.263961 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 5/9] 10.222.0.13 () {56 vars in 1198 bytes} [Wed Aug 31 09:32:49 2022] GET /identity/v3/users/fd836cf590a145df9acf19cf3ae2ac56 => generated 247 bytes in 21 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:32:51.334130 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60cab0ab-482c-4b34-a7c6-bba360998be0 None None] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:32:51.334678 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60cab0ab-482c-4b34-a7c6-bba360998be0 None None] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:32:51.334678 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60cab0ab-482c-4b34-a7c6-bba360998be0 None None] PATH_INFO: `/` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:32:51.335938 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 5/10] 10.222.0.13 () {54 vars in 927 bytes} [Wed Aug 31 09:32:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 31 09:32:51.338043 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db56e24-8a3a-4825-9415-2cf14971eec8 None None] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:32:51.338708 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db56e24-8a3a-4825-9415-2cf14971eec8 None None] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:32:51.338945 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db56e24-8a3a-4825-9415-2cf14971eec8 None None] PATH_INFO: `/` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:32:51.340919 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 6/11] 10.222.0.13 () {54 vars in 927 bytes} [Wed Aug 31 09:32:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:32:51.341656 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9655e32e-3a6e-4794-be76-f32cec4033d5 None None] REQUEST_METHOD: `POST` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:32:51.342057 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9655e32e-3a6e-4794-be76-f32cec4033d5 None None] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:32:51.342417 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9655e32e-3a6e-4794-be76-f32cec4033d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:32:51.346650 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe1d24b-94ed-4b55-97cc-d7b81dd67c6b None None] REQUEST_METHOD: `POST` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:32:51.347041 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe1d24b-94ed-4b55-97cc-d7b81dd67c6b None None] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:32:51.347440 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe1d24b-94ed-4b55-97cc-d7b81dd67c6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:32:51.366880 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.auth.core [None req-9fe1d24b-94ed-4b55-97cc-d7b81dd67c6b None None] MFA Rules not processed for user `fd836cf590a145df9acf19cf3ae2ac56`. Rule list: `[]` (Enabled: `True`). {{(pid=88113) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:32:51.368090 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.fernet_utils [None req-9fe1d24b-94ed-4b55-97cc-d7b81dd67c6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:32:51.388027 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG passlib.handlers.bcrypt [None req-9655e32e-3a6e-4794-be76-f32cec4033d5 None None] detected 'bcrypt' backend, version '4.0.0' {{(pid=88114) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Aug 31 09:32:51.388421 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG passlib.handlers.bcrypt [None req-9655e32e-3a6e-4794-be76-f32cec4033d5 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=88114) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Aug 31 09:32:51.438560 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.auth.core [None req-9655e32e-3a6e-4794-be76-f32cec4033d5 None None] MFA Rules not processed for user `fd836cf590a145df9acf19cf3ae2ac56`. Rule list: `[]` (Enabled: `True`). {{(pid=88114) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:32:51.440475 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.fernet_utils [None req-9655e32e-3a6e-4794-be76-f32cec4033d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88114) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:32:51.458602 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 7/12] 10.222.0.13 () {58 vars in 1024 bytes} [Wed Aug 31 09:32:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 31 09:32:51.574431 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 6/13] 10.222.0.13 () {58 vars in 1024 bytes} [Wed Aug 31 09:32:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 234 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 09:32:51.751335 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe1d24b-94ed-4b55-97cc-d7b81dd67c6b None None] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:32:51.778092 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=100 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:32:51.853690 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9655e32e-3a6e-4794-be76-f32cec4033d5 None None] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:32:51.872844 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe1d24b-94ed-4b55-97cc-d7b81dd67c6b None None] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:32:51.910671 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea73852c-adc2-46ec-b667-6531d2a13fcf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:32:51.911293 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea73852c-adc2-46ec-b667-6531d2a13fcf None admin] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:32:51.911569 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea73852c-adc2-46ec-b667-6531d2a13fcf None admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:32:51.911843 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea73852c-adc2-46ec-b667-6531d2a13fcf None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:32:51.964414 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9655e32e-3a6e-4794-be76-f32cec4033d5 None None] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:32:51.967632 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=100 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:32:51.988662 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea73852c-adc2-46ec-b667-6531d2a13fcf None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:32:51.990290 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea73852c-adc2-46ec-b667-6531d2a13fcf None admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:32:52.011412 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e8034d0-b6ac-4f40-9aa8-2450b4911f9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:32:52.012538 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8034d0-b6ac-4f40-9aa8-2450b4911f9b None admin] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:32:52.012825 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8034d0-b6ac-4f40-9aa8-2450b4911f9b None admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:32:52.013103 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8034d0-b6ac-4f40-9aa8-2450b4911f9b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:32:52.016023 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=100 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:32:52.026470 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e8034d0-b6ac-4f40-9aa8-2450b4911f9b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 09:32:52.035732 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application [None req-3e8034d0-b6ac-4f40-9aa8-2450b4911f9b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 09:32:52.035732 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:32:52.035732 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:32:52.035732 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:32:52.035732 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:32:52.035732 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:32:52.035732 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:32:52.035732 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:32:52.035732 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:32:52.035732 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:32:52.035732 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:32:52.035732 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:32:52.035732 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 09:32:52.035732 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 09:32:52.035732 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 09:32:52.035732 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 09:32:52.037973 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 09:32:52.037973 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 09:32:52.037973 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 09:32:52.037973 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 09:32:52.037973 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:32:52.037973 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:32:52.037973 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:32:52.037973 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:32:52.037973 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:32:52.037973 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:32:52.037973 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:32:52.037973 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application with Lock( Aug 31 09:32:52.037973 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:32:52.037973 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:32:52.037973 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:32:52.037973 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:32:52.039432 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:32:52.039432 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:32:52.039432 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:32:52.039432 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:32:52.039432 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 09:32:52.039432 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 09:32:52.039432 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 09:32:52.039432 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 09:32:52.039432 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 09:32:52.039432 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 09:32:52.039432 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 09:32:52.039432 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application Aug 31 09:32:52.039432 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 7/14] 10.222.0.13 () {56 vars in 1096 bytes} [Wed Aug 31 09:32:51 2022] GET /identity/v3/roles/admin => generated 83 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 09:32:52.042849 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e8034d0-b6ac-4f40-9aa8-2450b4911f9b None admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:32:52.098446 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 8/15] 10.222.0.13 () {58 vars in 1282 bytes} [Wed Aug 31 09:32:51 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 348 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 31 09:32:52.104024 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea73852c-adc2-46ec-b667-6531d2a13fcf None admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:32:52.149885 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e8034d0-b6ac-4f40-9aa8-2450b4911f9b None admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:32:52.185663 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-641568c9-3897-4184-8b75-97c3edf425fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:32:52.186283 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641568c9-3897-4184-8b75-97c3edf425fd None admin] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:32:52.186561 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641568c9-3897-4184-8b75-97c3edf425fd None admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:32:52.186827 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641568c9-3897-4184-8b75-97c3edf425fd None admin] PATH_INFO: `/v3/roles` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:32:52.187504 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-641568c9-3897-4184-8b75-97c3edf425fd None admin] RBAC: Adding query filter params (name=admin) {{(pid=88114) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:32:52.187804 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-641568c9-3897-4184-8b75-97c3edf425fd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:32:52.189253 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-641568c9-3897-4184-8b75-97c3edf425fd None admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:32:52.201538 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 8/16] 10.222.0.13 () {56 vars in 1099 bytes} [Wed Aug 31 09:32:52 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 09:32:52.207703 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-641568c9-3897-4184-8b75-97c3edf425fd None admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:32:52.210020 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea73852c-adc2-46ec-b667-6531d2a13fcf None admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:32:52.234791 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=100 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:32:52.236234 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=100 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:32:52.247454 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da0e3094-e5fe-40a9-abd8-cb33671526a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:32:52.248097 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0e3094-e5fe-40a9-abd8-cb33671526a7 None admin] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:32:52.248379 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0e3094-e5fe-40a9-abd8-cb33671526a7 None admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:32:52.248653 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0e3094-e5fe-40a9-abd8-cb33671526a7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:32:52.259595 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da0e3094-e5fe-40a9-abd8-cb33671526a7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:32:52.261095 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da0e3094-e5fe-40a9-abd8-cb33671526a7 None admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:32:52.282085 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application [None req-da0e3094-e5fe-40a9-abd8-cb33671526a7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 09:32:52.282085 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:32:52.282085 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:32:52.282085 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:32:52.282085 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:32:52.282085 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:32:52.282085 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:32:52.282085 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:32:52.282085 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:32:52.282085 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:32:52.282085 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:32:52.282085 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:32:52.282085 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 09:32:52.282085 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 09:32:52.282085 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 09:32:52.282085 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 09:32:52.283090 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:32:52.283090 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:32:52.283090 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:32:52.283090 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:32:52.283090 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:32:52.283090 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:32:52.283090 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:32:52.283090 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application with Lock( Aug 31 09:32:52.283090 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:32:52.283090 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:32:52.283090 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:32:52.283090 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:32:52.283090 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:32:52.283090 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:32:52.283090 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:32:52.283090 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:32:52.283090 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 09:32:52.284911 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 09:32:52.284911 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 09:32:52.284911 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application Aug 31 09:32:52.284911 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 9/17] 10.222.0.13 () {56 vars in 1108 bytes} [Wed Aug 31 09:32:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 09:32:52.289928 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da0e3094-e5fe-40a9-abd8-cb33671526a7 None admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:32:52.340293 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-641568c9-3897-4184-8b75-97c3edf425fd None admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:32:52.392225 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da0e3094-e5fe-40a9-abd8-cb33671526a7 None admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:32:52.399715 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-073bb58c-eec5-450b-b685-7810a4d0dbf4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:32:52.400796 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073bb58c-eec5-450b-b685-7810a4d0dbf4 None admin] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:32:52.401298 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073bb58c-eec5-450b-b685-7810a4d0dbf4 None admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:32:52.401831 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073bb58c-eec5-450b-b685-7810a4d0dbf4 None admin] PATH_INFO: `/v3/users/fd836cf590a145df9acf19cf3ae2ac56` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:32:52.404513 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-073bb58c-eec5-450b-b685-7810a4d0dbf4 None admin] RBAC: Authorizing `identity:get_user(user_id=fd836cf590a145df9acf19cf3ae2ac56)` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:32:52.407354 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-073bb58c-eec5-450b-b685-7810a4d0dbf4 None admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:32:52.415564 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 9/18] 10.222.0.13 () {56 vars in 1177 bytes} [Wed Aug 31 09:32:52 2022] GET /identity/v3/users/fd836cf590a145df9acf19cf3ae2ac56 => generated 247 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 09:32:52.416677 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-073bb58c-eec5-450b-b685-7810a4d0dbf4 None admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:32:52.430127 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a628b725-1fcb-4a88-974d-2e40621060bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:32:52.430755 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a628b725-1fcb-4a88-974d-2e40621060bf None admin] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:32:52.431034 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a628b725-1fcb-4a88-974d-2e40621060bf None admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:32:52.431305 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a628b725-1fcb-4a88-974d-2e40621060bf None admin] PATH_INFO: `/v3/domains` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:32:52.431970 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a628b725-1fcb-4a88-974d-2e40621060bf None admin] RBAC: Adding query filter params (name=Default) {{(pid=88113) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:32:52.432275 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a628b725-1fcb-4a88-974d-2e40621060bf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:32:52.433770 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a628b725-1fcb-4a88-974d-2e40621060bf None admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:32:52.454376 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 10/19] 10.222.0.13 () {56 vars in 1109 bytes} [Wed Aug 31 09:32:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 09:32:52.540557 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=100 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:32:52.553188 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-073bb58c-eec5-450b-b685-7810a4d0dbf4 None admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:32:52.614072 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed3d1f5-3448-4d9e-9dac-4ee8c7b8c50b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:32:52.615049 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed3d1f5-3448-4d9e-9dac-4ee8c7b8c50b None admin] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:32:52.615470 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed3d1f5-3448-4d9e-9dac-4ee8c7b8c50b None admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:32:52.615877 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed3d1f5-3448-4d9e-9dac-4ee8c7b8c50b None admin] PATH_INFO: `/v3/domains/default` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:32:52.617878 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ed3d1f5-3448-4d9e-9dac-4ee8c7b8c50b None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:32:52.620413 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ed3d1f5-3448-4d9e-9dac-4ee8c7b8c50b None admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:32:52.623279 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 10/20] 10.222.0.13 () {56 vars in 1108 bytes} [Wed Aug 31 09:32:52 2022] GET /identity/v3/domains/default => generated 199 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:32:52.633043 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a628b725-1fcb-4a88-974d-2e40621060bf None admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:32:52.689346 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=100 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:32:52.782014 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a628b725-1fcb-4a88-974d-2e40621060bf None admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:32:52.827037 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc5c0ebf-d7c0-43d9-87aa-1425e68d6c34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:32:52.827642 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5c0ebf-d7c0-43d9-87aa-1425e68d6c34 None admin] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:32:52.827926 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5c0ebf-d7c0-43d9-87aa-1425e68d6c34 None admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:32:52.828200 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5c0ebf-d7c0-43d9-87aa-1425e68d6c34 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:32:52.828951 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc5c0ebf-d7c0-43d9-87aa-1425e68d6c34 None admin] RBAC: Adding query filter params (role.id=5d4ee63a3add47fc9a6b8b16d39e0a1f, scope.domain.id=default, user.id=fd836cf590a145df9acf19cf3ae2ac56) {{(pid=88113) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:32:52.829262 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc5c0ebf-d7c0-43d9-87aa-1425e68d6c34 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:32:52.831457 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc5c0ebf-d7c0-43d9-87aa-1425e68d6c34 None admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:32:52.843105 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 11/21] 10.222.0.13 () {56 vars in 1322 bytes} [Wed Aug 31 09:32:52 2022] GET /identity/v3/role_assignments?user.id=fd836cf590a145df9acf19cf3ae2ac56&scope.domain.id=default&role.id=5d4ee63a3add47fc9a6b8b16d39e0a1f => generated 233 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:32:54.632270 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5175912c-4fdf-492a-b705-1e0ab72632e9 None None] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:32:54.632787 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5175912c-4fdf-492a-b705-1e0ab72632e9 None None] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:32:54.632860 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5175912c-4fdf-492a-b705-1e0ab72632e9 None None] PATH_INFO: `/` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:32:54.634195 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 11/22] 10.222.0.13 () {54 vars in 927 bytes} [Wed Aug 31 09:32:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:32:54.639782 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0156c12a-5b63-4749-b4c6-23bec59dc7a9 None None] REQUEST_METHOD: `POST` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:32:54.640261 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0156c12a-5b63-4749-b4c6-23bec59dc7a9 None None] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:32:54.640365 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0156c12a-5b63-4749-b4c6-23bec59dc7a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:32:54.665986 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.auth.core [None req-0156c12a-5b63-4749-b4c6-23bec59dc7a9 None None] MFA Rules not processed for user `fd836cf590a145df9acf19cf3ae2ac56`. Rule list: `[]` (Enabled: `True`). {{(pid=88113) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:32:54.667202 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.fernet_utils [None req-0156c12a-5b63-4749-b4c6-23bec59dc7a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:32:54.759253 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 12/23] 10.222.0.13 () {58 vars in 1024 bytes} [Wed Aug 31 09:32:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 09:32:55.322024 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5175912c-4fdf-492a-b705-1e0ab72632e9 None None] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:32:55.448400 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5175912c-4fdf-492a-b705-1e0ab72632e9 None None] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:32:55.496595 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d85a003-22cb-402a-bf2e-de23fd42850b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:32:55.497242 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d85a003-22cb-402a-bf2e-de23fd42850b None admin] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:32:55.497552 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d85a003-22cb-402a-bf2e-de23fd42850b None admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:32:55.497829 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d85a003-22cb-402a-bf2e-de23fd42850b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:32:55.525188 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d85a003-22cb-402a-bf2e-de23fd42850b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:32:55.526740 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d85a003-22cb-402a-bf2e-de23fd42850b None admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:32:55.551718 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application [None req-6d85a003-22cb-402a-bf2e-de23fd42850b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 09:32:55.551718 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:32:55.551718 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:32:55.551718 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:32:55.551718 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:32:55.551718 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:32:55.551718 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:32:55.551718 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:32:55.551718 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:32:55.551718 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:32:55.551718 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:32:55.551718 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:32:55.551718 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 09:32:55.551718 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 09:32:55.551718 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 09:32:55.551718 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 09:32:55.553100 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:32:55.553100 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:32:55.553100 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:32:55.553100 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:32:55.553100 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:32:55.553100 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:32:55.553100 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:32:55.553100 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application with Lock( Aug 31 09:32:55.553100 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:32:55.553100 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:32:55.553100 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:32:55.553100 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:32:55.553100 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:32:55.553100 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:32:55.553100 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:32:55.553100 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:32:55.553100 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 09:32:55.555740 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 09:32:55.555740 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 09:32:55.555740 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application Aug 31 09:32:55.555740 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 12/24] 10.222.0.13 () {56 vars in 1108 bytes} [Wed Aug 31 09:32:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:32:55.562787 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0156c12a-5b63-4749-b4c6-23bec59dc7a9 None None] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:32:55.596118 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=100 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:32:55.599627 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27939fd7-ea49-45bc-ba5e-61801a44611c None None] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:32:55.599627 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27939fd7-ea49-45bc-ba5e-61801a44611c None None] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:32:55.599627 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27939fd7-ea49-45bc-ba5e-61801a44611c None None] PATH_INFO: `/` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:32:55.599627 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 13/25] 10.222.0.13 () {54 vars in 927 bytes} [Wed Aug 31 09:32:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:32:55.611063 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7efc5be2-65e2-4229-9e24-009df97e20d2 None None] REQUEST_METHOD: `POST` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:32:55.611399 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7efc5be2-65e2-4229-9e24-009df97e20d2 None None] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:32:55.611660 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7efc5be2-65e2-4229-9e24-009df97e20d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:32:55.645031 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.auth.core [None req-7efc5be2-65e2-4229-9e24-009df97e20d2 None None] MFA Rules not processed for user `fd836cf590a145df9acf19cf3ae2ac56`. Rule list: `[]` (Enabled: `True`). {{(pid=88114) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:32:55.646586 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.fernet_utils [None req-7efc5be2-65e2-4229-9e24-009df97e20d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88114) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:32:55.665915 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=100 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:32:55.761499 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0156c12a-5b63-4749-b4c6-23bec59dc7a9 None None] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:32:55.829692 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c48b88da-d1a9-41ba-a6e4-5657823b725e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:32:55.832032 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48b88da-d1a9-41ba-a6e4-5657823b725e None admin] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:32:55.832538 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48b88da-d1a9-41ba-a6e4-5657823b725e None admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:32:55.832907 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 14/26] 10.222.0.13 () {58 vars in 1024 bytes} [Wed Aug 31 09:32:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 223 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 31 09:32:55.833307 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48b88da-d1a9-41ba-a6e4-5657823b725e None admin] PATH_INFO: `/v3/domains` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:32:55.834231 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c48b88da-d1a9-41ba-a6e4-5657823b725e None admin] RBAC: Adding query filter params (name=Default) {{(pid=88113) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:32:55.834747 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c48b88da-d1a9-41ba-a6e4-5657823b725e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:32:55.835305 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3eca4a0-4b17-4415-b524-ce5c0fe156fc None None] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:32:55.835803 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3eca4a0-4b17-4415-b524-ce5c0fe156fc None None] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:32:55.836296 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3eca4a0-4b17-4415-b524-ce5c0fe156fc None None] PATH_INFO: `/` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:32:55.837039 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c48b88da-d1a9-41ba-a6e4-5657823b725e None admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:32:55.839563 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 15/27] 10.222.0.13 () {54 vars in 927 bytes} [Wed Aug 31 09:32:55 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:32:55.845897 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0555b7c0-6e22-4245-9246-0f2b408cfe50 None None] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:32:55.845897 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0555b7c0-6e22-4245-9246-0f2b408cfe50 None None] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:32:55.845897 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0555b7c0-6e22-4245-9246-0f2b408cfe50 None None] PATH_INFO: `/` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:32:55.847589 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 16/28] 10.222.0.13 () {54 vars in 927 bytes} [Wed Aug 31 09:32:55 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:32:55.848724 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac26ecd-02fa-4efc-8f03-040deaf99830 None None] REQUEST_METHOD: `POST` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:32:55.849018 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac26ecd-02fa-4efc-8f03-040deaf99830 None None] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:32:55.849291 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac26ecd-02fa-4efc-8f03-040deaf99830 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:32:55.864458 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 13/29] 10.222.0.13 () {56 vars in 1109 bytes} [Wed Aug 31 09:32:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 302 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 09:32:55.865647 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c488f5b-16ec-486d-8b8c-7bfc24cc5860 None None] REQUEST_METHOD: `POST` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:32:55.866029 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c488f5b-16ec-486d-8b8c-7bfc24cc5860 None None] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:32:55.866315 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c488f5b-16ec-486d-8b8c-7bfc24cc5860 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:32:55.886724 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.auth.core [None req-8c488f5b-16ec-486d-8b8c-7bfc24cc5860 None None] MFA Rules not processed for user `fd836cf590a145df9acf19cf3ae2ac56`. Rule list: `[]` (Enabled: `True`). {{(pid=88113) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:32:55.887388 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.auth.core [None req-0ac26ecd-02fa-4efc-8f03-040deaf99830 None None] MFA Rules not processed for user `fd836cf590a145df9acf19cf3ae2ac56`. Rule list: `[]` (Enabled: `True`). {{(pid=88114) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:32:55.888027 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.fernet_utils [None req-8c488f5b-16ec-486d-8b8c-7bfc24cc5860 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:32:55.890480 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.fernet_utils [None req-0ac26ecd-02fa-4efc-8f03-040deaf99830 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88114) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:32:56.032261 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 14/30] 10.222.0.13 () {58 vars in 1024 bytes} [Wed Aug 31 09:32:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 09:32:56.036123 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c488f5b-16ec-486d-8b8c-7bfc24cc5860 None None] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:32:56.046504 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 17/31] 10.222.0.13 () {58 vars in 1024 bytes} [Wed Aug 31 09:32:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 09:32:56.098667 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=100 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:32:56.163315 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac26ecd-02fa-4efc-8f03-040deaf99830 None None] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:32:56.211870 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c488f5b-16ec-486d-8b8c-7bfc24cc5860 None None] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:32:56.279564 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c31089-9bfe-4838-8f36-c75f67ea8890 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:32:56.280191 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c31089-9bfe-4838-8f36-c75f67ea8890 None admin] REQUEST_METHOD: `POST` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:32:56.280476 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c31089-9bfe-4838-8f36-c75f67ea8890 None admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:32:56.280750 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c31089-9bfe-4838-8f36-c75f67ea8890 None admin] PATH_INFO: `/v3/projects` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:32:56.281719 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6c31089-9bfe-4838-8f36-c75f67ea8890 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:32:56.285106 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6c31089-9bfe-4838-8f36-c75f67ea8890 None admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:32:56.352597 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=100 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:32:56.397948 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac26ecd-02fa-4efc-8f03-040deaf99830 None None] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:32:56.477011 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c4664f-39b0-4987-b634-a022f96b2d5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:32:56.478056 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c4664f-39b0-4987-b634-a022f96b2d5b None admin] REQUEST_METHOD: `POST` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:32:56.478551 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c4664f-39b0-4987-b634-a022f96b2d5b None admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:32:56.479038 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c4664f-39b0-4987-b634-a022f96b2d5b None admin] PATH_INFO: `/v3/roles` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:32:56.479956 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26c4664f-39b0-4987-b634-a022f96b2d5b None admin] RBAC: Authorizing `identity:create_role()` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:32:56.481621 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26c4664f-39b0-4987-b634-a022f96b2d5b None admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:32:56.557232 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 15/32] 10.222.0.13 () {60 vars in 1141 bytes} [Wed Aug 31 09:32:56 2022] POST /identity/v3/projects => generated 301 bytes in 525 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 09:32:56.558664 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c31089-9bfe-4838-8f36-c75f67ea8890 None admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:32:56.648422 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 18/33] 10.222.0.13 () {60 vars in 1131 bytes} [Wed Aug 31 09:32:56 2022] POST /identity/v3/roles => generated 219 bytes in 486 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 09:32:56.653521 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:32:56.694695 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c31089-9bfe-4838-8f36-c75f67ea8890 None admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:32:56.736417 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f4f6a6-b13c-4b0b-92dd-e048cf4c18c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:32:56.737104 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f4f6a6-b13c-4b0b-92dd-e048cf4c18c2 None admin] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:32:56.737431 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f4f6a6-b13c-4b0b-92dd-e048cf4c18c2 None admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:32:56.737678 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f4f6a6-b13c-4b0b-92dd-e048cf4c18c2 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:32:56.752432 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81f4f6a6-b13c-4b0b-92dd-e048cf4c18c2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 09:32:56.758683 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application [None req-81f4f6a6-b13c-4b0b-92dd-e048cf4c18c2 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 09:32:56.758683 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:32:56.758683 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:32:56.758683 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:32:56.758683 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:32:56.758683 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:32:56.758683 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:32:56.758683 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:32:56.758683 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:32:56.758683 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:32:56.758683 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:32:56.758683 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:32:56.758683 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 09:32:56.758683 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 09:32:56.758683 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 09:32:56.758683 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 09:32:56.759601 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 09:32:56.759601 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 09:32:56.759601 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 09:32:56.759601 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 09:32:56.759601 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:32:56.759601 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:32:56.759601 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:32:56.759601 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:32:56.759601 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:32:56.759601 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:32:56.759601 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:32:56.759601 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application with Lock( Aug 31 09:32:56.759601 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:32:56.759601 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:32:56.759601 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:32:56.759601 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:32:56.760348 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:32:56.760348 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:32:56.760348 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:32:56.760348 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:32:56.760348 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 09:32:56.760348 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 09:32:56.760348 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 09:32:56.760348 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 09:32:56.760348 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 09:32:56.760348 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 09:32:56.760348 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 09:32:56.760348 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application Aug 31 09:32:56.760348 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 16/34] 10.222.0.13 () {56 vars in 1096 bytes} [Wed Aug 31 09:32:56 2022] GET /identity/v3/roles/admin => generated 83 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:32:56.771213 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c4664f-39b0-4987-b634-a022f96b2d5b None admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:32:56.927685 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c4664f-39b0-4987-b634-a022f96b2d5b None admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:32:56.986178 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3cf2436-ebf7-4e33-bfd9-67cbb311a34e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:32:56.987561 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3cf2436-ebf7-4e33-bfd9-67cbb311a34e None admin] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:32:56.987857 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3cf2436-ebf7-4e33-bfd9-67cbb311a34e None admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:32:56.988136 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3cf2436-ebf7-4e33-bfd9-67cbb311a34e None admin] PATH_INFO: `/v3/roles` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:32:56.988869 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3cf2436-ebf7-4e33-bfd9-67cbb311a34e None admin] RBAC: Adding query filter params (name=admin) {{(pid=88114) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:32:56.989177 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3cf2436-ebf7-4e33-bfd9-67cbb311a34e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:32:56.990805 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3cf2436-ebf7-4e33-bfd9-67cbb311a34e None admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:32:56.992589 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:32:57.003196 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 19/35] 10.222.0.13 () {56 vars in 1099 bytes} [Wed Aug 31 09:32:56 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:32:57.013302 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f4f6a6-b13c-4b0b-92dd-e048cf4c18c2 None admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:32:57.013863 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3cf2436-ebf7-4e33-bfd9-67cbb311a34e None admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:32:57.135235 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3cf2436-ebf7-4e33-bfd9-67cbb311a34e None admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:32:57.175878 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c72f73-623a-477a-adf2-d0279c994411 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:32:57.176506 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c72f73-623a-477a-adf2-d0279c994411 None admin] REQUEST_METHOD: `POST` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:32:57.176791 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c72f73-623a-477a-adf2-d0279c994411 None admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:32:57.177072 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c72f73-623a-477a-adf2-d0279c994411 None admin] PATH_INFO: `/v3/roles` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:32:57.177863 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27c72f73-623a-477a-adf2-d0279c994411 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:32:57.179219 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27c72f73-623a-477a-adf2-d0279c994411 None admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:32:57.200104 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 20/36] 10.222.0.13 () {60 vars in 1131 bytes} [Wed Aug 31 09:32:57 2022] POST /identity/v3/roles => generated 225 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 09:32:57.203841 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e30c4e1-aa3f-4ea8-b027-dad71fd710fd None None] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:32:57.204463 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e30c4e1-aa3f-4ea8-b027-dad71fd710fd None None] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:32:57.205917 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e30c4e1-aa3f-4ea8-b027-dad71fd710fd None None] PATH_INFO: `/` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:32:57.210840 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 21/37] 10.222.0.13 () {54 vars in 927 bytes} [Wed Aug 31 09:32:57 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:32:57.217871 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f4f6a6-b13c-4b0b-92dd-e048cf4c18c2 None admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:32:57.219901 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609c69df-998f-49a8-99b6-ed1054935e4c None None] REQUEST_METHOD: `POST` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:32:57.220619 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609c69df-998f-49a8-99b6-ed1054935e4c None None] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:32:57.220918 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609c69df-998f-49a8-99b6-ed1054935e4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:32:57.254399 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.auth.core [None req-609c69df-998f-49a8-99b6-ed1054935e4c None None] MFA Rules not processed for user `fd836cf590a145df9acf19cf3ae2ac56`. Rule list: `[]` (Enabled: `True`). {{(pid=88114) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:32:57.257374 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.fernet_utils [None req-609c69df-998f-49a8-99b6-ed1054935e4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88114) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:32:57.259190 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=100 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:32:57.269228 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-309db08a-41a4-4f93-a2da-687cd3c1f655 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:32:57.270682 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309db08a-41a4-4f93-a2da-687cd3c1f655 None admin] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:32:57.271210 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309db08a-41a4-4f93-a2da-687cd3c1f655 None admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:32:57.271673 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309db08a-41a4-4f93-a2da-687cd3c1f655 None admin] PATH_INFO: `/v3/users/fd836cf590a145df9acf19cf3ae2ac56` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:32:57.273542 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-309db08a-41a4-4f93-a2da-687cd3c1f655 None admin] RBAC: Authorizing `identity:get_user(user_id=fd836cf590a145df9acf19cf3ae2ac56)` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:32:57.275587 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-309db08a-41a4-4f93-a2da-687cd3c1f655 None admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:32:57.277932 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 17/38] 10.222.0.13 () {56 vars in 1177 bytes} [Wed Aug 31 09:32:57 2022] GET /identity/v3/users/fd836cf590a145df9acf19cf3ae2ac56 => generated 247 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:32:57.281923 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e83380-82fd-4d15-ae06-235f6d12b9aa None None] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:32:57.282304 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e83380-82fd-4d15-ae06-235f6d12b9aa None None] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:32:57.282686 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e83380-82fd-4d15-ae06-235f6d12b9aa None None] PATH_INFO: `/` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:32:57.284992 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 18/39] 10.222.0.13 () {54 vars in 927 bytes} [Wed Aug 31 09:32:57 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:32:57.289875 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e83380-82fd-4d15-ae06-235f6d12b9aa None None] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:32:57.339567 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:32:57.363012 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 22/40] 10.222.0.13 () {58 vars in 1024 bytes} [Wed Aug 31 09:32:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 31 09:32:57.364513 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca38c6a-3129-4e4d-9f68-f24c77ae6e7e None None] REQUEST_METHOD: `POST` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:32:57.364815 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca38c6a-3129-4e4d-9f68-f24c77ae6e7e None None] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:32:57.365099 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca38c6a-3129-4e4d-9f68-f24c77ae6e7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:32:57.385646 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.auth.core [None req-2ca38c6a-3129-4e4d-9f68-f24c77ae6e7e None None] MFA Rules not processed for user `fd836cf590a145df9acf19cf3ae2ac56`. Rule list: `[]` (Enabled: `True`). {{(pid=88114) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:32:57.386883 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.fernet_utils [None req-2ca38c6a-3129-4e4d-9f68-f24c77ae6e7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88114) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:32:57.458278 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e83380-82fd-4d15-ae06-235f6d12b9aa None None] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:32:57.484090 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 23/41] 10.222.0.13 () {58 vars in 1024 bytes} [Wed Aug 31 09:32:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 09:32:57.502346 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f151e4-1a42-49f6-9be1-6b5c78d9028b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:32:57.503068 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f151e4-1a42-49f6-9be1-6b5c78d9028b None admin] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:32:57.503405 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f151e4-1a42-49f6-9be1-6b5c78d9028b None admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:32:57.503684 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f151e4-1a42-49f6-9be1-6b5c78d9028b None admin] PATH_INFO: `/v3/domains/default` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:32:57.504954 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8f151e4-1a42-49f6-9be1-6b5c78d9028b None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:32:57.506469 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8f151e4-1a42-49f6-9be1-6b5c78d9028b None admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:32:57.508237 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 19/42] 10.222.0.13 () {56 vars in 1108 bytes} [Wed Aug 31 09:32:57 2022] GET /identity/v3/domains/default => generated 199 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:32:57.513811 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca38c6a-3129-4e4d-9f68-f24c77ae6e7e None None] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:32:57.622198 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca38c6a-3129-4e4d-9f68-f24c77ae6e7e None None] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:32:57.640030 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=100 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:32:57.660440 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faff5fde-b9cd-4010-b428-a23f50f1f1f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:32:57.661120 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faff5fde-b9cd-4010-b428-a23f50f1f1f9 None admin] REQUEST_METHOD: `PUT` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:32:57.661417 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faff5fde-b9cd-4010-b428-a23f50f1f1f9 None admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:32:57.661691 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faff5fde-b9cd-4010-b428-a23f50f1f1f9 None admin] PATH_INFO: `/v3/domains/default/users/fd836cf590a145df9acf19cf3ae2ac56/roles/5d4ee63a3add47fc9a6b8b16d39e0a1f` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:32:57.663646 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faff5fde-b9cd-4010-b428-a23f50f1f1f9 None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=fd836cf590a145df9acf19cf3ae2ac56, role_id=5d4ee63a3add47fc9a6b8b16d39e0a1f)` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:32:57.665114 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faff5fde-b9cd-4010-b428-a23f50f1f1f9 None admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:32:57.681888 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 24/43] 10.222.0.13 () {58 vars in 1361 bytes} [Wed Aug 31 09:32:57 2022] PUT /identity/v3/domains/default/users/fd836cf590a145df9acf19cf3ae2ac56/roles/5d4ee63a3add47fc9a6b8b16d39e0a1f => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 09:32:57.755467 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f151e4-1a42-49f6-9be1-6b5c78d9028b None admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:32:57.828611 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faff5fde-b9cd-4010-b428-a23f50f1f1f9 None admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:32:57.845635 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=100 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:32:57.888021 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f151e4-1a42-49f6-9be1-6b5c78d9028b None admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:32:57.927163 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08fe2ff4-cd53-450d-8dee-3f07b68b341d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:32:57.927797 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08fe2ff4-cd53-450d-8dee-3f07b68b341d None admin] REQUEST_METHOD: `POST` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:32:57.928083 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08fe2ff4-cd53-450d-8dee-3f07b68b341d None admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:32:57.928407 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08fe2ff4-cd53-450d-8dee-3f07b68b341d None admin] PATH_INFO: `/v3/roles` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:32:57.929105 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08fe2ff4-cd53-450d-8dee-3f07b68b341d None admin] RBAC: Authorizing `identity:create_role()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:32:57.930560 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08fe2ff4-cd53-450d-8dee-3f07b68b341d None admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:32:57.949058 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 20/44] 10.222.0.13 () {60 vars in 1131 bytes} [Wed Aug 31 09:32:57 2022] POST /identity/v3/roles => generated 223 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 09:32:57.957173 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faff5fde-b9cd-4010-b428-a23f50f1f1f9 None admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:32:58.007244 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df4729c6-5ef5-4c3d-8de0-a631f4f80518 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:32:58.007838 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4729c6-5ef5-4c3d-8de0-a631f4f80518 None admin] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:32:58.008126 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4729c6-5ef5-4c3d-8de0-a631f4f80518 None admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:32:58.008408 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4729c6-5ef5-4c3d-8de0-a631f4f80518 None admin] PATH_INFO: `/v3/domains/default` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:32:58.009808 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df4729c6-5ef5-4c3d-8de0-a631f4f80518 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:32:58.011387 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df4729c6-5ef5-4c3d-8de0-a631f4f80518 None admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:32:58.013371 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 25/45] 10.222.0.13 () {56 vars in 1108 bytes} [Wed Aug 31 09:32:57 2022] GET /identity/v3/domains/default => generated 199 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:32:58.019568 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08fe2ff4-cd53-450d-8dee-3f07b68b341d None admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:32:58.247637 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08fe2ff4-cd53-450d-8dee-3f07b68b341d None admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:32:58.276224 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:32:58.287568 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f1be97-8510-4648-a957-e62ace3a5551 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:32:58.288192 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f1be97-8510-4648-a957-e62ace3a5551 None admin] REQUEST_METHOD: `POST` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:32:58.288473 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f1be97-8510-4648-a957-e62ace3a5551 None admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:32:58.288748 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f1be97-8510-4648-a957-e62ace3a5551 None admin] PATH_INFO: `/v3/projects` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:32:58.289450 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88f1be97-8510-4648-a957-e62ace3a5551 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:32:58.290912 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88f1be97-8510-4648-a957-e62ace3a5551 None admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:32:58.327836 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 21/46] 10.222.0.13 () {60 vars in 1141 bytes} [Wed Aug 31 09:32:58 2022] POST /identity/v3/projects => generated 312 bytes in 309 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 09:32:59.679553 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47513c2-f02f-4c2b-9d49-0a2bdb037b42 None None] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:32:59.680086 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47513c2-f02f-4c2b-9d49-0a2bdb037b42 None None] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:32:59.680086 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47513c2-f02f-4c2b-9d49-0a2bdb037b42 None None] PATH_INFO: `/` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:32:59.681574 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 26/47] 10.222.0.13 () {54 vars in 927 bytes} [Wed Aug 31 09:32:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:32:59.687212 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e898c2fe-0049-48b1-864f-16ac9e796fee None None] REQUEST_METHOD: `POST` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:32:59.687759 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e898c2fe-0049-48b1-864f-16ac9e796fee None None] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:32:59.687759 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e898c2fe-0049-48b1-864f-16ac9e796fee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:32:59.718866 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.auth.core [None req-e898c2fe-0049-48b1-864f-16ac9e796fee None None] MFA Rules not processed for user `fd836cf590a145df9acf19cf3ae2ac56`. Rule list: `[]` (Enabled: `True`). {{(pid=88113) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:32:59.721997 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.fernet_utils [None req-e898c2fe-0049-48b1-864f-16ac9e796fee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:32:59.818280 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 22/48] 10.222.0.13 () {58 vars in 1024 bytes} [Wed Aug 31 09:32:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 09:33:00.092309 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c47513c2-f02f-4c2b-9d49-0a2bdb037b42 None None] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:00.148777 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:33:00.210444 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c47513c2-f02f-4c2b-9d49-0a2bdb037b42 None None] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:00.246361 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60840237-2ef6-4aa4-8461-73e2be627e15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:00.246925 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60840237-2ef6-4aa4-8461-73e2be627e15 None admin] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:00.247210 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60840237-2ef6-4aa4-8461-73e2be627e15 None admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:00.247485 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60840237-2ef6-4aa4-8461-73e2be627e15 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:00.259229 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60840237-2ef6-4aa4-8461-73e2be627e15 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 09:33:00.264669 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application [None req-60840237-2ef6-4aa4-8461-73e2be627e15 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 09:33:00.264669 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:33:00.264669 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:33:00.264669 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:33:00.264669 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:33:00.264669 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:33:00.264669 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:33:00.264669 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:33:00.264669 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:33:00.264669 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:33:00.264669 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:33:00.264669 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:33:00.264669 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 09:33:00.264669 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 09:33:00.264669 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 09:33:00.264669 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 09:33:00.266480 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 09:33:00.266480 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 09:33:00.266480 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 09:33:00.266480 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 09:33:00.266480 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:33:00.266480 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:33:00.266480 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:33:00.266480 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:33:00.266480 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:33:00.266480 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:33:00.266480 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:33:00.266480 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application with Lock( Aug 31 09:33:00.266480 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:33:00.266480 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:33:00.266480 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:33:00.266480 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:33:00.268101 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:33:00.268101 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:33:00.268101 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:33:00.268101 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:33:00.268101 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 09:33:00.268101 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 09:33:00.268101 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 09:33:00.268101 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 09:33:00.268101 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 09:33:00.268101 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 09:33:00.268101 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 09:33:00.268101 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application Aug 31 09:33:00.268101 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 27/49] 10.222.0.13 () {56 vars in 1096 bytes} [Wed Aug 31 09:33:00 2022] GET /identity/v3/roles/admin => generated 83 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:33:00.275827 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e898c2fe-0049-48b1-864f-16ac9e796fee None None] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:00.403663 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32768b4-269d-4f3c-81c6-c9925d3bf1a1 None None] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:00.404104 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32768b4-269d-4f3c-81c6-c9925d3bf1a1 None None] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:00.404596 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32768b4-269d-4f3c-81c6-c9925d3bf1a1 None None] PATH_INFO: `/` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:00.406726 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 28/50] 10.222.0.13 () {54 vars in 927 bytes} [Wed Aug 31 09:33:00 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:33:00.414912 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb9988d-a831-4118-8075-91ad28f0e7bd None None] REQUEST_METHOD: `POST` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:00.415318 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb9988d-a831-4118-8075-91ad28f0e7bd None None] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:00.415755 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb9988d-a831-4118-8075-91ad28f0e7bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:00.416193 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e898c2fe-0049-48b1-864f-16ac9e796fee None None] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:00.426202 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:33:00.444288 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.auth.core [None req-dbb9988d-a831-4118-8075-91ad28f0e7bd None None] MFA Rules not processed for user `fd836cf590a145df9acf19cf3ae2ac56`. Rule list: `[]` (Enabled: `True`). {{(pid=88114) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:33:00.446106 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.fernet_utils [None req-dbb9988d-a831-4118-8075-91ad28f0e7bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88114) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:33:00.452782 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebbddb1d-a9d9-442f-b02e-d6098574fc3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:00.453432 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebbddb1d-a9d9-442f-b02e-d6098574fc3b None admin] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:00.453705 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebbddb1d-a9d9-442f-b02e-d6098574fc3b None admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:00.453978 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebbddb1d-a9d9-442f-b02e-d6098574fc3b None admin] PATH_INFO: `/v3/roles` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:00.454650 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebbddb1d-a9d9-442f-b02e-d6098574fc3b None admin] RBAC: Adding query filter params (name=admin) {{(pid=88113) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:33:00.455470 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebbddb1d-a9d9-442f-b02e-d6098574fc3b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:33:00.457541 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebbddb1d-a9d9-442f-b02e-d6098574fc3b None admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:33:00.469856 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 23/51] 10.222.0.13 () {56 vars in 1099 bytes} [Wed Aug 31 09:33:00 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:33:00.478603 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebbddb1d-a9d9-442f-b02e-d6098574fc3b None admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:00.540617 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 29/52] 10.222.0.13 () {58 vars in 1024 bytes} [Wed Aug 31 09:33:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 31 09:33:00.606704 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebbddb1d-a9d9-442f-b02e-d6098574fc3b None admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:00.656118 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd910563-2c85-48e1-a946-3396f8eb82a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:00.656930 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd910563-2c85-48e1-a946-3396f8eb82a6 None admin] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:00.657363 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd910563-2c85-48e1-a946-3396f8eb82a6 None admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:00.657793 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd910563-2c85-48e1-a946-3396f8eb82a6 None admin] PATH_INFO: `/v3/users/fd836cf590a145df9acf19cf3ae2ac56` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:00.659846 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd910563-2c85-48e1-a946-3396f8eb82a6 None admin] RBAC: Authorizing `identity:get_user(user_id=fd836cf590a145df9acf19cf3ae2ac56)` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:33:00.661738 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd910563-2c85-48e1-a946-3396f8eb82a6 None admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:33:00.663566 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 24/53] 10.222.0.13 () {56 vars in 1177 bytes} [Wed Aug 31 09:33:00 2022] GET /identity/v3/users/fd836cf590a145df9acf19cf3ae2ac56 => generated 247 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:33:00.671550 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb9988d-a831-4118-8075-91ad28f0e7bd None None] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:00.723559 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=100 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:33:00.797918 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb9988d-a831-4118-8075-91ad28f0e7bd None None] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:00.813177 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd910563-2c85-48e1-a946-3396f8eb82a6 None admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:00.846461 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d5374a2-981e-45ef-8791-6c92d930b1e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:00.847253 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5374a2-981e-45ef-8791-6c92d930b1e6 None admin] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:00.847542 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5374a2-981e-45ef-8791-6c92d930b1e6 None admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:00.847825 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5374a2-981e-45ef-8791-6c92d930b1e6 None admin] PATH_INFO: `/v3/domains/default` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:00.849469 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d5374a2-981e-45ef-8791-6c92d930b1e6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:33:00.851287 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d5374a2-981e-45ef-8791-6c92d930b1e6 None admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:33:00.853026 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 30/54] 10.222.0.13 () {56 vars in 1108 bytes} [Wed Aug 31 09:33:00 2022] GET /identity/v3/domains/default => generated 199 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:33:00.859288 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d5374a2-981e-45ef-8791-6c92d930b1e6 None admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:00.869374 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=100 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:33:00.968408 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd910563-2c85-48e1-a946-3396f8eb82a6 None admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:01.012930 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d5374a2-981e-45ef-8791-6c92d930b1e6 None admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:01.015262 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=100 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:33:01.022804 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf5a26c-b5bf-49d4-993e-1e24e0cb9f0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:01.023460 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf5a26c-b5bf-49d4-993e-1e24e0cb9f0b None admin] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:01.023736 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf5a26c-b5bf-49d4-993e-1e24e0cb9f0b None admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:01.024004 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf5a26c-b5bf-49d4-993e-1e24e0cb9f0b None admin] PATH_INFO: `/v3/domains/default` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:01.026254 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdf5a26c-b5bf-49d4-993e-1e24e0cb9f0b None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:33:01.027768 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdf5a26c-b5bf-49d4-993e-1e24e0cb9f0b None admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:33:01.029633 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 25/55] 10.222.0.13 () {56 vars in 1108 bytes} [Wed Aug 31 09:33:00 2022] GET /identity/v3/domains/default => generated 199 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 09:33:01.038446 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf5a26c-b5bf-49d4-993e-1e24e0cb9f0b None admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:01.064201 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f932fa83-9bdb-41eb-9b01-3bebd7b28e78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:01.065743 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f932fa83-9bdb-41eb-9b01-3bebd7b28e78 None admin] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:01.066165 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f932fa83-9bdb-41eb-9b01-3bebd7b28e78 None admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:01.066628 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f932fa83-9bdb-41eb-9b01-3bebd7b28e78 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:01.067919 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f932fa83-9bdb-41eb-9b01-3bebd7b28e78 None admin] RBAC: Adding query filter params (role.id=5d4ee63a3add47fc9a6b8b16d39e0a1f, scope.domain.id=default, user.id=fd836cf590a145df9acf19cf3ae2ac56) {{(pid=88114) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:33:01.068424 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f932fa83-9bdb-41eb-9b01-3bebd7b28e78 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:33:01.070785 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f932fa83-9bdb-41eb-9b01-3bebd7b28e78 None admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:33:01.085868 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 31/56] 10.222.0.13 () {56 vars in 1322 bytes} [Wed Aug 31 09:33:00 2022] GET /identity/v3/role_assignments?user.id=fd836cf590a145df9acf19cf3ae2ac56&scope.domain.id=default&role.id=5d4ee63a3add47fc9a6b8b16d39e0a1f => generated 530 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 09:33:01.180586 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf5a26c-b5bf-49d4-993e-1e24e0cb9f0b None admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:01.189249 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=100 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:33:01.227220 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d7502df-c566-434d-9c3c-ef11a1fb09fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:01.228236 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7502df-c566-434d-9c3c-ef11a1fb09fe None admin] REQUEST_METHOD: `POST` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:01.228712 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7502df-c566-434d-9c3c-ef11a1fb09fe None admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:01.229143 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7502df-c566-434d-9c3c-ef11a1fb09fe None admin] PATH_INFO: `/v3/projects` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:01.230345 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d7502df-c566-434d-9c3c-ef11a1fb09fe None admin] RBAC: Authorizing `identity:create_project()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:33:01.233049 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d7502df-c566-434d-9c3c-ef11a1fb09fe None admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:33:01.275400 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 26/57] 10.222.0.13 () {60 vars in 1140 bytes} [Wed Aug 31 09:33:01 2022] POST /identity/v3/projects => generated 298 bytes in 237 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 09:33:03.308677 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87535637-b3ff-4756-9b9c-9baff3cd6933 None None] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:03.309169 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87535637-b3ff-4756-9b9c-9baff3cd6933 None None] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:03.309169 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87535637-b3ff-4756-9b9c-9baff3cd6933 None None] PATH_INFO: `/` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:03.310506 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 32/58] 10.222.0.13 () {54 vars in 927 bytes} [Wed Aug 31 09:33:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:33:03.316417 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11bc9b5-c301-4d6f-b170-e1879ef39b9e None None] REQUEST_METHOD: `POST` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:03.316919 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11bc9b5-c301-4d6f-b170-e1879ef39b9e None None] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:03.317048 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11bc9b5-c301-4d6f-b170-e1879ef39b9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:03.336865 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.auth.core [None req-a11bc9b5-c301-4d6f-b170-e1879ef39b9e None None] MFA Rules not processed for user `fd836cf590a145df9acf19cf3ae2ac56`. Rule list: `[]` (Enabled: `True`). {{(pid=88113) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:33:03.338132 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.fernet_utils [None req-a11bc9b5-c301-4d6f-b170-e1879ef39b9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:33:03.413351 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 27/59] 10.222.0.13 () {58 vars in 1024 bytes} [Wed Aug 31 09:33:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 09:33:03.686392 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87535637-b3ff-4756-9b9c-9baff3cd6933 None None] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:03.903895 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87535637-b3ff-4756-9b9c-9baff3cd6933 None None] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:03.939581 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62fa51f2-e55a-45ff-8e97-04f403a2e981 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:03.940525 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62fa51f2-e55a-45ff-8e97-04f403a2e981 None admin] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:03.940808 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62fa51f2-e55a-45ff-8e97-04f403a2e981 None admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:03.941082 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62fa51f2-e55a-45ff-8e97-04f403a2e981 None admin] PATH_INFO: `/v3/domains/default` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:03.942239 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62fa51f2-e55a-45ff-8e97-04f403a2e981 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:33:03.943700 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62fa51f2-e55a-45ff-8e97-04f403a2e981 None admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:33:03.945275 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 33/60] 10.222.0.13 () {56 vars in 1108 bytes} [Wed Aug 31 09:33:03 2022] GET /identity/v3/domains/default => generated 199 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:33:03.950876 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a11bc9b5-c301-4d6f-b170-e1879ef39b9e None None] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:04.045718 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:33:04.060602 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a11bc9b5-c301-4d6f-b170-e1879ef39b9e None None] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:04.099916 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ac7c51-9a33-41b1-99bf-53ed2efce9c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:04.100534 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ac7c51-9a33-41b1-99bf-53ed2efce9c5 None admin] REQUEST_METHOD: `POST` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:04.100816 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ac7c51-9a33-41b1-99bf-53ed2efce9c5 None admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:04.101091 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ac7c51-9a33-41b1-99bf-53ed2efce9c5 None admin] PATH_INFO: `/v3/users` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:04.101814 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6ac7c51-9a33-41b1-99bf-53ed2efce9c5 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:33:04.103235 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6ac7c51-9a33-41b1-99bf-53ed2efce9c5 None admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:33:04.132245 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 28/61] 10.222.0.13 () {60 vars in 1132 bytes} [Wed Aug 31 09:33:03 2022] POST /identity/v3/users => generated 275 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 09:33:07.479946 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c67e08-cd00-48a4-8146-38e4ff89d894 None None] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:07.480992 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c67e08-cd00-48a4-8146-38e4ff89d894 None None] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:07.481515 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c67e08-cd00-48a4-8146-38e4ff89d894 None None] PATH_INFO: `/` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:07.483099 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 34/62] 10.222.0.13 () {54 vars in 927 bytes} [Wed Aug 31 09:33:07 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:33:07.487004 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3085baa-c088-49b3-b26d-4297d8dbd53f None None] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:07.487868 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3085baa-c088-49b3-b26d-4297d8dbd53f None None] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:07.488368 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3085baa-c088-49b3-b26d-4297d8dbd53f None None] PATH_INFO: `/` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:07.489974 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 29/63] 10.222.0.13 () {54 vars in 927 bytes} [Wed Aug 31 09:33:07 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:33:07.494282 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77187e1-c510-45c8-8841-c1286232b14f None None] REQUEST_METHOD: `POST` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:07.494571 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77187e1-c510-45c8-8841-c1286232b14f None None] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:07.494846 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77187e1-c510-45c8-8841-c1286232b14f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:07.498671 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a15f07-a2ad-45cd-955e-d9e05955aa6f None None] REQUEST_METHOD: `POST` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:07.498671 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a15f07-a2ad-45cd-955e-d9e05955aa6f None None] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:07.498671 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a15f07-a2ad-45cd-955e-d9e05955aa6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:07.518709 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.auth.core [None req-b7a15f07-a2ad-45cd-955e-d9e05955aa6f None None] MFA Rules not processed for user `fd836cf590a145df9acf19cf3ae2ac56`. Rule list: `[]` (Enabled: `True`). {{(pid=88113) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:33:07.520725 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.fernet_utils [None req-b7a15f07-a2ad-45cd-955e-d9e05955aa6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:33:07.530533 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.auth.core [None req-e77187e1-c510-45c8-8841-c1286232b14f None None] MFA Rules not processed for user `fd836cf590a145df9acf19cf3ae2ac56`. Rule list: `[]` (Enabled: `True`). {{(pid=88114) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:33:07.531797 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.fernet_utils [None req-e77187e1-c510-45c8-8841-c1286232b14f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88114) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:33:07.570678 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=100 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:33:07.613544 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 30/64] 10.222.0.13 () {58 vars in 1024 bytes} [Wed Aug 31 09:33:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 09:33:07.615106 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2c19e9-ce21-4400-b988-dbfaca3a04af None None] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:07.615539 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2c19e9-ce21-4400-b988-dbfaca3a04af None None] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:07.615983 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2c19e9-ce21-4400-b988-dbfaca3a04af None None] PATH_INFO: `/` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:07.617460 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 31/65] 10.222.0.13 () {54 vars in 927 bytes} [Wed Aug 31 09:33:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:33:07.627283 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a94b97d-058e-4193-a73a-0411a9d0810d None None] REQUEST_METHOD: `POST` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:07.627955 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a94b97d-058e-4193-a73a-0411a9d0810d None None] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:07.628330 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a94b97d-058e-4193-a73a-0411a9d0810d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:07.632954 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 35/66] 10.222.0.13 () {58 vars in 1024 bytes} [Wed Aug 31 09:33:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 09:33:07.664476 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.auth.core [None req-4a94b97d-058e-4193-a73a-0411a9d0810d None None] MFA Rules not processed for user `fd836cf590a145df9acf19cf3ae2ac56`. Rule list: `[]` (Enabled: `True`). {{(pid=88113) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:33:07.665716 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.fernet_utils [None req-4a94b97d-058e-4193-a73a-0411a9d0810d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:33:07.773905 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43629e1e-7a0f-4038-941c-69cf6b715937 None None] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:07.774107 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43629e1e-7a0f-4038-941c-69cf6b715937 None None] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:07.774609 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43629e1e-7a0f-4038-941c-69cf6b715937 None None] PATH_INFO: `/` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:07.776180 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 36/67] 10.222.0.13 () {54 vars in 927 bytes} [Wed Aug 31 09:33:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 31 09:33:07.779056 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 32/68] 10.222.0.13 () {58 vars in 1024 bytes} [Wed Aug 31 09:33:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 09:33:07.784218 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae288fa-838a-44cb-bd61-297d1829441e None None] REQUEST_METHOD: `POST` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:07.784791 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae288fa-838a-44cb-bd61-297d1829441e None None] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:07.785266 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae288fa-838a-44cb-bd61-297d1829441e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:07.820614 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.auth.core [None req-9ae288fa-838a-44cb-bd61-297d1829441e None None] MFA Rules not processed for user `fd836cf590a145df9acf19cf3ae2ac56`. Rule list: `[]` (Enabled: `True`). {{(pid=88114) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:33:07.822241 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.fernet_utils [None req-9ae288fa-838a-44cb-bd61-297d1829441e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88114) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:33:07.919704 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 37/69] 10.222.0.13 () {58 vars in 1024 bytes} [Wed Aug 31 09:33:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 09:33:07.945937 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a94b97d-058e-4193-a73a-0411a9d0810d None None] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:07.950038 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:33:07.965407 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae288fa-838a-44cb-bd61-297d1829441e None None] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:08.060705 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a94b97d-058e-4193-a73a-0411a9d0810d None None] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:08.099376 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f25442-e4e0-4304-80be-006bc5b6eee0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:08.100035 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f25442-e4e0-4304-80be-006bc5b6eee0 None admin] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:08.100319 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f25442-e4e0-4304-80be-006bc5b6eee0 None admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:08.100598 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f25442-e4e0-4304-80be-006bc5b6eee0 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:08.111886 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4f25442-e4e0-4304-80be-006bc5b6eee0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 09:33:08.115535 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=100 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:33:08.116870 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application [None req-e4f25442-e4e0-4304-80be-006bc5b6eee0 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 09:33:08.116870 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:33:08.116870 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:33:08.116870 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:33:08.116870 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:33:08.116870 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:33:08.116870 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:33:08.116870 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:33:08.116870 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:33:08.116870 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:33:08.116870 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:33:08.116870 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:33:08.116870 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 09:33:08.116870 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 09:33:08.116870 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 09:33:08.116870 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 09:33:08.117724 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 09:33:08.117724 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 09:33:08.117724 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 09:33:08.117724 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 09:33:08.117724 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:33:08.117724 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:33:08.117724 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:33:08.117724 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:33:08.117724 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:33:08.117724 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:33:08.117724 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:33:08.117724 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application with Lock( Aug 31 09:33:08.117724 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:33:08.117724 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:33:08.117724 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:33:08.117724 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:33:08.119268 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:33:08.119268 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:33:08.119268 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:33:08.119268 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:33:08.119268 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 09:33:08.119268 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 09:33:08.119268 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 09:33:08.119268 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 09:33:08.119268 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 09:33:08.119268 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 09:33:08.119268 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 09:33:08.119268 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application Aug 31 09:33:08.119268 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 33/70] 10.222.0.13 () {56 vars in 1096 bytes} [Wed Aug 31 09:33:07 2022] GET /identity/v3/roles/admin => generated 83 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 09:33:08.123328 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae288fa-838a-44cb-bd61-297d1829441e None None] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:08.126986 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f25442-e4e0-4304-80be-006bc5b6eee0 None admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:08.175897 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf4d6e9-f0e7-4e68-ac38-34fa8827aa7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:08.176845 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf4d6e9-f0e7-4e68-ac38-34fa8827aa7d None admin] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:08.177269 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf4d6e9-f0e7-4e68-ac38-34fa8827aa7d None admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:08.177900 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf4d6e9-f0e7-4e68-ac38-34fa8827aa7d None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:08.207533 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbf4d6e9-f0e7-4e68-ac38-34fa8827aa7d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 09:33:08.213377 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application [None req-dbf4d6e9-f0e7-4e68-ac38-34fa8827aa7d None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 31 09:33:08.213377 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:33:08.213377 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:33:08.213377 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:33:08.213377 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:33:08.213377 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:33:08.213377 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:33:08.213377 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:33:08.213377 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:33:08.213377 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:33:08.213377 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:33:08.213377 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:33:08.213377 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 09:33:08.213377 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 09:33:08.213377 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 09:33:08.213377 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 09:33:08.214290 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 09:33:08.214290 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 09:33:08.214290 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 09:33:08.214290 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 09:33:08.214290 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:33:08.214290 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:33:08.214290 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:33:08.214290 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:33:08.214290 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:33:08.214290 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:33:08.214290 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:33:08.214290 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application with Lock( Aug 31 09:33:08.214290 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:33:08.214290 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:33:08.214290 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:33:08.214290 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:33:08.215053 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:33:08.215053 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:33:08.215053 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:33:08.215053 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:33:08.215053 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 09:33:08.215053 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 09:33:08.215053 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 09:33:08.215053 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 09:33:08.215053 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 09:33:08.215053 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 09:33:08.215053 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 31 09:33:08.215053 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application Aug 31 09:33:08.215053 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 38/71] 10.222.0.13 () {56 vars in 1114 bytes} [Wed Aug 31 09:33:07 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 255 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:33:08.216230 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf4d6e9-f0e7-4e68-ac38-34fa8827aa7d None admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:08.229364 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=100 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:33:08.291203 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f25442-e4e0-4304-80be-006bc5b6eee0 None admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:08.334244 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-368d2988-78a2-4895-a16e-967f5b97d45f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:08.334777 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368d2988-78a2-4895-a16e-967f5b97d45f None admin] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:08.335063 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368d2988-78a2-4895-a16e-967f5b97d45f None admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:08.335338 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368d2988-78a2-4895-a16e-967f5b97d45f None admin] PATH_INFO: `/v3/roles` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:08.336004 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-368d2988-78a2-4895-a16e-967f5b97d45f None admin] RBAC: Adding query filter params (name=admin) {{(pid=88113) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:33:08.336301 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-368d2988-78a2-4895-a16e-967f5b97d45f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:33:08.337723 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-368d2988-78a2-4895-a16e-967f5b97d45f None admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:33:08.346258 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 34/72] 10.222.0.13 () {56 vars in 1099 bytes} [Wed Aug 31 09:33:08 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:33:08.349598 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-368d2988-78a2-4895-a16e-967f5b97d45f None admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:08.368547 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf4d6e9-f0e7-4e68-ac38-34fa8827aa7d None admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:08.405313 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b11077d-94d9-4a94-a81f-71dc04a5ea1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:08.405970 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b11077d-94d9-4a94-a81f-71dc04a5ea1a None admin] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:08.406248 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b11077d-94d9-4a94-a81f-71dc04a5ea1a None admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:08.406514 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b11077d-94d9-4a94-a81f-71dc04a5ea1a None admin] PATH_INFO: `/v3/roles/member` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:08.416923 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b11077d-94d9-4a94-a81f-71dc04a5ea1a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 09:33:08.421787 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application [None req-7b11077d-94d9-4a94-a81f-71dc04a5ea1a None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 31 09:33:08.421787 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:33:08.421787 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:33:08.421787 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:33:08.421787 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:33:08.421787 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:33:08.421787 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:33:08.421787 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:33:08.421787 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:33:08.421787 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:33:08.421787 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:33:08.421787 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:33:08.421787 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 09:33:08.421787 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 09:33:08.421787 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 09:33:08.421787 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 09:33:08.422664 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 09:33:08.422664 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 09:33:08.422664 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 09:33:08.422664 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 09:33:08.422664 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:33:08.422664 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:33:08.422664 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:33:08.422664 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:33:08.422664 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:33:08.422664 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:33:08.422664 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:33:08.422664 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application with Lock( Aug 31 09:33:08.422664 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:33:08.422664 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:33:08.422664 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:33:08.422664 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:33:08.423449 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:33:08.423449 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:33:08.423449 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:33:08.423449 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:33:08.423449 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 09:33:08.423449 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 09:33:08.423449 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 09:33:08.423449 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 09:33:08.423449 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 09:33:08.423449 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 09:33:08.423449 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 31 09:33:08.423449 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application Aug 31 09:33:08.423449 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 39/73] 10.222.0.13 () {56 vars in 1099 bytes} [Wed Aug 31 09:33:08 2022] GET /identity/v3/roles/member => generated 84 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:33:08.424415 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b11077d-94d9-4a94-a81f-71dc04a5ea1a None admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:08.442458 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=100 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:33:08.460947 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-368d2988-78a2-4895-a16e-967f5b97d45f None admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:08.488872 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=100 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:33:08.498467 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d7fa0a-75c7-41b4-8587-d9f48a58d525 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:08.499076 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d7fa0a-75c7-41b4-8587-d9f48a58d525 None admin] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:08.499355 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d7fa0a-75c7-41b4-8587-d9f48a58d525 None admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:08.499631 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d7fa0a-75c7-41b4-8587-d9f48a58d525 None admin] PATH_INFO: `/v3/roles` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:08.500294 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42d7fa0a-75c7-41b4-8587-d9f48a58d525 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=88113) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:33:08.500599 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42d7fa0a-75c7-41b4-8587-d9f48a58d525 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:33:08.502119 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42d7fa0a-75c7-41b4-8587-d9f48a58d525 None admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:33:08.510282 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 35/74] 10.222.0.13 () {56 vars in 1111 bytes} [Wed Aug 31 09:33:08 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:33:08.511556 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454d6f84-18bc-43f4-9013-0d88e51cb623 None None] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:08.511843 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454d6f84-18bc-43f4-9013-0d88e51cb623 None None] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:08.512114 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454d6f84-18bc-43f4-9013-0d88e51cb623 None None] PATH_INFO: `/` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:08.513512 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 36/75] 10.222.0.13 () {54 vars in 927 bytes} [Wed Aug 31 09:33:08 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:33:08.514020 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-454d6f84-18bc-43f4-9013-0d88e51cb623 None None] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:08.537748 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b11077d-94d9-4a94-a81f-71dc04a5ea1a None admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:08.572613 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b316e98-2da4-4e82-afbf-db4a643763d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:08.573251 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b316e98-2da4-4e82-afbf-db4a643763d5 None admin] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:08.573558 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b316e98-2da4-4e82-afbf-db4a643763d5 None admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:08.573829 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b316e98-2da4-4e82-afbf-db4a643763d5 None admin] PATH_INFO: `/v3/roles/member` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:08.584637 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b316e98-2da4-4e82-afbf-db4a643763d5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 09:33:08.589535 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application [None req-0b316e98-2da4-4e82-afbf-db4a643763d5 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 31 09:33:08.589535 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:33:08.589535 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:33:08.589535 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:33:08.589535 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:33:08.589535 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:33:08.589535 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:33:08.589535 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:33:08.589535 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:33:08.589535 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:33:08.589535 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:33:08.589535 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:33:08.589535 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 09:33:08.589535 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 09:33:08.589535 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 09:33:08.589535 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 09:33:08.590402 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 09:33:08.590402 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 09:33:08.590402 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 09:33:08.590402 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 09:33:08.590402 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:33:08.590402 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:33:08.590402 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:33:08.590402 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:33:08.590402 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:33:08.590402 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:33:08.590402 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:33:08.590402 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application with Lock( Aug 31 09:33:08.590402 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:33:08.590402 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:33:08.590402 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:33:08.590402 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:33:08.591180 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:33:08.591180 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:33:08.591180 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:33:08.591180 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:33:08.591180 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 09:33:08.591180 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 09:33:08.591180 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 09:33:08.591180 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 09:33:08.591180 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 09:33:08.591180 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 09:33:08.591180 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 31 09:33:08.591180 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application Aug 31 09:33:08.591180 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 40/76] 10.222.0.13 () {56 vars in 1099 bytes} [Wed Aug 31 09:33:08 2022] GET /identity/v3/roles/member => generated 84 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:33:08.591822 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b316e98-2da4-4e82-afbf-db4a643763d5 None admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:08.624711 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-454d6f84-18bc-43f4-9013-0d88e51cb623 None None] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:08.661784 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ec2c99-87fc-40d2-8d8e-3b3209c80082 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:08.662425 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ec2c99-87fc-40d2-8d8e-3b3209c80082 None admin] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:08.662709 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ec2c99-87fc-40d2-8d8e-3b3209c80082 None admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:08.662991 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ec2c99-87fc-40d2-8d8e-3b3209c80082 None admin] PATH_INFO: `/v3/users/fd836cf590a145df9acf19cf3ae2ac56` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:08.664189 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14ec2c99-87fc-40d2-8d8e-3b3209c80082 None admin] RBAC: Authorizing `identity:get_user(user_id=fd836cf590a145df9acf19cf3ae2ac56)` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:33:08.665690 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14ec2c99-87fc-40d2-8d8e-3b3209c80082 None admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:33:08.667345 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 37/77] 10.222.0.13 () {56 vars in 1177 bytes} [Wed Aug 31 09:33:08 2022] GET /identity/v3/users/fd836cf590a145df9acf19cf3ae2ac56 => generated 247 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:33:08.668120 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ec2c99-87fc-40d2-8d8e-3b3209c80082 None admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:08.670691 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=100 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:33:08.700405 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b316e98-2da4-4e82-afbf-db4a643763d5 None admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:08.731450 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=100 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:33:08.738770 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55818ae9-e14a-47d9-9106-fc5ca97026d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:08.739386 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55818ae9-e14a-47d9-9106-fc5ca97026d7 None admin] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:08.739670 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55818ae9-e14a-47d9-9106-fc5ca97026d7 None admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:08.739939 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55818ae9-e14a-47d9-9106-fc5ca97026d7 None admin] PATH_INFO: `/v3/roles` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:08.740615 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55818ae9-e14a-47d9-9106-fc5ca97026d7 None admin] RBAC: Adding query filter params (name=member) {{(pid=88114) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:33:08.740916 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55818ae9-e14a-47d9-9106-fc5ca97026d7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:33:08.742532 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55818ae9-e14a-47d9-9106-fc5ca97026d7 None admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:33:08.750918 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 41/78] 10.222.0.13 () {56 vars in 1101 bytes} [Wed Aug 31 09:33:08 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:33:08.753747 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed88413-96c6-45d6-b3ba-cce98f2f8f49 None None] REQUEST_METHOD: `POST` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:08.754090 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed88413-96c6-45d6-b3ba-cce98f2f8f49 None None] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:08.754368 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed88413-96c6-45d6-b3ba-cce98f2f8f49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:08.773147 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.auth.core [None req-8ed88413-96c6-45d6-b3ba-cce98f2f8f49 None None] MFA Rules not processed for user `fd836cf590a145df9acf19cf3ae2ac56`. Rule list: `[]` (Enabled: `True`). {{(pid=88114) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:33:08.774398 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.fernet_utils [None req-8ed88413-96c6-45d6-b3ba-cce98f2f8f49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88114) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:33:08.778559 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ec2c99-87fc-40d2-8d8e-3b3209c80082 None admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:08.813559 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c5c139-39d8-4c66-ba90-545e63fadc20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:08.814288 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c5c139-39d8-4c66-ba90-545e63fadc20 None admin] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:08.814779 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c5c139-39d8-4c66-ba90-545e63fadc20 None admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:08.815068 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c5c139-39d8-4c66-ba90-545e63fadc20 None admin] PATH_INFO: `/v3/users/1dc7ae39ad83425fb2f7ddb6c98b0a20` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:08.831547 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7c5c139-39d8-4c66-ba90-545e63fadc20 None admin] RBAC: Authorizing `identity:get_user(user_id=1dc7ae39ad83425fb2f7ddb6c98b0a20)` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:33:08.833603 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7c5c139-39d8-4c66-ba90-545e63fadc20 None admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:33:08.840625 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 38/79] 10.222.0.13 () {56 vars in 1177 bytes} [Wed Aug 31 09:33:08 2022] GET /identity/v3/users/1dc7ae39ad83425fb2f7ddb6c98b0a20 => generated 275 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:33:08.840625 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c5c139-39d8-4c66-ba90-545e63fadc20 None admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:08.852392 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 42/80] 10.222.0.13 () {58 vars in 1024 bytes} [Wed Aug 31 09:33:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 09:33:08.853265 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed88413-96c6-45d6-b3ba-cce98f2f8f49 None None] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:08.948577 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=100 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:33:08.966105 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c5c139-39d8-4c66-ba90-545e63fadc20 None admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:08.983506 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=100 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:33:09.005125 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed88413-96c6-45d6-b3ba-cce98f2f8f49 None None] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:09.021271 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcdf793b-1827-45e9-80ff-f134dafae90d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:09.022280 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcdf793b-1827-45e9-80ff-f134dafae90d None admin] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:09.022741 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcdf793b-1827-45e9-80ff-f134dafae90d None admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:09.023216 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcdf793b-1827-45e9-80ff-f134dafae90d None admin] PATH_INFO: `/v3/roles` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:09.024344 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcdf793b-1827-45e9-80ff-f134dafae90d None admin] RBAC: Adding query filter params (name=member) {{(pid=88113) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:33:09.024877 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcdf793b-1827-45e9-80ff-f134dafae90d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:33:09.027231 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcdf793b-1827-45e9-80ff-f134dafae90d None admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:33:09.040735 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 39/81] 10.222.0.13 () {56 vars in 1101 bytes} [Wed Aug 31 09:33:08 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:33:09.042135 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcdf793b-1827-45e9-80ff-f134dafae90d None admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:09.070370 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-145ca8fb-591f-4c30-b470-522bce7f8b9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:09.071725 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145ca8fb-591f-4c30-b470-522bce7f8b9a None admin] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:09.072452 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145ca8fb-591f-4c30-b470-522bce7f8b9a None admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:09.073148 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145ca8fb-591f-4c30-b470-522bce7f8b9a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:09.150455 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-145ca8fb-591f-4c30-b470-522bce7f8b9a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:33:09.152170 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-145ca8fb-591f-4c30-b470-522bce7f8b9a None admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:33:09.166878 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcdf793b-1827-45e9-80ff-f134dafae90d None admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:09.202965 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c116c260-eb97-4c9b-b39b-328119300da7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:09.203608 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c116c260-eb97-4c9b-b39b-328119300da7 None admin] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:09.203898 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c116c260-eb97-4c9b-b39b-328119300da7 None admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:09.204176 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c116c260-eb97-4c9b-b39b-328119300da7 None admin] PATH_INFO: `/v3/users/1dc7ae39ad83425fb2f7ddb6c98b0a20` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:09.205642 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c116c260-eb97-4c9b-b39b-328119300da7 None admin] RBAC: Authorizing `identity:get_user(user_id=1dc7ae39ad83425fb2f7ddb6c98b0a20)` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:33:09.207126 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c116c260-eb97-4c9b-b39b-328119300da7 None admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:33:09.209000 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 40/82] 10.222.0.13 () {56 vars in 1177 bytes} [Wed Aug 31 09:33:09 2022] GET /identity/v3/users/1dc7ae39ad83425fb2f7ddb6c98b0a20 => generated 275 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:33:09.209605 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c116c260-eb97-4c9b-b39b-328119300da7 None admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:09.225326 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=100 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:33:09.257053 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=100 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:33:09.268751 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 43/83] 10.222.0.13 () {58 vars in 1282 bytes} [Wed Aug 31 09:33:08 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 411 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 31 09:33:09.268751 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-145ca8fb-591f-4c30-b470-522bce7f8b9a None admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:09.321666 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c116c260-eb97-4c9b-b39b-328119300da7 None admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:09.358434 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e78344a3-5dde-4a43-b973-35746c7b037f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:09.359052 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78344a3-5dde-4a43-b973-35746c7b037f None admin] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:09.359334 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78344a3-5dde-4a43-b973-35746c7b037f None admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:09.359611 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78344a3-5dde-4a43-b973-35746c7b037f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:09.366884 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-145ca8fb-591f-4c30-b470-522bce7f8b9a None admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:09.405914 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc43d737-ed92-48d6-a5fb-58b3741bea7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:09.406477 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc43d737-ed92-48d6-a5fb-58b3741bea7e None admin] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:09.406757 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc43d737-ed92-48d6-a5fb-58b3741bea7e None admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:09.407034 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc43d737-ed92-48d6-a5fb-58b3741bea7e None admin] PATH_INFO: `/v3/users/1dc7ae39ad83425fb2f7ddb6c98b0a20` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:09.408870 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc43d737-ed92-48d6-a5fb-58b3741bea7e None admin] RBAC: Authorizing `identity:get_user(user_id=1dc7ae39ad83425fb2f7ddb6c98b0a20)` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:33:09.410609 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc43d737-ed92-48d6-a5fb-58b3741bea7e None admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:33:09.412281 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 44/84] 10.222.0.13 () {56 vars in 1177 bytes} [Wed Aug 31 09:33:09 2022] GET /identity/v3/users/1dc7ae39ad83425fb2f7ddb6c98b0a20 => generated 275 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:33:09.413123 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc43d737-ed92-48d6-a5fb-58b3741bea7e None admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:09.435191 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e78344a3-5dde-4a43-b973-35746c7b037f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:33:09.436710 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e78344a3-5dde-4a43-b973-35746c7b037f None admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:33:09.448111 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=100 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:33:09.488380 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=100 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:33:09.521945 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc43d737-ed92-48d6-a5fb-58b3741bea7e None admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:09.547188 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 41/85] 10.222.0.13 () {58 vars in 1282 bytes} [Wed Aug 31 09:33:09 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 338 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 31 09:33:09.548100 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e78344a3-5dde-4a43-b973-35746c7b037f None admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:09.560250 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ec0a91-5c86-4779-8097-8b67d7d70a79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:09.560905 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ec0a91-5c86-4779-8097-8b67d7d70a79 None admin] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:09.561188 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ec0a91-5c86-4779-8097-8b67d7d70a79 None admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:09.561491 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ec0a91-5c86-4779-8097-8b67d7d70a79 None admin] PATH_INFO: `/v3/domains/default` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:09.562725 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3ec0a91-5c86-4779-8097-8b67d7d70a79 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:33:09.564286 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3ec0a91-5c86-4779-8097-8b67d7d70a79 None admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:33:09.566931 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 45/86] 10.222.0.13 () {56 vars in 1108 bytes} [Wed Aug 31 09:33:09 2022] GET /identity/v3/domains/default => generated 199 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:33:09.566931 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ec0a91-5c86-4779-8097-8b67d7d70a79 None admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:09.660762 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e78344a3-5dde-4a43-b973-35746c7b037f None admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:09.676715 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ec0a91-5c86-4779-8097-8b67d7d70a79 None admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:09.678620 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=100 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:33:09.700660 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc146af-13ac-4d2f-ab9c-b259af7fc192 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:09.701301 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc146af-13ac-4d2f-ab9c-b259af7fc192 None admin] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:09.701613 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc146af-13ac-4d2f-ab9c-b259af7fc192 None admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:09.701890 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc146af-13ac-4d2f-ab9c-b259af7fc192 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:09.719037 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-619da994-4383-413e-b6e4-2f7bc2b232f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:09.719721 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619da994-4383-413e-b6e4-2f7bc2b232f5 None admin] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:09.719825 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=100 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:33:09.720072 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619da994-4383-413e-b6e4-2f7bc2b232f5 None admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:09.720387 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619da994-4383-413e-b6e4-2f7bc2b232f5 None admin] PATH_INFO: `/v3/projects/1001686acf834f2a8e65b3fc8369f4f6` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:09.722095 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-619da994-4383-413e-b6e4-2f7bc2b232f5 None admin] RBAC: Authorizing `identity:get_project(project_id=1001686acf834f2a8e65b3fc8369f4f6)` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:33:09.723636 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-619da994-4383-413e-b6e4-2f7bc2b232f5 None admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:33:09.725689 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 46/87] 10.222.0.13 () {56 vars in 1186 bytes} [Wed Aug 31 09:33:09 2022] GET /identity/v3/projects/1001686acf834f2a8e65b3fc8369f4f6 => generated 298 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:33:09.726651 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-619da994-4383-413e-b6e4-2f7bc2b232f5 None admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:09.783127 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bc146af-13ac-4d2f-ab9c-b259af7fc192 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:33:09.784960 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bc146af-13ac-4d2f-ab9c-b259af7fc192 None admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:33:09.840493 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-619da994-4383-413e-b6e4-2f7bc2b232f5 None admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:09.876416 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00083725-42df-4f95-9e2d-e41ee58f216a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:09.877877 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00083725-42df-4f95-9e2d-e41ee58f216a None admin] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:09.877877 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00083725-42df-4f95-9e2d-e41ee58f216a None admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:09.878275 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00083725-42df-4f95-9e2d-e41ee58f216a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:09.889495 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 42/88] 10.222.0.13 () {58 vars in 1282 bytes} [Wed Aug 31 09:33:09 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 342 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 31 09:33:09.890372 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc146af-13ac-4d2f-ab9c-b259af7fc192 None admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:09.925737 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=100 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:33:09.938536 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=100 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:33:09.957965 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00083725-42df-4f95-9e2d-e41ee58f216a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:33:09.959478 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00083725-42df-4f95-9e2d-e41ee58f216a None admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:33:09.994795 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc146af-13ac-4d2f-ab9c-b259af7fc192 None admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:10.031803 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d4bbbd7-37a9-4de7-a208-1d232e7bc9a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:10.032481 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4bbbd7-37a9-4de7-a208-1d232e7bc9a1 None admin] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:10.032766 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4bbbd7-37a9-4de7-a208-1d232e7bc9a1 None admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:10.033064 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4bbbd7-37a9-4de7-a208-1d232e7bc9a1 None admin] PATH_INFO: `/v3/projects/1001686acf834f2a8e65b3fc8369f4f6` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:10.034293 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d4bbbd7-37a9-4de7-a208-1d232e7bc9a1 None admin] RBAC: Authorizing `identity:get_project(project_id=1001686acf834f2a8e65b3fc8369f4f6)` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:33:10.035812 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d4bbbd7-37a9-4de7-a208-1d232e7bc9a1 None admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:33:10.037558 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 43/89] 10.222.0.13 () {56 vars in 1186 bytes} [Wed Aug 31 09:33:09 2022] GET /identity/v3/projects/1001686acf834f2a8e65b3fc8369f4f6 => generated 298 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:33:10.038327 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d4bbbd7-37a9-4de7-a208-1d232e7bc9a1 None admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:10.067562 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 47/90] 10.222.0.13 () {58 vars in 1282 bytes} [Wed Aug 31 09:33:09 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 342 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 31 09:33:10.068727 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00083725-42df-4f95-9e2d-e41ee58f216a None admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:10.151922 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=100 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:33:10.152127 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d4bbbd7-37a9-4de7-a208-1d232e7bc9a1 None admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:10.166446 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=100 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:33:10.188487 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00083725-42df-4f95-9e2d-e41ee58f216a None admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:10.189149 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c06c522-4cac-48b7-a414-5449a0b47b66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:10.190010 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c06c522-4cac-48b7-a414-5449a0b47b66 None admin] REQUEST_METHOD: `POST` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:10.190121 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c06c522-4cac-48b7-a414-5449a0b47b66 None admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:10.190346 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c06c522-4cac-48b7-a414-5449a0b47b66 None admin] PATH_INFO: `/v3/users` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:10.191005 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c06c522-4cac-48b7-a414-5449a0b47b66 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:33:10.192468 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c06c522-4cac-48b7-a414-5449a0b47b66 None admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:33:10.215370 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 44/91] 10.222.0.13 () {60 vars in 1132 bytes} [Wed Aug 31 09:33:10 2022] POST /identity/v3/users => generated 289 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 09:33:10.216215 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c06c522-4cac-48b7-a414-5449a0b47b66 None admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:10.226078 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8112d67b-ba64-41bb-ad0e-56662dc0f3c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:10.226713 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8112d67b-ba64-41bb-ad0e-56662dc0f3c6 None admin] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:10.227651 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8112d67b-ba64-41bb-ad0e-56662dc0f3c6 None admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:10.227959 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8112d67b-ba64-41bb-ad0e-56662dc0f3c6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:10.228728 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8112d67b-ba64-41bb-ad0e-56662dc0f3c6 None admin] RBAC: Adding query filter params (role.id=5d4ee63a3add47fc9a6b8b16d39e0a1f, scope.project.id=1001686acf834f2a8e65b3fc8369f4f6, user.id=fd836cf590a145df9acf19cf3ae2ac56) {{(pid=88114) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:33:10.229041 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8112d67b-ba64-41bb-ad0e-56662dc0f3c6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:33:10.231193 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8112d67b-ba64-41bb-ad0e-56662dc0f3c6 None admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:33:10.239217 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 48/92] 10.222.0.13 () {56 vars in 1374 bytes} [Wed Aug 31 09:33:10 2022] GET /identity/v3/role_assignments?user.id=fd836cf590a145df9acf19cf3ae2ac56&scope.project.id=1001686acf834f2a8e65b3fc8369f4f6&role.id=5d4ee63a3add47fc9a6b8b16d39e0a1f => generated 259 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:33:10.239932 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8112d67b-ba64-41bb-ad0e-56662dc0f3c6 None admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:10.385835 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c06c522-4cac-48b7-a414-5449a0b47b66 None admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:10.412696 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8112d67b-ba64-41bb-ad0e-56662dc0f3c6 None admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:10.434265 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f611f7-a1eb-4305-9691-fbfa701fb65d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:10.434930 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f611f7-a1eb-4305-9691-fbfa701fb65d None admin] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:10.435214 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f611f7-a1eb-4305-9691-fbfa701fb65d None admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:10.435494 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f611f7-a1eb-4305-9691-fbfa701fb65d None admin] PATH_INFO: `/v3/projects/1001686acf834f2a8e65b3fc8369f4f6` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:10.437644 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73f611f7-a1eb-4305-9691-fbfa701fb65d None admin] RBAC: Authorizing `identity:get_project(project_id=1001686acf834f2a8e65b3fc8369f4f6)` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:33:10.439735 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73f611f7-a1eb-4305-9691-fbfa701fb65d None admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:33:10.441745 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 45/93] 10.222.0.13 () {56 vars in 1186 bytes} [Wed Aug 31 09:33:10 2022] GET /identity/v3/projects/1001686acf834f2a8e65b3fc8369f4f6 => generated 298 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:33:10.442526 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f611f7-a1eb-4305-9691-fbfa701fb65d None admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:10.453825 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea198e4e-1577-4024-87cd-2e4f8cc68c67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:10.454545 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea198e4e-1577-4024-87cd-2e4f8cc68c67 None admin] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:10.455568 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea198e4e-1577-4024-87cd-2e4f8cc68c67 None admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:10.455898 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea198e4e-1577-4024-87cd-2e4f8cc68c67 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:10.456711 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea198e4e-1577-4024-87cd-2e4f8cc68c67 None admin] RBAC: Adding query filter params (role.id=889857b056624d47b572d4a80ecc3f8e, scope.project.id=1001686acf834f2a8e65b3fc8369f4f6, user.id=1dc7ae39ad83425fb2f7ddb6c98b0a20) {{(pid=88114) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:33:10.457086 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea198e4e-1577-4024-87cd-2e4f8cc68c67 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:33:10.458820 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea198e4e-1577-4024-87cd-2e4f8cc68c67 None admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:33:10.460752 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=100 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:33:10.467589 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 49/94] 10.222.0.13 () {56 vars in 1374 bytes} [Wed Aug 31 09:33:10 2022] GET /identity/v3/role_assignments?user.id=1dc7ae39ad83425fb2f7ddb6c98b0a20&scope.project.id=1001686acf834f2a8e65b3fc8369f4f6&role.id=889857b056624d47b572d4a80ecc3f8e => generated 259 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:33:10.468482 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea198e4e-1577-4024-87cd-2e4f8cc68c67 None admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:10.523168 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:33:10.615653 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea198e4e-1577-4024-87cd-2e4f8cc68c67 None admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:10.624660 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f611f7-a1eb-4305-9691-fbfa701fb65d None admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:10.679641 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91643f2d-d51f-4157-808a-2af63dd30530 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:10.680309 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da26099-5b8c-4dfc-bd70-74ea8e11ea23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:10.680470 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91643f2d-d51f-4157-808a-2af63dd30530 None admin] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:10.680669 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91643f2d-d51f-4157-808a-2af63dd30530 None admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:10.680861 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91643f2d-d51f-4157-808a-2af63dd30530 None admin] PATH_INFO: `/v3/projects/9f41d93b9d96487f867195c8bdd23cbc` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:10.681701 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da26099-5b8c-4dfc-bd70-74ea8e11ea23 None admin] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:10.682048 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91643f2d-d51f-4157-808a-2af63dd30530 None admin] RBAC: Authorizing `identity:get_project(project_id=9f41d93b9d96487f867195c8bdd23cbc)` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:33:10.682418 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da26099-5b8c-4dfc-bd70-74ea8e11ea23 None admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:10.682891 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da26099-5b8c-4dfc-bd70-74ea8e11ea23 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:10.683539 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91643f2d-d51f-4157-808a-2af63dd30530 None admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:33:10.684060 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0da26099-5b8c-4dfc-bd70-74ea8e11ea23 None admin] RBAC: Adding query filter params (role.id=6c99f5d9f1f243eaa134d8540f21095d, scope.project.id=1001686acf834f2a8e65b3fc8369f4f6, user.id=1dc7ae39ad83425fb2f7ddb6c98b0a20) {{(pid=88114) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:33:10.684573 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0da26099-5b8c-4dfc-bd70-74ea8e11ea23 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:33:10.686536 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 46/95] 10.222.0.13 () {56 vars in 1186 bytes} [Wed Aug 31 09:33:10 2022] GET /identity/v3/projects/9f41d93b9d96487f867195c8bdd23cbc => generated 312 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:33:10.687927 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0da26099-5b8c-4dfc-bd70-74ea8e11ea23 None admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:33:10.692785 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91643f2d-d51f-4157-808a-2af63dd30530 None admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:10.696269 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 50/96] 10.222.0.13 () {56 vars in 1374 bytes} [Wed Aug 31 09:33:10 2022] GET /identity/v3/role_assignments?user.id=1dc7ae39ad83425fb2f7ddb6c98b0a20&scope.project.id=1001686acf834f2a8e65b3fc8369f4f6&role.id=6c99f5d9f1f243eaa134d8540f21095d => generated 259 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:33:10.829292 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91643f2d-d51f-4157-808a-2af63dd30530 None admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:10.833108 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=100 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:33:10.894522 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f46ed2d1-cbc9-4236-bd60-1be3ed329bfb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:10.896302 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46ed2d1-cbc9-4236-bd60-1be3ed329bfb None admin] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:10.897099 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46ed2d1-cbc9-4236-bd60-1be3ed329bfb None admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:10.897985 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46ed2d1-cbc9-4236-bd60-1be3ed329bfb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:10.899681 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f46ed2d1-cbc9-4236-bd60-1be3ed329bfb None admin] RBAC: Adding query filter params (role.id=6c99f5d9f1f243eaa134d8540f21095d, scope.project.id=9f41d93b9d96487f867195c8bdd23cbc, user.id=1dc7ae39ad83425fb2f7ddb6c98b0a20) {{(pid=88113) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:33:10.900488 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f46ed2d1-cbc9-4236-bd60-1be3ed329bfb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:33:10.903550 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f46ed2d1-cbc9-4236-bd60-1be3ed329bfb None admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:33:10.913367 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 47/97] 10.222.0.13 () {56 vars in 1374 bytes} [Wed Aug 31 09:33:10 2022] GET /identity/v3/role_assignments?user.id=1dc7ae39ad83425fb2f7ddb6c98b0a20&scope.project.id=9f41d93b9d96487f867195c8bdd23cbc&role.id=6c99f5d9f1f243eaa134d8540f21095d => generated 259 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 09:33:13.177589 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408400ad-1b0b-44f5-b730-8bd0aa63cac6 None None] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:13.178095 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408400ad-1b0b-44f5-b730-8bd0aa63cac6 None None] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:13.178521 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408400ad-1b0b-44f5-b730-8bd0aa63cac6 None None] PATH_INFO: `/` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:13.180868 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 51/98] 10.222.0.13 () {54 vars in 927 bytes} [Wed Aug 31 09:33:13 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:33:13.219710 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518f36c9-8971-4ade-ac35-8eec6c5435c7 None None] REQUEST_METHOD: `POST` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:13.220224 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518f36c9-8971-4ade-ac35-8eec6c5435c7 None None] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:13.220322 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518f36c9-8971-4ade-ac35-8eec6c5435c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:13.258675 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.auth.core [None req-518f36c9-8971-4ade-ac35-8eec6c5435c7 None None] MFA Rules not processed for user `fd836cf590a145df9acf19cf3ae2ac56`. Rule list: `[]` (Enabled: `True`). {{(pid=88113) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:33:13.259836 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.fernet_utils [None req-518f36c9-8971-4ade-ac35-8eec6c5435c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:33:13.443263 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 48/99] 10.222.0.13 () {58 vars in 1024 bytes} [Wed Aug 31 09:33:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 224 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 09:33:13.736256 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd00308-dfd9-4738-953d-86aad2086a0b None None] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:13.736523 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd00308-dfd9-4738-953d-86aad2086a0b None None] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:13.736914 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd00308-dfd9-4738-953d-86aad2086a0b None None] PATH_INFO: `/` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:13.738393 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 52/100] 10.222.0.13 () {54 vars in 927 bytes} [Wed Aug 31 09:33:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:33:13.744839 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c648e0d2-3ce5-4fba-97f3-6bfbe9efb245 None None] REQUEST_METHOD: `POST` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:13.744839 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c648e0d2-3ce5-4fba-97f3-6bfbe9efb245 None None] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:13.744839 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c648e0d2-3ce5-4fba-97f3-6bfbe9efb245 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:13.775495 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.auth.core [None req-c648e0d2-3ce5-4fba-97f3-6bfbe9efb245 None None] MFA Rules not processed for user `fd836cf590a145df9acf19cf3ae2ac56`. Rule list: `[]` (Enabled: `True`). {{(pid=88113) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:33:13.777070 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.fernet_utils [None req-c648e0d2-3ce5-4fba-97f3-6bfbe9efb245 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:33:13.874883 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb703c08-713b-4008-b846-d30908ee91b7 None None] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:13.875280 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb703c08-713b-4008-b846-d30908ee91b7 None None] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:13.875553 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb703c08-713b-4008-b846-d30908ee91b7 None None] PATH_INFO: `/` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:13.877683 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 53/101] 10.222.0.13 () {54 vars in 927 bytes} [Wed Aug 31 09:33:13 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:33:13.892829 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 49/102] 10.222.0.13 () {58 vars in 1024 bytes} [Wed Aug 31 09:33:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 363 bytes (3 switches on core 0) Aug 31 09:33:13.918258 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6ca450-3682-44cc-ac1e-a911d52bc961 None None] REQUEST_METHOD: `POST` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:13.918601 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6ca450-3682-44cc-ac1e-a911d52bc961 None None] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:13.918821 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6ca450-3682-44cc-ac1e-a911d52bc961 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:13.947357 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.auth.core [None req-6c6ca450-3682-44cc-ac1e-a911d52bc961 None None] MFA Rules not processed for user `fd836cf590a145df9acf19cf3ae2ac56`. Rule list: `[]` (Enabled: `True`). {{(pid=88114) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:33:13.948608 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.fernet_utils [None req-6c6ca450-3682-44cc-ac1e-a911d52bc961 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88114) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:33:14.044322 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=100 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:33:14.050618 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0dfab06-a4f2-49ae-8341-f1088e3d3d99 None None] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:14.050903 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0dfab06-a4f2-49ae-8341-f1088e3d3d99 None None] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:14.051209 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0dfab06-a4f2-49ae-8341-f1088e3d3d99 None None] PATH_INFO: `/` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:14.052953 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 50/103] 10.222.0.13 () {54 vars in 927 bytes} [Wed Aug 31 09:33:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:33:14.058919 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8effc321-f322-46e8-85b7-15b26ff25dda None None] REQUEST_METHOD: `POST` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:14.059248 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8effc321-f322-46e8-85b7-15b26ff25dda None None] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:14.059473 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8effc321-f322-46e8-85b7-15b26ff25dda None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:14.059777 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 54/104] 10.222.0.13 () {58 vars in 1024 bytes} [Wed Aug 31 09:33:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 09:33:14.091431 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.auth.core [None req-8effc321-f322-46e8-85b7-15b26ff25dda None None] MFA Rules not processed for user `fd836cf590a145df9acf19cf3ae2ac56`. Rule list: `[]` (Enabled: `True`). {{(pid=88113) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:33:14.092665 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.fernet_utils [None req-8effc321-f322-46e8-85b7-15b26ff25dda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:33:14.136746 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=100 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:33:14.206017 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 51/105] 10.222.0.13 () {58 vars in 1024 bytes} [Wed Aug 31 09:33:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 31 09:33:14.244481 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c6ca450-3682-44cc-ac1e-a911d52bc961 None None] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:14.266479 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8effc321-f322-46e8-85b7-15b26ff25dda None None] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:14.440154 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c6ca450-3682-44cc-ac1e-a911d52bc961 None None] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:14.476500 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8effc321-f322-46e8-85b7-15b26ff25dda None None] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:14.508853 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4259c1e-2049-453b-8a69-772205a55d19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:14.510130 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4259c1e-2049-453b-8a69-772205a55d19 None admin] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:14.510411 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4259c1e-2049-453b-8a69-772205a55d19 None admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:14.510813 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4259c1e-2049-453b-8a69-772205a55d19 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:14.525025 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4259c1e-2049-453b-8a69-772205a55d19 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 09:33:14.526130 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9322cd5f-ad6f-4de0-a4f1-d891bc98e135 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:14.526836 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9322cd5f-ad6f-4de0-a4f1-d891bc98e135 None admin] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:14.527185 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9322cd5f-ad6f-4de0-a4f1-d891bc98e135 None admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:14.527466 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9322cd5f-ad6f-4de0-a4f1-d891bc98e135 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:14.530694 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application [None req-b4259c1e-2049-453b-8a69-772205a55d19 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 09:33:14.530694 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:33:14.530694 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:33:14.530694 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:33:14.530694 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:33:14.530694 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:33:14.530694 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:33:14.530694 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:33:14.530694 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:33:14.530694 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:33:14.530694 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:33:14.530694 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:33:14.530694 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 09:33:14.530694 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 09:33:14.530694 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 09:33:14.530694 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 09:33:14.531534 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 09:33:14.531534 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 09:33:14.531534 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 09:33:14.531534 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 09:33:14.531534 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:33:14.531534 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:33:14.531534 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:33:14.531534 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:33:14.531534 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:33:14.531534 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:33:14.531534 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:33:14.531534 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application with Lock( Aug 31 09:33:14.531534 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:33:14.531534 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:33:14.531534 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:33:14.531534 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:33:14.532305 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:33:14.532305 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:33:14.532305 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:33:14.532305 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:33:14.532305 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 09:33:14.532305 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 09:33:14.532305 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 09:33:14.532305 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 09:33:14.532305 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 09:33:14.532305 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 09:33:14.532305 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 09:33:14.532305 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application Aug 31 09:33:14.532305 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 55/106] 10.222.0.13 () {56 vars in 1096 bytes} [Wed Aug 31 09:33:14 2022] GET /identity/v3/roles/admin => generated 83 bytes in 288 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:33:14.534554 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284ab196-9b40-4dd2-b012-82944617a1ff None None] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:14.534856 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284ab196-9b40-4dd2-b012-82944617a1ff None None] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:14.535203 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284ab196-9b40-4dd2-b012-82944617a1ff None None] PATH_INFO: `/` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:14.536716 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 56/107] 10.222.0.13 () {54 vars in 927 bytes} [Wed Aug 31 09:33:14 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:33:14.537234 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-284ab196-9b40-4dd2-b012-82944617a1ff None None] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:14.545789 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9322cd5f-ad6f-4de0-a4f1-d891bc98e135 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 09:33:14.553226 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application [None req-9322cd5f-ad6f-4de0-a4f1-d891bc98e135 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 31 09:33:14.553226 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:33:14.553226 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:33:14.553226 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:33:14.553226 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:33:14.553226 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:33:14.553226 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:33:14.553226 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:33:14.553226 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:33:14.553226 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:33:14.553226 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:33:14.553226 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:33:14.553226 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 09:33:14.553226 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 09:33:14.553226 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 09:33:14.553226 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 09:33:14.554268 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 09:33:14.554268 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 09:33:14.554268 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 09:33:14.554268 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 09:33:14.554268 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:33:14.554268 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:33:14.554268 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:33:14.554268 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:33:14.554268 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:33:14.554268 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:33:14.554268 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:33:14.554268 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application with Lock( Aug 31 09:33:14.554268 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:33:14.554268 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:33:14.554268 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:33:14.554268 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:33:14.555209 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:33:14.555209 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:33:14.555209 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:33:14.555209 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:33:14.555209 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 09:33:14.555209 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 09:33:14.555209 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 09:33:14.555209 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 09:33:14.555209 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 09:33:14.555209 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 09:33:14.555209 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 31 09:33:14.555209 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application Aug 31 09:33:14.557901 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 52/108] 10.222.0.13 () {56 vars in 1114 bytes} [Wed Aug 31 09:33:14 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 292 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:33:14.563854 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ab13d5-6253-4966-8065-c315378a146b None None] REQUEST_METHOD: `POST` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:14.564206 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ab13d5-6253-4966-8065-c315378a146b None None] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:14.564642 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ab13d5-6253-4966-8065-c315378a146b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:14.583022 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.auth.core [None req-49ab13d5-6253-4966-8065-c315378a146b None None] MFA Rules not processed for user `fd836cf590a145df9acf19cf3ae2ac56`. Rule list: `[]` (Enabled: `True`). {{(pid=88113) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:33:14.584238 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.fernet_utils [None req-49ab13d5-6253-4966-8065-c315378a146b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:33:14.601175 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=100 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:33:14.611404 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=100 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:33:14.660982 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 53/109] 10.222.0.13 () {58 vars in 1024 bytes} [Wed Aug 31 09:33:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 09:33:14.661889 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ab13d5-6253-4966-8065-c315378a146b None None] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:14.664990 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-284ab196-9b40-4dd2-b012-82944617a1ff None None] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:14.701451 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b4849d9-c5a2-4904-867e-7c0e4de54096 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:14.702043 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4849d9-c5a2-4904-867e-7c0e4de54096 None admin] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:14.702309 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4849d9-c5a2-4904-867e-7c0e4de54096 None admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:14.702578 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4849d9-c5a2-4904-867e-7c0e4de54096 None admin] PATH_INFO: `/v3/roles/member` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:14.713973 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b4849d9-c5a2-4904-867e-7c0e4de54096 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 09:33:14.718891 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application [None req-9b4849d9-c5a2-4904-867e-7c0e4de54096 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 31 09:33:14.718891 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:33:14.718891 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:33:14.718891 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:33:14.718891 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:33:14.718891 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:33:14.718891 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:33:14.718891 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:33:14.718891 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:33:14.718891 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:33:14.718891 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:33:14.718891 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:33:14.718891 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 09:33:14.718891 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 09:33:14.718891 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 09:33:14.718891 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 09:33:14.719965 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 09:33:14.719965 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 09:33:14.719965 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 09:33:14.719965 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 09:33:14.719965 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:33:14.719965 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:33:14.719965 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:33:14.719965 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:33:14.719965 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:33:14.719965 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:33:14.719965 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:33:14.719965 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application with Lock( Aug 31 09:33:14.719965 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:33:14.719965 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:33:14.719965 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:33:14.719965 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:33:14.721430 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:33:14.721430 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:33:14.721430 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:33:14.721430 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:33:14.721430 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 09:33:14.721430 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 09:33:14.721430 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 09:33:14.721430 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 09:33:14.721430 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 09:33:14.721430 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 09:33:14.721430 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 31 09:33:14.721430 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application Aug 31 09:33:14.721430 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 57/110] 10.222.0.13 () {56 vars in 1099 bytes} [Wed Aug 31 09:33:14 2022] GET /identity/v3/roles/member => generated 84 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:33:14.722271 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b4849d9-c5a2-4904-867e-7c0e4de54096 None admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:14.790758 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ab13d5-6253-4966-8065-c315378a146b None None] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:14.828852 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1bda10c-1344-462e-8a9c-8d6ffe1f5a0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:14.829516 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1bda10c-1344-462e-8a9c-8d6ffe1f5a0b None admin] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:14.830489 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1bda10c-1344-462e-8a9c-8d6ffe1f5a0b None admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:14.830787 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1bda10c-1344-462e-8a9c-8d6ffe1f5a0b None admin] PATH_INFO: `/v3/roles` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:14.831596 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1bda10c-1344-462e-8a9c-8d6ffe1f5a0b None admin] RBAC: Adding query filter params (name=admin) {{(pid=88113) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:33:14.831912 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1bda10c-1344-462e-8a9c-8d6ffe1f5a0b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:33:14.833471 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1bda10c-1344-462e-8a9c-8d6ffe1f5a0b None admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:33:14.834720 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b4849d9-c5a2-4904-867e-7c0e4de54096 None admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:14.843066 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 54/111] 10.222.0.13 () {56 vars in 1099 bytes} [Wed Aug 31 09:33:14 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:33:14.843762 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1bda10c-1344-462e-8a9c-8d6ffe1f5a0b None admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:14.852944 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=100 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:33:14.859089 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=100 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:33:14.881226 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-414ab803-288f-476f-8191-8f091e7b4ef6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:14.882213 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414ab803-288f-476f-8191-8f091e7b4ef6 None admin] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:14.882495 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414ab803-288f-476f-8191-8f091e7b4ef6 None admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:14.882768 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414ab803-288f-476f-8191-8f091e7b4ef6 None admin] PATH_INFO: `/v3/domains/default` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:14.883985 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-414ab803-288f-476f-8191-8f091e7b4ef6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:33:14.885492 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-414ab803-288f-476f-8191-8f091e7b4ef6 None admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:33:14.887267 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 58/112] 10.222.0.13 () {56 vars in 1108 bytes} [Wed Aug 31 09:33:14 2022] GET /identity/v3/domains/default => generated 199 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:33:14.888272 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f55e498-a648-426b-b2cd-907c953708ca None None] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:14.888554 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f55e498-a648-426b-b2cd-907c953708ca None None] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:14.888823 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f55e498-a648-426b-b2cd-907c953708ca None None] PATH_INFO: `/` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:14.890079 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 59/113] 10.222.0.13 () {54 vars in 927 bytes} [Wed Aug 31 09:33:14 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:33:14.890795 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f55e498-a648-426b-b2cd-907c953708ca None None] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:14.962530 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1bda10c-1344-462e-8a9c-8d6ffe1f5a0b None admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:15.000831 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d39dbd30-6062-4c03-809d-16d1ca10b039 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:15.001872 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39dbd30-6062-4c03-809d-16d1ca10b039 None admin] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:15.002374 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39dbd30-6062-4c03-809d-16d1ca10b039 None admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:15.002739 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39dbd30-6062-4c03-809d-16d1ca10b039 None admin] PATH_INFO: `/v3/roles` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:15.003428 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d39dbd30-6062-4c03-809d-16d1ca10b039 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=88113) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:33:15.003732 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d39dbd30-6062-4c03-809d-16d1ca10b039 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:33:15.005135 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d39dbd30-6062-4c03-809d-16d1ca10b039 None admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:33:15.005239 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f55e498-a648-426b-b2cd-907c953708ca None None] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:15.013557 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 55/114] 10.222.0.13 () {56 vars in 1111 bytes} [Wed Aug 31 09:33:14 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:33:15.014411 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d39dbd30-6062-4c03-809d-16d1ca10b039 None admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:15.044355 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c2dcb7-9331-4d73-ad73-56fb6263f8d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:15.044974 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c2dcb7-9331-4d73-ad73-56fb6263f8d2 None admin] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:15.045253 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c2dcb7-9331-4d73-ad73-56fb6263f8d2 None admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:15.045556 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c2dcb7-9331-4d73-ad73-56fb6263f8d2 None admin] PATH_INFO: `/v3/roles` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:15.046230 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6c2dcb7-9331-4d73-ad73-56fb6263f8d2 None admin] RBAC: Adding query filter params (name=member) {{(pid=88114) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:33:15.046533 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6c2dcb7-9331-4d73-ad73-56fb6263f8d2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:33:15.047912 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6c2dcb7-9331-4d73-ad73-56fb6263f8d2 None admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:33:15.058030 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 60/115] 10.222.0.13 () {56 vars in 1101 bytes} [Wed Aug 31 09:33:14 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:33:15.058655 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c2dcb7-9331-4d73-ad73-56fb6263f8d2 None admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:15.095087 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=100 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:33:15.106117 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=100 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:33:15.128091 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d39dbd30-6062-4c03-809d-16d1ca10b039 None admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:15.162872 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d70171e0-778c-44f9-bd84-f2df6b54f684 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:15.163644 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70171e0-778c-44f9-bd84-f2df6b54f684 None admin] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:15.164184 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70171e0-778c-44f9-bd84-f2df6b54f684 None admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:15.164184 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70171e0-778c-44f9-bd84-f2df6b54f684 None admin] PATH_INFO: `/v3/users/fd836cf590a145df9acf19cf3ae2ac56` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:15.165413 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d70171e0-778c-44f9-bd84-f2df6b54f684 None admin] RBAC: Authorizing `identity:get_user(user_id=fd836cf590a145df9acf19cf3ae2ac56)` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:33:15.166830 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c2dcb7-9331-4d73-ad73-56fb6263f8d2 None admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:15.166953 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d70171e0-778c-44f9-bd84-f2df6b54f684 None admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:33:15.168549 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 56/116] 10.222.0.13 () {56 vars in 1177 bytes} [Wed Aug 31 09:33:15 2022] GET /identity/v3/users/fd836cf590a145df9acf19cf3ae2ac56 => generated 247 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:33:15.169964 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9120608b-f054-426a-bff0-d3e5941b6d73 None None] REQUEST_METHOD: `POST` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:15.171125 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9120608b-f054-426a-bff0-d3e5941b6d73 None None] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:15.171125 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9120608b-f054-426a-bff0-d3e5941b6d73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:15.189991 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.auth.core [None req-9120608b-f054-426a-bff0-d3e5941b6d73 None None] MFA Rules not processed for user `fd836cf590a145df9acf19cf3ae2ac56`. Rule list: `[]` (Enabled: `True`). {{(pid=88113) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:33:15.191315 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.fernet_utils [None req-9120608b-f054-426a-bff0-d3e5941b6d73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:33:15.205243 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd6e3a0-a16a-4be2-977d-6db490a9654c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:15.206167 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd6e3a0-a16a-4be2-977d-6db490a9654c None admin] REQUEST_METHOD: `POST` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:15.206450 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd6e3a0-a16a-4be2-977d-6db490a9654c None admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:15.206727 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd6e3a0-a16a-4be2-977d-6db490a9654c None admin] PATH_INFO: `/v3/projects` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:15.207411 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddd6e3a0-a16a-4be2-977d-6db490a9654c None admin] RBAC: Authorizing `identity:create_project()` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:33:15.208915 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddd6e3a0-a16a-4be2-977d-6db490a9654c None admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:33:15.242141 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 61/117] 10.222.0.13 () {60 vars in 1141 bytes} [Wed Aug 31 09:33:15 2022] POST /identity/v3/projects => generated 302 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 09:33:15.242917 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd6e3a0-a16a-4be2-977d-6db490a9654c None admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:15.272172 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 57/118] 10.222.0.13 () {58 vars in 1024 bytes} [Wed Aug 31 09:33:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 09:33:15.273127 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9120608b-f054-426a-bff0-d3e5941b6d73 None None] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:15.369616 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:33:15.381693 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd6e3a0-a16a-4be2-977d-6db490a9654c None admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:15.402391 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=100 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:33:15.434443 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ee2bb2-dd71-481d-ad90-e1b91e3ff883 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:15.435272 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ee2bb2-dd71-481d-ad90-e1b91e3ff883 None admin] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:15.435564 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ee2bb2-dd71-481d-ad90-e1b91e3ff883 None admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:15.436214 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ee2bb2-dd71-481d-ad90-e1b91e3ff883 None admin] PATH_INFO: `/v3/roles/member` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:15.445516 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9120608b-f054-426a-bff0-d3e5941b6d73 None None] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:15.451712 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3ee2bb2-dd71-481d-ad90-e1b91e3ff883 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 09:33:15.457306 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application [None req-b3ee2bb2-dd71-481d-ad90-e1b91e3ff883 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 31 09:33:15.457306 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:33:15.457306 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:33:15.457306 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:33:15.457306 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:33:15.457306 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:33:15.457306 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:33:15.457306 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:33:15.457306 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:33:15.457306 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:33:15.457306 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:33:15.457306 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:33:15.457306 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 09:33:15.457306 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 09:33:15.457306 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 09:33:15.457306 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 09:33:15.458242 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 09:33:15.458242 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 09:33:15.458242 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 09:33:15.458242 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 09:33:15.458242 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:33:15.458242 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:33:15.458242 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:33:15.458242 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:33:15.458242 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:33:15.458242 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:33:15.458242 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:33:15.458242 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application with Lock( Aug 31 09:33:15.458242 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:33:15.458242 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:33:15.458242 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:33:15.458242 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:33:15.459009 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:33:15.459009 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:33:15.459009 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:33:15.459009 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:33:15.459009 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 09:33:15.459009 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 09:33:15.459009 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 09:33:15.459009 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 09:33:15.459009 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 09:33:15.459009 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 09:33:15.459009 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 31 09:33:15.459009 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application Aug 31 09:33:15.459009 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 62/119] 10.222.0.13 () {56 vars in 1099 bytes} [Wed Aug 31 09:33:15 2022] GET /identity/v3/roles/member => generated 84 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:33:15.459739 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ee2bb2-dd71-481d-ad90-e1b91e3ff883 None admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:15.492308 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f1e3e7b-6925-4e4e-96ce-cb8415b154f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:15.492885 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f1e3e7b-6925-4e4e-96ce-cb8415b154f0 None admin] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:15.493167 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f1e3e7b-6925-4e4e-96ce-cb8415b154f0 None admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:15.493477 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f1e3e7b-6925-4e4e-96ce-cb8415b154f0 None admin] PATH_INFO: `/v3/users/1dc7ae39ad83425fb2f7ddb6c98b0a20` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:15.494667 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f1e3e7b-6925-4e4e-96ce-cb8415b154f0 None admin] RBAC: Authorizing `identity:get_user(user_id=1dc7ae39ad83425fb2f7ddb6c98b0a20)` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:33:15.496132 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f1e3e7b-6925-4e4e-96ce-cb8415b154f0 None admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:33:15.498001 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 58/120] 10.222.0.13 () {56 vars in 1177 bytes} [Wed Aug 31 09:33:15 2022] GET /identity/v3/users/1dc7ae39ad83425fb2f7ddb6c98b0a20 => generated 275 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:33:15.498697 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f1e3e7b-6925-4e4e-96ce-cb8415b154f0 None admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:15.600535 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ee2bb2-dd71-481d-ad90-e1b91e3ff883 None admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:15.614099 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f1e3e7b-6925-4e4e-96ce-cb8415b154f0 None admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:15.641174 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7544b597-62b6-4c7e-bb85-5453e1fbdb96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:15.641848 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7544b597-62b6-4c7e-bb85-5453e1fbdb96 None admin] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:15.642703 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7544b597-62b6-4c7e-bb85-5453e1fbdb96 None admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:15.643043 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7544b597-62b6-4c7e-bb85-5453e1fbdb96 None admin] PATH_INFO: `/v3/users/1dc7ae39ad83425fb2f7ddb6c98b0a20` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:15.644244 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7544b597-62b6-4c7e-bb85-5453e1fbdb96 None admin] RBAC: Authorizing `identity:get_user(user_id=1dc7ae39ad83425fb2f7ddb6c98b0a20)` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:33:15.645998 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7544b597-62b6-4c7e-bb85-5453e1fbdb96 None admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:33:15.647608 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 63/121] 10.222.0.13 () {56 vars in 1177 bytes} [Wed Aug 31 09:33:15 2022] GET /identity/v3/users/1dc7ae39ad83425fb2f7ddb6c98b0a20 => generated 275 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:33:15.647608 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7544b597-62b6-4c7e-bb85-5453e1fbdb96 None admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:15.660300 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ae74d1-eea1-4d9b-a5dc-19159054c34c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:15.660960 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ae74d1-eea1-4d9b-a5dc-19159054c34c None admin] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:15.661305 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ae74d1-eea1-4d9b-a5dc-19159054c34c None admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:15.661549 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ae74d1-eea1-4d9b-a5dc-19159054c34c None admin] PATH_INFO: `/v3/projects/1001686acf834f2a8e65b3fc8369f4f6` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:15.662841 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16ae74d1-eea1-4d9b-a5dc-19159054c34c None admin] RBAC: Authorizing `identity:get_project(project_id=1001686acf834f2a8e65b3fc8369f4f6)` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:33:15.664176 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16ae74d1-eea1-4d9b-a5dc-19159054c34c None admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:33:15.666094 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 59/122] 10.222.0.13 () {56 vars in 1186 bytes} [Wed Aug 31 09:33:15 2022] GET /identity/v3/projects/1001686acf834f2a8e65b3fc8369f4f6 => generated 298 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:33:15.671410 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ae74d1-eea1-4d9b-a5dc-19159054c34c None admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:15.677673 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=100 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:33:15.683225 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=100 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:33:15.765292 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7544b597-62b6-4c7e-bb85-5453e1fbdb96 None admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:15.791150 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ae74d1-eea1-4d9b-a5dc-19159054c34c None admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:15.803399 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e2b7bd-c95c-49f5-bdbb-00c6610dc37c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:15.804018 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e2b7bd-c95c-49f5-bdbb-00c6610dc37c None admin] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:15.804298 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e2b7bd-c95c-49f5-bdbb-00c6610dc37c None admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:15.804572 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e2b7bd-c95c-49f5-bdbb-00c6610dc37c None admin] PATH_INFO: `/v3/roles` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:15.805245 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63e2b7bd-c95c-49f5-bdbb-00c6610dc37c None admin] RBAC: Adding query filter params (name=member) {{(pid=88114) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:33:15.805589 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63e2b7bd-c95c-49f5-bdbb-00c6610dc37c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:33:15.806970 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63e2b7bd-c95c-49f5-bdbb-00c6610dc37c None admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:33:15.821250 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 64/123] 10.222.0.13 () {56 vars in 1101 bytes} [Wed Aug 31 09:33:15 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:33:15.821250 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e2b7bd-c95c-49f5-bdbb-00c6610dc37c None admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:15.837589 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c6fd7d-fbca-4bbe-a4d2-8fb421be94f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:15.838634 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c6fd7d-fbca-4bbe-a4d2-8fb421be94f3 None admin] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:15.838998 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c6fd7d-fbca-4bbe-a4d2-8fb421be94f3 None admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:15.839282 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c6fd7d-fbca-4bbe-a4d2-8fb421be94f3 None admin] PATH_INFO: `/v3/projects/1001686acf834f2a8e65b3fc8369f4f6` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:15.843184 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88c6fd7d-fbca-4bbe-a4d2-8fb421be94f3 None admin] RBAC: Authorizing `identity:get_project(project_id=1001686acf834f2a8e65b3fc8369f4f6)` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:33:15.844692 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88c6fd7d-fbca-4bbe-a4d2-8fb421be94f3 None admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:33:15.846809 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 60/124] 10.222.0.13 () {56 vars in 1186 bytes} [Wed Aug 31 09:33:15 2022] GET /identity/v3/projects/1001686acf834f2a8e65b3fc8369f4f6 => generated 298 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:33:15.847412 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c6fd7d-fbca-4bbe-a4d2-8fb421be94f3 None admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:15.921827 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=100 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:33:15.934490 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=100 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:33:15.941936 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e2b7bd-c95c-49f5-bdbb-00c6610dc37c None admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:15.964456 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c6fd7d-fbca-4bbe-a4d2-8fb421be94f3 None admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:15.982855 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c799cef-54d0-47b4-ac22-d4ce00db0783 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:15.983510 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c799cef-54d0-47b4-ac22-d4ce00db0783 None admin] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:15.983790 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c799cef-54d0-47b4-ac22-d4ce00db0783 None admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:15.984071 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c799cef-54d0-47b4-ac22-d4ce00db0783 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:15.995670 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c799cef-54d0-47b4-ac22-d4ce00db0783 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 09:33:16.001102 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application [None req-9c799cef-54d0-47b4-ac22-d4ce00db0783 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 31 09:33:16.001102 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:33:16.001102 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:33:16.001102 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:33:16.001102 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:33:16.001102 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:33:16.001102 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:33:16.001102 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:33:16.001102 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:33:16.001102 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:33:16.001102 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:33:16.001102 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:33:16.001102 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 09:33:16.001102 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 09:33:16.001102 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 09:33:16.001102 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 09:33:16.002046 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 09:33:16.002046 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 09:33:16.002046 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 09:33:16.002046 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 09:33:16.002046 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:33:16.002046 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:33:16.002046 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:33:16.002046 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:33:16.002046 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:33:16.002046 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:33:16.002046 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:33:16.002046 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application with Lock( Aug 31 09:33:16.002046 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:33:16.002046 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:33:16.002046 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:33:16.002046 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:33:16.006242 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:33:16.006242 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:33:16.006242 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:33:16.006242 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:33:16.006242 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 09:33:16.006242 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 09:33:16.006242 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 09:33:16.006242 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 09:33:16.006242 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 09:33:16.006242 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 09:33:16.006242 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 31 09:33:16.006242 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application Aug 31 09:33:16.006242 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 65/125] 10.222.0.13 () {56 vars in 1099 bytes} [Wed Aug 31 09:33:15 2022] GET /identity/v3/roles/reader => generated 84 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:33:16.006242 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c799cef-54d0-47b4-ac22-d4ce00db0783 None admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:16.012387 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e33d155-bfa3-4e5e-844f-72ba50a23e76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:16.013508 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e33d155-bfa3-4e5e-844f-72ba50a23e76 None admin] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:16.013899 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e33d155-bfa3-4e5e-844f-72ba50a23e76 None admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:16.014282 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e33d155-bfa3-4e5e-844f-72ba50a23e76 None admin] PATH_INFO: `/v3/projects/1001686acf834f2a8e65b3fc8369f4f6` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:16.015679 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e33d155-bfa3-4e5e-844f-72ba50a23e76 None admin] RBAC: Authorizing `identity:get_project(project_id=1001686acf834f2a8e65b3fc8369f4f6)` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:33:16.017958 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e33d155-bfa3-4e5e-844f-72ba50a23e76 None admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:33:16.025503 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 61/126] 10.222.0.13 () {56 vars in 1186 bytes} [Wed Aug 31 09:33:15 2022] GET /identity/v3/projects/1001686acf834f2a8e65b3fc8369f4f6 => generated 298 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:33:16.025503 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e33d155-bfa3-4e5e-844f-72ba50a23e76 None admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:16.119588 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c799cef-54d0-47b4-ac22-d4ce00db0783 None admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:16.154969 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e33d155-bfa3-4e5e-844f-72ba50a23e76 None admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:16.159405 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd6f86ea-8283-44d8-bea4-5badf02e7f5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:16.160106 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6f86ea-8283-44d8-bea4-5badf02e7f5c None admin] REQUEST_METHOD: `PUT` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:16.160391 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6f86ea-8283-44d8-bea4-5badf02e7f5c None admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:16.160669 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6f86ea-8283-44d8-bea4-5badf02e7f5c None admin] PATH_INFO: `/v3/projects/1001686acf834f2a8e65b3fc8369f4f6/users/fd836cf590a145df9acf19cf3ae2ac56/roles/5d4ee63a3add47fc9a6b8b16d39e0a1f` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:16.162780 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd6f86ea-8283-44d8-bea4-5badf02e7f5c None admin] RBAC: Authorizing `identity:create_grant(project_id=1001686acf834f2a8e65b3fc8369f4f6, user_id=fd836cf590a145df9acf19cf3ae2ac56, role_id=5d4ee63a3add47fc9a6b8b16d39e0a1f)` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:33:16.164285 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd6f86ea-8283-44d8-bea4-5badf02e7f5c None admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:33:16.189424 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 66/127] 10.222.0.13 () {58 vars in 1439 bytes} [Wed Aug 31 09:33:16 2022] PUT /identity/v3/projects/1001686acf834f2a8e65b3fc8369f4f6/users/fd836cf590a145df9acf19cf3ae2ac56/roles/5d4ee63a3add47fc9a6b8b16d39e0a1f => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:33:16.189424 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd6f86ea-8283-44d8-bea4-5badf02e7f5c None admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:16.196373 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:33:16.214918 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6421239f-e74c-464c-ba53-7a6ea0346db2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:16.215867 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6421239f-e74c-464c-ba53-7a6ea0346db2 None admin] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:16.216230 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6421239f-e74c-464c-ba53-7a6ea0346db2 None admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:16.216538 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6421239f-e74c-464c-ba53-7a6ea0346db2 None admin] PATH_INFO: `/v3/users/1dc7ae39ad83425fb2f7ddb6c98b0a20` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:16.218174 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6421239f-e74c-464c-ba53-7a6ea0346db2 None admin] RBAC: Authorizing `identity:get_user(user_id=1dc7ae39ad83425fb2f7ddb6c98b0a20)` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:33:16.221110 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6421239f-e74c-464c-ba53-7a6ea0346db2 None admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:33:16.223885 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 62/128] 10.222.0.13 () {56 vars in 1177 bytes} [Wed Aug 31 09:33:16 2022] GET /identity/v3/users/1dc7ae39ad83425fb2f7ddb6c98b0a20 => generated 275 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:33:16.230874 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6421239f-e74c-464c-ba53-7a6ea0346db2 None admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:16.261994 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=100 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:33:16.375632 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6421239f-e74c-464c-ba53-7a6ea0346db2 None admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:16.383188 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd6f86ea-8283-44d8-bea4-5badf02e7f5c None admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:16.426176 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c74736b6-31a9-4ba1-9060-38f45f624e16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:16.426801 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74736b6-31a9-4ba1-9060-38f45f624e16 None admin] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:16.427083 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74736b6-31a9-4ba1-9060-38f45f624e16 None admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:16.427352 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74736b6-31a9-4ba1-9060-38f45f624e16 None admin] PATH_INFO: `/v3/roles` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:16.428039 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c74736b6-31a9-4ba1-9060-38f45f624e16 None admin] RBAC: Adding query filter params (name=reader) {{(pid=88113) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:33:16.428345 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c74736b6-31a9-4ba1-9060-38f45f624e16 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:33:16.429966 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c74736b6-31a9-4ba1-9060-38f45f624e16 None admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:33:16.432122 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5abc8306-9812-4e95-9697-f6aae12c59d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:16.433011 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5abc8306-9812-4e95-9697-f6aae12c59d6 None admin] REQUEST_METHOD: `PUT` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:16.433426 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5abc8306-9812-4e95-9697-f6aae12c59d6 None admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:16.433794 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5abc8306-9812-4e95-9697-f6aae12c59d6 None admin] PATH_INFO: `/v3/projects/1001686acf834f2a8e65b3fc8369f4f6/users/1dc7ae39ad83425fb2f7ddb6c98b0a20/roles/889857b056624d47b572d4a80ecc3f8e` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:16.436285 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5abc8306-9812-4e95-9697-f6aae12c59d6 None admin] RBAC: Authorizing `identity:create_grant(project_id=1001686acf834f2a8e65b3fc8369f4f6, user_id=1dc7ae39ad83425fb2f7ddb6c98b0a20, role_id=889857b056624d47b572d4a80ecc3f8e)` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:33:16.437888 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5abc8306-9812-4e95-9697-f6aae12c59d6 None admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:33:16.440079 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 63/129] 10.222.0.13 () {56 vars in 1101 bytes} [Wed Aug 31 09:33:16 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:33:16.440967 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c74736b6-31a9-4ba1-9060-38f45f624e16 None admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:16.456776 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 67/130] 10.222.0.13 () {58 vars in 1439 bytes} [Wed Aug 31 09:33:16 2022] PUT /identity/v3/projects/1001686acf834f2a8e65b3fc8369f4f6/users/1dc7ae39ad83425fb2f7ddb6c98b0a20/roles/889857b056624d47b572d4a80ecc3f8e => generated 0 bytes in 274 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:33:16.457998 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5abc8306-9812-4e95-9697-f6aae12c59d6 None admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:16.547027 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:33:16.587146 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=100 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:33:16.633570 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5abc8306-9812-4e95-9697-f6aae12c59d6 None admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:16.644726 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c74736b6-31a9-4ba1-9060-38f45f624e16 None admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:16.702324 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ecdf27-b580-4a60-9ff7-bf5b31d13c00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:16.702983 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ecdf27-b580-4a60-9ff7-bf5b31d13c00 None admin] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:16.703317 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ecdf27-b580-4a60-9ff7-bf5b31d13c00 None admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:16.703598 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ecdf27-b580-4a60-9ff7-bf5b31d13c00 None admin] PATH_INFO: `/v3/projects/9f41d93b9d96487f867195c8bdd23cbc` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:16.705421 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17ecdf27-b580-4a60-9ff7-bf5b31d13c00 None admin] RBAC: Authorizing `identity:get_project(project_id=9f41d93b9d96487f867195c8bdd23cbc)` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:33:16.707163 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17ecdf27-b580-4a60-9ff7-bf5b31d13c00 None admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:33:16.709252 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 68/131] 10.222.0.13 () {56 vars in 1186 bytes} [Wed Aug 31 09:33:16 2022] GET /identity/v3/projects/9f41d93b9d96487f867195c8bdd23cbc => generated 312 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:33:16.712099 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ecdf27-b580-4a60-9ff7-bf5b31d13c00 None admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:16.718707 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ec353a-178e-46f5-8759-330c54ceaf00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:16.719886 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ec353a-178e-46f5-8759-330c54ceaf00 None admin] REQUEST_METHOD: `PUT` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:16.720928 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ec353a-178e-46f5-8759-330c54ceaf00 None admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:16.721474 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ec353a-178e-46f5-8759-330c54ceaf00 None admin] PATH_INFO: `/v3/projects/1001686acf834f2a8e65b3fc8369f4f6/users/1dc7ae39ad83425fb2f7ddb6c98b0a20/roles/6c99f5d9f1f243eaa134d8540f21095d` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:16.725756 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6ec353a-178e-46f5-8759-330c54ceaf00 None admin] RBAC: Authorizing `identity:create_grant(project_id=1001686acf834f2a8e65b3fc8369f4f6, user_id=1dc7ae39ad83425fb2f7ddb6c98b0a20, role_id=6c99f5d9f1f243eaa134d8540f21095d)` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:33:16.728624 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6ec353a-178e-46f5-8759-330c54ceaf00 None admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:33:16.761208 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 64/132] 10.222.0.13 () {58 vars in 1439 bytes} [Wed Aug 31 09:33:16 2022] PUT /identity/v3/projects/1001686acf834f2a8e65b3fc8369f4f6/users/1dc7ae39ad83425fb2f7ddb6c98b0a20/roles/6c99f5d9f1f243eaa134d8540f21095d => generated 0 bytes in 320 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:33:16.767474 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ec353a-178e-46f5-8759-330c54ceaf00 None admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:16.847276 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ecdf27-b580-4a60-9ff7-bf5b31d13c00 None admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:16.886184 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=100 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:33:16.895006 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e4b196-78df-4531-88d4-2fda4b2f6e15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:16.896093 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e4b196-78df-4531-88d4-2fda4b2f6e15 None admin] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:16.896608 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e4b196-78df-4531-88d4-2fda4b2f6e15 None admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:16.897230 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e4b196-78df-4531-88d4-2fda4b2f6e15 None admin] PATH_INFO: `/v3/users/32b40e2909174cf086806d3a85cd5cc1` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:16.916039 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2e4b196-78df-4531-88d4-2fda4b2f6e15 None admin] RBAC: Authorizing `identity:get_user(user_id=32b40e2909174cf086806d3a85cd5cc1)` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:33:16.917641 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2e4b196-78df-4531-88d4-2fda4b2f6e15 None admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:33:16.919559 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 69/133] 10.222.0.13 () {56 vars in 1177 bytes} [Wed Aug 31 09:33:16 2022] GET /identity/v3/users/32b40e2909174cf086806d3a85cd5cc1 => generated 289 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:33:16.926297 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e4b196-78df-4531-88d4-2fda4b2f6e15 None admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:17.007540 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ec353a-178e-46f5-8759-330c54ceaf00 None admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:17.041427 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e4b196-78df-4531-88d4-2fda4b2f6e15 None admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:17.070200 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:33:17.085944 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a814c74-74eb-4aca-a9f5-c3cd01316b26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:17.086455 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85766f03-b7bc-4a36-b12e-5715a01a3a7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:17.087543 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a814c74-74eb-4aca-a9f5-c3cd01316b26 None admin] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:17.088011 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a814c74-74eb-4aca-a9f5-c3cd01316b26 None admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:17.088127 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85766f03-b7bc-4a36-b12e-5715a01a3a7d None admin] REQUEST_METHOD: `PUT` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:17.088578 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a814c74-74eb-4aca-a9f5-c3cd01316b26 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:17.088679 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85766f03-b7bc-4a36-b12e-5715a01a3a7d None admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:17.089094 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85766f03-b7bc-4a36-b12e-5715a01a3a7d None admin] PATH_INFO: `/v3/projects/9f41d93b9d96487f867195c8bdd23cbc/users/1dc7ae39ad83425fb2f7ddb6c98b0a20/roles/6c99f5d9f1f243eaa134d8540f21095d` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:17.097404 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85766f03-b7bc-4a36-b12e-5715a01a3a7d None admin] RBAC: Authorizing `identity:create_grant(project_id=9f41d93b9d96487f867195c8bdd23cbc, user_id=1dc7ae39ad83425fb2f7ddb6c98b0a20, role_id=6c99f5d9f1f243eaa134d8540f21095d)` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:33:17.100463 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85766f03-b7bc-4a36-b12e-5715a01a3a7d None admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:33:17.151053 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 65/134] 10.222.0.13 () {58 vars in 1439 bytes} [Wed Aug 31 09:33:16 2022] PUT /identity/v3/projects/9f41d93b9d96487f867195c8bdd23cbc/users/1dc7ae39ad83425fb2f7ddb6c98b0a20/roles/6c99f5d9f1f243eaa134d8540f21095d => generated 0 bytes in 389 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:33:17.167027 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=100 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:33:17.180918 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a814c74-74eb-4aca-a9f5-c3cd01316b26 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:33:17.182768 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a814c74-74eb-4aca-a9f5-c3cd01316b26 None admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:33:17.313279 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 70/135] 10.222.0.13 () {58 vars in 1282 bytes} [Wed Aug 31 09:33:16 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 388 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 31 09:33:17.320883 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85766f03-b7bc-4a36-b12e-5715a01a3a7d None admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:17.556564 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85766f03-b7bc-4a36-b12e-5715a01a3a7d None admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:17.628928 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f6c4792-01b7-4853-bce2-7e617ca33993 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:17.630173 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6c4792-01b7-4853-bce2-7e617ca33993 None admin] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:17.630646 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6c4792-01b7-4853-bce2-7e617ca33993 None admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:17.631259 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6c4792-01b7-4853-bce2-7e617ca33993 None admin] PATH_INFO: `/v3/projects/1001686acf834f2a8e65b3fc8369f4f6` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:17.633514 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f6c4792-01b7-4853-bce2-7e617ca33993 None admin] RBAC: Authorizing `identity:get_project(project_id=1001686acf834f2a8e65b3fc8369f4f6)` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:33:17.636602 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f6c4792-01b7-4853-bce2-7e617ca33993 None admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:33:17.640007 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 66/136] 10.222.0.13 () {56 vars in 1186 bytes} [Wed Aug 31 09:33:17 2022] GET /identity/v3/projects/1001686acf834f2a8e65b3fc8369f4f6 => generated 298 bytes in 320 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:33:17.647726 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a814c74-74eb-4aca-a9f5-c3cd01316b26 None admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:17.784792 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=100 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:33:17.797100 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a814c74-74eb-4aca-a9f5-c3cd01316b26 None admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:17.853057 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed1fbf54-8f67-4943-a7dd-e5d73408ec6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:17.853985 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1fbf54-8f67-4943-a7dd-e5d73408ec6d None admin] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:17.854426 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1fbf54-8f67-4943-a7dd-e5d73408ec6d None admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:17.854901 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1fbf54-8f67-4943-a7dd-e5d73408ec6d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:17.855899 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed1fbf54-8f67-4943-a7dd-e5d73408ec6d None admin] RBAC: Adding query filter params (role.id=9814fca799974e3e87ffc04473b6510e, scope.project.id=1001686acf834f2a8e65b3fc8369f4f6, user.id=32b40e2909174cf086806d3a85cd5cc1) {{(pid=88114) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:33:17.856291 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e70ec3-d48c-42e4-8f44-9a0f5394f241 None None] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:17.856567 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed1fbf54-8f67-4943-a7dd-e5d73408ec6d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:33:17.857013 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e70ec3-d48c-42e4-8f44-9a0f5394f241 None None] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:17.857633 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e70ec3-d48c-42e4-8f44-9a0f5394f241 None None] PATH_INFO: `/` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:17.858790 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed1fbf54-8f67-4943-a7dd-e5d73408ec6d None admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:33:17.860170 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 67/137] 10.222.0.13 () {54 vars in 927 bytes} [Wed Aug 31 09:33:17 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:33:17.870981 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f32cb0-c0c2-479a-8d04-ea55d04c8020 None None] REQUEST_METHOD: `POST` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:17.871360 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 71/138] 10.222.0.13 () {56 vars in 1374 bytes} [Wed Aug 31 09:33:17 2022] GET /identity/v3/role_assignments?user.id=32b40e2909174cf086806d3a85cd5cc1&scope.project.id=1001686acf834f2a8e65b3fc8369f4f6&role.id=9814fca799974e3e87ffc04473b6510e => generated 259 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:33:17.871547 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f32cb0-c0c2-479a-8d04-ea55d04c8020 None None] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:17.871879 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f32cb0-c0c2-479a-8d04-ea55d04c8020 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:17.925302 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.auth.core [None req-d4f32cb0-c0c2-479a-8d04-ea55d04c8020 None None] MFA Rules not processed for user `fd836cf590a145df9acf19cf3ae2ac56`. Rule list: `[]` (Enabled: `True`). {{(pid=88113) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:33:17.926691 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.fernet_utils [None req-d4f32cb0-c0c2-479a-8d04-ea55d04c8020 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:33:18.049831 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:33:18.062678 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 68/139] 10.222.0.13 () {58 vars in 1024 bytes} [Wed Aug 31 09:33:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 09:33:18.462912 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed1fbf54-8f67-4943-a7dd-e5d73408ec6d None admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:18.604231 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed1fbf54-8f67-4943-a7dd-e5d73408ec6d None admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:18.652974 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e3de62-ea49-4eb3-87ea-4d5c630e3c12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:18.653641 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e3de62-ea49-4eb3-87ea-4d5c630e3c12 None admin] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:18.653907 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e3de62-ea49-4eb3-87ea-4d5c630e3c12 None admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:18.654177 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e3de62-ea49-4eb3-87ea-4d5c630e3c12 None admin] PATH_INFO: `/v3/domains/default` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:18.658898 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1e3de62-ea49-4eb3-87ea-4d5c630e3c12 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:33:18.660370 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1e3de62-ea49-4eb3-87ea-4d5c630e3c12 None admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:33:18.662289 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 72/140] 10.222.0.13 () {56 vars in 1108 bytes} [Wed Aug 31 09:33:18 2022] GET /identity/v3/domains/default => generated 199 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 09:33:18.669962 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f32cb0-c0c2-479a-8d04-ea55d04c8020 None None] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:18.866855 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f32cb0-c0c2-479a-8d04-ea55d04c8020 None None] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:18.914431 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e715fe8-9eae-44a0-9b26-8b63aa9bd293 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:18.915404 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e715fe8-9eae-44a0-9b26-8b63aa9bd293 None admin] REQUEST_METHOD: `POST` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:18.916091 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e715fe8-9eae-44a0-9b26-8b63aa9bd293 None admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:18.916564 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e715fe8-9eae-44a0-9b26-8b63aa9bd293 None admin] PATH_INFO: `/v3/users` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:18.917476 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e715fe8-9eae-44a0-9b26-8b63aa9bd293 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:33:18.919144 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e715fe8-9eae-44a0-9b26-8b63aa9bd293 None admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:33:18.957257 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 69/141] 10.222.0.13 () {60 vars in 1132 bytes} [Wed Aug 31 09:33:18 2022] POST /identity/v3/users => generated 283 bytes in 289 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 09:33:19.034917 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2c487a-fb04-4e01-9503-eeedc3946a04 None None] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:19.035248 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2c487a-fb04-4e01-9503-eeedc3946a04 None None] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:19.035482 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2c487a-fb04-4e01-9503-eeedc3946a04 None None] PATH_INFO: `/` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:19.036967 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 73/142] 10.222.0.13 () {54 vars in 927 bytes} [Wed Aug 31 09:33:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:33:19.047545 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf92158-4144-47ba-a1ea-386e3f11c08c None None] REQUEST_METHOD: `POST` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:19.048227 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf92158-4144-47ba-a1ea-386e3f11c08c None None] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:19.050021 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf92158-4144-47ba-a1ea-386e3f11c08c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:19.076933 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.auth.core [None req-acf92158-4144-47ba-a1ea-386e3f11c08c None None] MFA Rules not processed for user `fd836cf590a145df9acf19cf3ae2ac56`. Rule list: `[]` (Enabled: `True`). {{(pid=88113) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:33:19.078525 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.fernet_utils [None req-acf92158-4144-47ba-a1ea-386e3f11c08c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:33:19.141947 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:33:19.179248 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 70/143] 10.222.0.13 () {58 vars in 1024 bytes} [Wed Aug 31 09:33:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 09:33:19.575781 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad2c487a-fb04-4e01-9503-eeedc3946a04 None None] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:19.604091 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=100 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:33:19.703714 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad2c487a-fb04-4e01-9503-eeedc3946a04 None None] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:19.747956 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df3a9675-928f-40ff-9026-7d1919da7281 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:19.748604 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3a9675-928f-40ff-9026-7d1919da7281 None admin] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:19.748891 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3a9675-928f-40ff-9026-7d1919da7281 None admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:19.749169 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3a9675-928f-40ff-9026-7d1919da7281 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:19.763029 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df3a9675-928f-40ff-9026-7d1919da7281 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 09:33:19.768761 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application [None req-df3a9675-928f-40ff-9026-7d1919da7281 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 09:33:19.768761 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:33:19.768761 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:33:19.768761 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:33:19.768761 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:33:19.768761 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:33:19.768761 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:33:19.768761 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:33:19.768761 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:33:19.768761 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:33:19.768761 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:33:19.768761 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:33:19.768761 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 09:33:19.768761 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 09:33:19.768761 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 09:33:19.768761 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 09:33:19.770617 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 09:33:19.770617 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 09:33:19.770617 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 09:33:19.770617 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 09:33:19.770617 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:33:19.770617 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:33:19.770617 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:33:19.770617 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:33:19.770617 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:33:19.770617 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:33:19.770617 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:33:19.770617 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application with Lock( Aug 31 09:33:19.770617 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:33:19.770617 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:33:19.770617 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:33:19.770617 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:33:19.772130 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:33:19.772130 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:33:19.772130 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:33:19.772130 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:33:19.772130 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 09:33:19.772130 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 09:33:19.772130 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 09:33:19.772130 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 09:33:19.772130 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 09:33:19.772130 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 09:33:19.772130 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 09:33:19.772130 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application Aug 31 09:33:19.774468 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 74/144] 10.222.0.13 () {56 vars in 1096 bytes} [Wed Aug 31 09:33:19 2022] GET /identity/v3/roles/admin => generated 83 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:33:19.778342 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf92158-4144-47ba-a1ea-386e3f11c08c None None] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:19.848053 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ef3075-b377-40b6-b599-4f448e6aea42 None None] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:19.848053 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ef3075-b377-40b6-b599-4f448e6aea42 None None] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:19.848053 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ef3075-b377-40b6-b599-4f448e6aea42 None None] PATH_INFO: `/` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:19.848053 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 75/145] 10.222.0.13 () {54 vars in 927 bytes} [Wed Aug 31 09:33:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 31 09:33:19.854934 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d09ca32-5e5d-49d0-87a8-dabb8965191e None None] REQUEST_METHOD: `POST` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:19.855213 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d09ca32-5e5d-49d0-87a8-dabb8965191e None None] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:19.855487 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d09ca32-5e5d-49d0-87a8-dabb8965191e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:19.879700 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.auth.core [None req-4d09ca32-5e5d-49d0-87a8-dabb8965191e None None] MFA Rules not processed for user `fd836cf590a145df9acf19cf3ae2ac56`. Rule list: `[]` (Enabled: `True`). {{(pid=88114) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:33:19.880921 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.fernet_utils [None req-4d09ca32-5e5d-49d0-87a8-dabb8965191e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88114) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:33:19.936716 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf92158-4144-47ba-a1ea-386e3f11c08c None None] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:20.003372 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 76/146] 10.222.0.13 () {58 vars in 1024 bytes} [Wed Aug 31 09:33:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 31 09:33:20.007715 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f98097f2-e1d2-42b4-9eda-5d84f5ea618e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:20.008348 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98097f2-e1d2-42b4-9eda-5d84f5ea618e None admin] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:20.008630 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98097f2-e1d2-42b4-9eda-5d84f5ea618e None admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:20.008902 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98097f2-e1d2-42b4-9eda-5d84f5ea618e None admin] PATH_INFO: `/v3/roles` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:20.009612 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f98097f2-e1d2-42b4-9eda-5d84f5ea618e None admin] RBAC: Adding query filter params (name=admin) {{(pid=88113) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:33:20.009918 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f98097f2-e1d2-42b4-9eda-5d84f5ea618e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:33:20.011317 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f98097f2-e1d2-42b4-9eda-5d84f5ea618e None admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:33:20.027593 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 71/147] 10.222.0.13 () {56 vars in 1099 bytes} [Wed Aug 31 09:33:19 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:33:20.033886 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d09ca32-5e5d-49d0-87a8-dabb8965191e None None] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:20.060182 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=100 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:33:20.174159 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d09ca32-5e5d-49d0-87a8-dabb8965191e None None] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:20.219568 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef7d0fc-091e-4679-bbfe-2a3285310a8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:20.220250 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef7d0fc-091e-4679-bbfe-2a3285310a8d None admin] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:20.220534 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef7d0fc-091e-4679-bbfe-2a3285310a8d None admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:20.220850 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef7d0fc-091e-4679-bbfe-2a3285310a8d None admin] PATH_INFO: `/v3/users/fd836cf590a145df9acf19cf3ae2ac56` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:20.222115 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ef7d0fc-091e-4679-bbfe-2a3285310a8d None admin] RBAC: Authorizing `identity:get_user(user_id=fd836cf590a145df9acf19cf3ae2ac56)` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:33:20.223566 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ef7d0fc-091e-4679-bbfe-2a3285310a8d None admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:33:20.225219 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 77/148] 10.222.0.13 () {56 vars in 1177 bytes} [Wed Aug 31 09:33:20 2022] GET /identity/v3/users/fd836cf590a145df9acf19cf3ae2ac56 => generated 247 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:33:20.230983 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f98097f2-e1d2-42b4-9eda-5d84f5ea618e None admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:20.281739 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=100 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:33:20.371249 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f98097f2-e1d2-42b4-9eda-5d84f5ea618e None admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:20.415299 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58980c54-e3bc-4ee5-90be-f3609fab0639 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:20.415952 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58980c54-e3bc-4ee5-90be-f3609fab0639 None admin] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:20.416235 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58980c54-e3bc-4ee5-90be-f3609fab0639 None admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:20.416545 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58980c54-e3bc-4ee5-90be-f3609fab0639 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:20.479846 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef7d0fc-091e-4679-bbfe-2a3285310a8d None admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:20.536384 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58980c54-e3bc-4ee5-90be-f3609fab0639 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:33:20.538099 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58980c54-e3bc-4ee5-90be-f3609fab0639 None admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:33:20.627489 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=100 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:33:20.660433 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=100 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:33:20.711255 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef7d0fc-091e-4679-bbfe-2a3285310a8d None admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:20.748534 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 72/149] 10.222.0.13 () {58 vars in 1282 bytes} [Wed Aug 31 09:33:20 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 517 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 31 09:33:20.750003 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c959314a-52f7-490d-b048-1982bf1dcf04 None None] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:20.750297 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c959314a-52f7-490d-b048-1982bf1dcf04 None None] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:20.750579 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c959314a-52f7-490d-b048-1982bf1dcf04 None None] PATH_INFO: `/` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:20.752973 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 73/150] 10.222.0.13 () {54 vars in 927 bytes} [Wed Aug 31 09:33:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:33:20.753636 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c959314a-52f7-490d-b048-1982bf1dcf04 None None] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:20.806121 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb5b8e9-5544-4085-8734-c345ef517b0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:20.806773 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb5b8e9-5544-4085-8734-c345ef517b0a None admin] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:20.807047 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb5b8e9-5544-4085-8734-c345ef517b0a None admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:20.807313 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb5b8e9-5544-4085-8734-c345ef517b0a None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:20.819180 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cb5b8e9-5544-4085-8734-c345ef517b0a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 09:33:20.824068 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application [None req-6cb5b8e9-5544-4085-8734-c345ef517b0a None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 31 09:33:20.824068 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:33:20.824068 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:33:20.824068 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:33:20.824068 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:33:20.824068 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:33:20.824068 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:33:20.824068 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:33:20.824068 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:33:20.824068 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:33:20.824068 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:33:20.824068 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:33:20.824068 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 09:33:20.824068 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 09:33:20.824068 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 09:33:20.824068 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 09:33:20.824911 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 09:33:20.824911 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 09:33:20.824911 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 09:33:20.824911 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 09:33:20.824911 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:33:20.824911 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:33:20.824911 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:33:20.824911 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:33:20.824911 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:33:20.824911 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:33:20.824911 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:33:20.824911 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application with Lock( Aug 31 09:33:20.824911 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:33:20.824911 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:33:20.824911 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:33:20.824911 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:33:20.825712 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:33:20.825712 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:33:20.825712 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:33:20.825712 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:33:20.825712 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 09:33:20.825712 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 09:33:20.825712 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 09:33:20.825712 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 09:33:20.825712 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 09:33:20.825712 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 09:33:20.825712 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 31 09:33:20.825712 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application Aug 31 09:33:20.830139 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 78/151] 10.222.0.13 () {56 vars in 1114 bytes} [Wed Aug 31 09:33:20 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 346 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 09:33:20.832124 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e97246-5e99-422c-a494-0a144be816c9 None None] REQUEST_METHOD: `POST` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:20.832406 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e97246-5e99-422c-a494-0a144be816c9 None None] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:20.832676 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e97246-5e99-422c-a494-0a144be816c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:20.858769 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.auth.core [None req-14e97246-5e99-422c-a494-0a144be816c9 None None] MFA Rules not processed for user `fd836cf590a145df9acf19cf3ae2ac56`. Rule list: `[]` (Enabled: `True`). {{(pid=88114) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:33:20.861010 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.fernet_utils [None req-14e97246-5e99-422c-a494-0a144be816c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88114) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:33:20.900614 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c959314a-52f7-490d-b048-1982bf1dcf04 None None] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:20.980275 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79177a42-e6a6-4c4e-8acf-2c580a4a0ce6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:20.981016 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79177a42-e6a6-4c4e-8acf-2c580a4a0ce6 None admin] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:20.981299 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79177a42-e6a6-4c4e-8acf-2c580a4a0ce6 None admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:20.981597 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79177a42-e6a6-4c4e-8acf-2c580a4a0ce6 None admin] PATH_INFO: `/v3/projects/1001686acf834f2a8e65b3fc8369f4f6` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:20.982807 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79177a42-e6a6-4c4e-8acf-2c580a4a0ce6 None admin] RBAC: Authorizing `identity:get_project(project_id=1001686acf834f2a8e65b3fc8369f4f6)` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:33:20.984303 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79177a42-e6a6-4c4e-8acf-2c580a4a0ce6 None admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:33:20.986456 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 74/152] 10.222.0.13 () {56 vars in 1186 bytes} [Wed Aug 31 09:33:20 2022] GET /identity/v3/projects/1001686acf834f2a8e65b3fc8369f4f6 => generated 298 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:33:20.992317 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5b6dc6-cd89-4c02-a418-10a4ac9c4b49 None None] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:20.992627 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5b6dc6-cd89-4c02-a418-10a4ac9c4b49 None None] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:20.992869 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5b6dc6-cd89-4c02-a418-10a4ac9c4b49 None None] PATH_INFO: `/` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:20.994388 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 75/153] 10.222.0.13 () {54 vars in 927 bytes} [Wed Aug 31 09:33:20 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:33:20.996586 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be5b6dc6-cd89-4c02-a418-10a4ac9c4b49 None None] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:21.011268 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 79/154] 10.222.0.13 () {58 vars in 1024 bytes} [Wed Aug 31 09:33:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 09:33:21.012219 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e97246-5e99-422c-a494-0a144be816c9 None None] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:21.018189 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=100 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:33:21.129371 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be5b6dc6-cd89-4c02-a418-10a4ac9c4b49 None None] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:21.171741 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e8b91e9-2a80-45b2-81d8-9e5cbf7e7721 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:21.172395 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8b91e9-2a80-45b2-81d8-9e5cbf7e7721 None admin] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:21.172834 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8b91e9-2a80-45b2-81d8-9e5cbf7e7721 None admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:21.173117 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8b91e9-2a80-45b2-81d8-9e5cbf7e7721 None admin] PATH_INFO: `/v3/roles` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:21.174700 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e8b91e9-2a80-45b2-81d8-9e5cbf7e7721 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=88113) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:33:21.174899 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e8b91e9-2a80-45b2-81d8-9e5cbf7e7721 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:33:21.176668 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e8b91e9-2a80-45b2-81d8-9e5cbf7e7721 None admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:33:21.178213 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=100 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:33:21.189020 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 76/155] 10.222.0.13 () {56 vars in 1111 bytes} [Wed Aug 31 09:33:20 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:33:21.190569 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa9971f-04c8-4942-9eb5-4c1a3ad328e1 None None] REQUEST_METHOD: `POST` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:21.190874 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa9971f-04c8-4942-9eb5-4c1a3ad328e1 None None] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:21.191155 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa9971f-04c8-4942-9eb5-4c1a3ad328e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:21.212975 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.auth.core [None req-3fa9971f-04c8-4942-9eb5-4c1a3ad328e1 None None] MFA Rules not processed for user `fd836cf590a145df9acf19cf3ae2ac56`. Rule list: `[]` (Enabled: `True`). {{(pid=88113) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:33:21.214249 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.fernet_utils [None req-3fa9971f-04c8-4942-9eb5-4c1a3ad328e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:33:21.231426 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e97246-5e99-422c-a494-0a144be816c9 None None] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:21.293944 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 77/156] 10.222.0.13 () {58 vars in 1024 bytes} [Wed Aug 31 09:33:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 09:33:21.295526 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8b3a7a-d36a-49be-b83a-7a28b9793ff4 None None] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:21.295881 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8b3a7a-d36a-49be-b83a-7a28b9793ff4 None None] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:21.296167 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8b3a7a-d36a-49be-b83a-7a28b9793ff4 None None] PATH_INFO: `/` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:21.298021 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 78/157] 10.222.0.13 () {54 vars in 927 bytes} [Wed Aug 31 09:33:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:33:21.299704 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f8b3a7a-d36a-49be-b83a-7a28b9793ff4 None None] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:21.300292 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=100 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:33:21.366150 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac34faf-8c3f-45ad-98fb-2f991d5b64d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:21.366784 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac34faf-8c3f-45ad-98fb-2f991d5b64d2 None admin] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:21.367067 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac34faf-8c3f-45ad-98fb-2f991d5b64d2 None admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:21.367342 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac34faf-8c3f-45ad-98fb-2f991d5b64d2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:21.368086 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eac34faf-8c3f-45ad-98fb-2f991d5b64d2 None admin] RBAC: Adding query filter params (role.id=5d4ee63a3add47fc9a6b8b16d39e0a1f, scope.project.id=1001686acf834f2a8e65b3fc8369f4f6, user.id=fd836cf590a145df9acf19cf3ae2ac56) {{(pid=88114) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:33:21.368397 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eac34faf-8c3f-45ad-98fb-2f991d5b64d2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:33:21.369974 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eac34faf-8c3f-45ad-98fb-2f991d5b64d2 None admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:33:21.391426 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 80/158] 10.222.0.13 () {56 vars in 1374 bytes} [Wed Aug 31 09:33:21 2022] GET /identity/v3/role_assignments?user.id=fd836cf590a145df9acf19cf3ae2ac56&scope.project.id=1001686acf834f2a8e65b3fc8369f4f6&role.id=5d4ee63a3add47fc9a6b8b16d39e0a1f => generated 608 bytes in 380 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:33:21.393361 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf78f341-d1b1-4c79-9ed6-d831e3a5d528 None None] REQUEST_METHOD: `POST` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:21.393696 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf78f341-d1b1-4c79-9ed6-d831e3a5d528 None None] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:21.393976 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf78f341-d1b1-4c79-9ed6-d831e3a5d528 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:21.446933 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f8b3a7a-d36a-49be-b83a-7a28b9793ff4 None None] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:21.455025 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.auth.core [None req-cf78f341-d1b1-4c79-9ed6-d831e3a5d528 None None] MFA Rules not processed for user `fd836cf590a145df9acf19cf3ae2ac56`. Rule list: `[]` (Enabled: `True`). {{(pid=88114) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:33:21.456271 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.fernet_utils [None req-cf78f341-d1b1-4c79-9ed6-d831e3a5d528 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88114) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:33:21.505429 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2059fd6d-6e42-4a51-96b9-a96e82476aea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:21.506155 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2059fd6d-6e42-4a51-96b9-a96e82476aea None admin] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:21.506443 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2059fd6d-6e42-4a51-96b9-a96e82476aea None admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:21.506723 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2059fd6d-6e42-4a51-96b9-a96e82476aea None admin] PATH_INFO: `/v3/users/1dc7ae39ad83425fb2f7ddb6c98b0a20` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:21.508030 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2059fd6d-6e42-4a51-96b9-a96e82476aea None admin] RBAC: Authorizing `identity:get_user(user_id=1dc7ae39ad83425fb2f7ddb6c98b0a20)` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:33:21.511634 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2059fd6d-6e42-4a51-96b9-a96e82476aea None admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:33:21.514822 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 79/159] 10.222.0.13 () {56 vars in 1177 bytes} [Wed Aug 31 09:33:21 2022] GET /identity/v3/users/1dc7ae39ad83425fb2f7ddb6c98b0a20 => generated 275 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:33:21.515915 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2059fd6d-6e42-4a51-96b9-a96e82476aea None admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:21.631070 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=100 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:33:21.663032 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2059fd6d-6e42-4a51-96b9-a96e82476aea None admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:21.710302 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92257587-d764-42a7-a6b1-fa5df4c552aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:21.710973 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92257587-d764-42a7-a6b1-fa5df4c552aa None admin] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:21.711391 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92257587-d764-42a7-a6b1-fa5df4c552aa None admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:21.711648 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92257587-d764-42a7-a6b1-fa5df4c552aa None admin] PATH_INFO: `/v3/roles/member` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:21.718851 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 81/160] 10.222.0.13 () {58 vars in 1024 bytes} [Wed Aug 31 09:33:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 327 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 09:33:21.719819 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf78f341-d1b1-4c79-9ed6-d831e3a5d528 None None] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:21.725276 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92257587-d764-42a7-a6b1-fa5df4c552aa None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 09:33:21.730452 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application [None req-92257587-d764-42a7-a6b1-fa5df4c552aa None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 31 09:33:21.730452 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:33:21.730452 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:33:21.730452 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:33:21.730452 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:33:21.730452 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:33:21.730452 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:33:21.730452 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:33:21.730452 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:33:21.730452 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:33:21.730452 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:33:21.730452 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:33:21.730452 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 09:33:21.730452 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 09:33:21.730452 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 09:33:21.730452 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 09:33:21.731447 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 09:33:21.731447 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 09:33:21.731447 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 09:33:21.731447 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 09:33:21.731447 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:33:21.731447 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:33:21.731447 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:33:21.731447 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:33:21.731447 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:33:21.731447 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:33:21.731447 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:33:21.731447 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application with Lock( Aug 31 09:33:21.731447 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:33:21.731447 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:33:21.731447 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:33:21.731447 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:33:21.732253 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:33:21.732253 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:33:21.732253 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:33:21.732253 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:33:21.732253 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 09:33:21.732253 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 09:33:21.732253 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 09:33:21.732253 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 09:33:21.732253 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 09:33:21.732253 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 09:33:21.732253 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 31 09:33:21.732253 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application Aug 31 09:33:21.732253 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 80/161] 10.222.0.13 () {56 vars in 1099 bytes} [Wed Aug 31 09:33:21 2022] GET /identity/v3/roles/member => generated 84 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:33:21.738135 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92257587-d764-42a7-a6b1-fa5df4c552aa None admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:21.825913 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=100 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:33:21.874233 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92257587-d764-42a7-a6b1-fa5df4c552aa None admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:21.893581 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf78f341-d1b1-4c79-9ed6-d831e3a5d528 None None] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:21.916034 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=100 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:33:21.926646 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab4e012d-832b-4fe3-9a6d-f9d86f2c0fcd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:21.927301 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4e012d-832b-4fe3-9a6d-f9d86f2c0fcd None admin] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:21.927572 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4e012d-832b-4fe3-9a6d-f9d86f2c0fcd None admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:21.927865 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4e012d-832b-4fe3-9a6d-f9d86f2c0fcd None admin] PATH_INFO: `/v3/roles` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:21.928582 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab4e012d-832b-4fe3-9a6d-f9d86f2c0fcd None admin] RBAC: Adding query filter params (name=member) {{(pid=88113) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:33:21.928894 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab4e012d-832b-4fe3-9a6d-f9d86f2c0fcd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:33:21.930506 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab4e012d-832b-4fe3-9a6d-f9d86f2c0fcd None admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:33:21.945912 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 81/162] 10.222.0.13 () {56 vars in 1101 bytes} [Wed Aug 31 09:33:21 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:33:21.948167 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab4e012d-832b-4fe3-9a6d-f9d86f2c0fcd None admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:21.990254 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-845bb8ec-43cc-46b8-bb99-4fb81b8eaf66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:21.990906 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845bb8ec-43cc-46b8-bb99-4fb81b8eaf66 None admin] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:21.991186 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845bb8ec-43cc-46b8-bb99-4fb81b8eaf66 None admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:21.991464 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845bb8ec-43cc-46b8-bb99-4fb81b8eaf66 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:22.119439 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-845bb8ec-43cc-46b8-bb99-4fb81b8eaf66 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:33:22.120404 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab4e012d-832b-4fe3-9a6d-f9d86f2c0fcd None admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:22.120925 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-845bb8ec-43cc-46b8-bb99-4fb81b8eaf66 None admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:33:22.169125 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb6a623-478c-44f0-81e3-c304aabd9e0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:22.169754 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb6a623-478c-44f0-81e3-c304aabd9e0c None admin] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:22.170034 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb6a623-478c-44f0-81e3-c304aabd9e0c None admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:22.170304 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb6a623-478c-44f0-81e3-c304aabd9e0c None admin] PATH_INFO: `/v3/users/1dc7ae39ad83425fb2f7ddb6c98b0a20` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:22.173549 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eb6a623-478c-44f0-81e3-c304aabd9e0c None admin] RBAC: Authorizing `identity:get_user(user_id=1dc7ae39ad83425fb2f7ddb6c98b0a20)` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:33:22.175779 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eb6a623-478c-44f0-81e3-c304aabd9e0c None admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:33:22.178449 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 82/163] 10.222.0.13 () {56 vars in 1177 bytes} [Wed Aug 31 09:33:21 2022] GET /identity/v3/users/1dc7ae39ad83425fb2f7ddb6c98b0a20 => generated 275 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 09:33:22.179329 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb6a623-478c-44f0-81e3-c304aabd9e0c None admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:22.239301 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=100 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:33:22.274467 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=100 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:33:22.340384 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 82/164] 10.222.0.13 () {58 vars in 1282 bytes} [Wed Aug 31 09:33:21 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 621 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 31 09:33:22.341249 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-845bb8ec-43cc-46b8-bb99-4fb81b8eaf66 None admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:22.376394 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb6a623-478c-44f0-81e3-c304aabd9e0c None admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:22.463816 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-845bb8ec-43cc-46b8-bb99-4fb81b8eaf66 None admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:22.482179 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33219fcf-386b-43d5-aaea-aeab54edb2a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:22.482848 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33219fcf-386b-43d5-aaea-aeab54edb2a0 None admin] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:22.483133 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33219fcf-386b-43d5-aaea-aeab54edb2a0 None admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:22.483415 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33219fcf-386b-43d5-aaea-aeab54edb2a0 None admin] PATH_INFO: `/v3/roles/member` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:22.504308 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33219fcf-386b-43d5-aaea-aeab54edb2a0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 09:33:22.514015 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d5fb6f6-b86e-40db-986e-d2e4b07a0269 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:22.514683 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5fb6f6-b86e-40db-986e-d2e4b07a0269 None admin] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:22.514968 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5fb6f6-b86e-40db-986e-d2e4b07a0269 None admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:22.515237 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5fb6f6-b86e-40db-986e-d2e4b07a0269 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:22.518975 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application [None req-33219fcf-386b-43d5-aaea-aeab54edb2a0 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 31 09:33:22.518975 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:33:22.518975 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:33:22.518975 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:33:22.518975 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:33:22.518975 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:33:22.518975 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:33:22.518975 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:33:22.518975 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:33:22.518975 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:33:22.518975 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:33:22.518975 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:33:22.518975 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 09:33:22.518975 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 09:33:22.518975 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 09:33:22.518975 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 09:33:22.519861 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 09:33:22.519861 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 09:33:22.519861 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 09:33:22.519861 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 09:33:22.519861 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:33:22.519861 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:33:22.519861 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:33:22.519861 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:33:22.519861 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:33:22.519861 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:33:22.519861 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:33:22.519861 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application with Lock( Aug 31 09:33:22.519861 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:33:22.519861 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:33:22.519861 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:33:22.519861 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:33:22.521095 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:33:22.521095 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:33:22.521095 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:33:22.521095 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:33:22.521095 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 09:33:22.521095 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 09:33:22.521095 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 09:33:22.521095 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 09:33:22.521095 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 09:33:22.521095 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 09:33:22.521095 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 31 09:33:22.521095 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application Aug 31 09:33:22.521095 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 83/165] 10.222.0.13 () {56 vars in 1099 bytes} [Wed Aug 31 09:33:22 2022] GET /identity/v3/roles/member => generated 84 bytes in 342 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:33:22.522711 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33219fcf-386b-43d5-aaea-aeab54edb2a0 None admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:22.553714 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=100 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:33:22.601729 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d5fb6f6-b86e-40db-986e-d2e4b07a0269 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:33:22.603318 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d5fb6f6-b86e-40db-986e-d2e4b07a0269 None admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:33:22.727012 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 83/166] 10.222.0.13 () {58 vars in 1282 bytes} [Wed Aug 31 09:33:22 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 386 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 31 09:33:22.728376 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d5fb6f6-b86e-40db-986e-d2e4b07a0269 None admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:22.787218 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33219fcf-386b-43d5-aaea-aeab54edb2a0 None admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:22.808434 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=100 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:33:22.813447 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=100 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:33:22.858169 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0cfd836-844a-4cab-b1cc-56a3a193451b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:22.858408 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d5fb6f6-b86e-40db-986e-d2e4b07a0269 None admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:22.858840 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0cfd836-844a-4cab-b1cc-56a3a193451b None admin] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:22.859125 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0cfd836-844a-4cab-b1cc-56a3a193451b None admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:22.859400 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0cfd836-844a-4cab-b1cc-56a3a193451b None admin] PATH_INFO: `/v3/projects/1001686acf834f2a8e65b3fc8369f4f6` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:22.860699 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0cfd836-844a-4cab-b1cc-56a3a193451b None admin] RBAC: Authorizing `identity:get_project(project_id=1001686acf834f2a8e65b3fc8369f4f6)` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:33:22.862559 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0cfd836-844a-4cab-b1cc-56a3a193451b None admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:33:22.864621 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 84/167] 10.222.0.13 () {56 vars in 1186 bytes} [Wed Aug 31 09:33:22 2022] GET /identity/v3/projects/1001686acf834f2a8e65b3fc8369f4f6 => generated 298 bytes in 343 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:33:22.865504 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0cfd836-844a-4cab-b1cc-56a3a193451b None admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:22.899772 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0848b588-b779-433e-bf28-5a972b0eff85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:22.900419 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0848b588-b779-433e-bf28-5a972b0eff85 None admin] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:22.900698 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0848b588-b779-433e-bf28-5a972b0eff85 None admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:22.900965 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0848b588-b779-433e-bf28-5a972b0eff85 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:22.914109 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0848b588-b779-433e-bf28-5a972b0eff85 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 09:33:22.919561 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application [None req-0848b588-b779-433e-bf28-5a972b0eff85 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 31 09:33:22.919561 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:33:22.919561 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:33:22.919561 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:33:22.919561 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:33:22.919561 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:33:22.919561 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:33:22.919561 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:33:22.919561 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:33:22.919561 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:33:22.919561 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:33:22.919561 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:33:22.919561 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 09:33:22.919561 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 09:33:22.919561 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 09:33:22.919561 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 09:33:22.920459 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 09:33:22.920459 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 09:33:22.920459 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 09:33:22.920459 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 09:33:22.920459 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:33:22.920459 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:33:22.920459 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:33:22.920459 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:33:22.920459 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:33:22.920459 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:33:22.920459 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:33:22.920459 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application with Lock( Aug 31 09:33:22.920459 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:33:22.920459 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:33:22.920459 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:33:22.920459 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:33:22.921314 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:33:22.921314 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:33:22.921314 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:33:22.921314 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:33:22.921314 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 09:33:22.921314 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 09:33:22.921314 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 09:33:22.921314 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 09:33:22.921314 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 09:33:22.921314 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 09:33:22.921314 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 31 09:33:22.921314 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application Aug 31 09:33:22.921314 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 84/168] 10.222.0.13 () {56 vars in 1099 bytes} [Wed Aug 31 09:33:22 2022] GET /identity/v3/roles/reader => generated 84 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:33:22.921984 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0848b588-b779-433e-bf28-5a972b0eff85 None admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:23.042204 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0848b588-b779-433e-bf28-5a972b0eff85 None admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:23.070465 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=100 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:33:23.085079 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0cfd836-844a-4cab-b1cc-56a3a193451b None admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:23.087075 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be3c4975-4f0d-47e7-aa01-9210ff2479c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:23.087800 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3c4975-4f0d-47e7-aa01-9210ff2479c0 None admin] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:23.088081 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3c4975-4f0d-47e7-aa01-9210ff2479c0 None admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:23.088359 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3c4975-4f0d-47e7-aa01-9210ff2479c0 None admin] PATH_INFO: `/v3/projects/1001686acf834f2a8e65b3fc8369f4f6` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:23.089574 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be3c4975-4f0d-47e7-aa01-9210ff2479c0 None admin] RBAC: Authorizing `identity:get_project(project_id=1001686acf834f2a8e65b3fc8369f4f6)` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:33:23.091069 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be3c4975-4f0d-47e7-aa01-9210ff2479c0 None admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:33:23.092831 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 85/169] 10.222.0.13 () {56 vars in 1186 bytes} [Wed Aug 31 09:33:22 2022] GET /identity/v3/projects/1001686acf834f2a8e65b3fc8369f4f6 => generated 298 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:33:23.099009 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be3c4975-4f0d-47e7-aa01-9210ff2479c0 None admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:23.154765 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5115fce2-82e6-4531-a85c-089bede75bc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:23.155394 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5115fce2-82e6-4531-a85c-089bede75bc6 None admin] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:23.155675 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5115fce2-82e6-4531-a85c-089bede75bc6 None admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:23.155950 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5115fce2-82e6-4531-a85c-089bede75bc6 None admin] PATH_INFO: `/v3/roles` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:23.156630 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5115fce2-82e6-4531-a85c-089bede75bc6 None admin] RBAC: Adding query filter params (name=member) {{(pid=88113) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:33:23.156933 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5115fce2-82e6-4531-a85c-089bede75bc6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:33:23.159389 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5115fce2-82e6-4531-a85c-089bede75bc6 None admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:33:23.171383 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 85/170] 10.222.0.13 () {56 vars in 1101 bytes} [Wed Aug 31 09:33:22 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 306 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:33:23.172247 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5115fce2-82e6-4531-a85c-089bede75bc6 None admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:23.214339 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be3c4975-4f0d-47e7-aa01-9210ff2479c0 None admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:23.236022 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=100 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:33:23.256007 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9280c66d-2337-40cc-8481-04b0402baf2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:23.256720 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9280c66d-2337-40cc-8481-04b0402baf2c None admin] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:23.257089 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9280c66d-2337-40cc-8481-04b0402baf2c None admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:23.257417 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9280c66d-2337-40cc-8481-04b0402baf2c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:23.258227 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9280c66d-2337-40cc-8481-04b0402baf2c None admin] RBAC: Adding query filter params (role.id=889857b056624d47b572d4a80ecc3f8e, scope.project.id=1001686acf834f2a8e65b3fc8369f4f6, user.id=1dc7ae39ad83425fb2f7ddb6c98b0a20) {{(pid=88114) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:33:23.258549 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9280c66d-2337-40cc-8481-04b0402baf2c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:33:23.260221 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9280c66d-2337-40cc-8481-04b0402baf2c None admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:33:23.272750 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 86/171] 10.222.0.13 () {56 vars in 1374 bytes} [Wed Aug 31 09:33:23 2022] GET /identity/v3/role_assignments?user.id=1dc7ae39ad83425fb2f7ddb6c98b0a20&scope.project.id=1001686acf834f2a8e65b3fc8369f4f6&role.id=889857b056624d47b572d4a80ecc3f8e => generated 608 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:33:23.272750 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e916e1f-e65d-4b29-84bb-e9e29a0fbf27 None None] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:23.272750 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e916e1f-e65d-4b29-84bb-e9e29a0fbf27 None None] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:23.272750 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e916e1f-e65d-4b29-84bb-e9e29a0fbf27 None None] PATH_INFO: `/` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:23.272750 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 87/172] 10.222.0.13 () {54 vars in 927 bytes} [Wed Aug 31 09:33:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:33:23.272750 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e916e1f-e65d-4b29-84bb-e9e29a0fbf27 None None] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:23.331142 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5115fce2-82e6-4531-a85c-089bede75bc6 None admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:23.381115 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e41da146-0d9f-4d54-a54c-459e6aa5eccc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:23.382285 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41da146-0d9f-4d54-a54c-459e6aa5eccc None admin] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:23.382778 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41da146-0d9f-4d54-a54c-459e6aa5eccc None admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:23.383248 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41da146-0d9f-4d54-a54c-459e6aa5eccc None admin] PATH_INFO: `/v3/roles` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:23.384175 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e41da146-0d9f-4d54-a54c-459e6aa5eccc None admin] RBAC: Adding query filter params (name=reader) {{(pid=88113) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:33:23.384683 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e41da146-0d9f-4d54-a54c-459e6aa5eccc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:33:23.386405 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e41da146-0d9f-4d54-a54c-459e6aa5eccc None admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:33:23.399637 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 86/173] 10.222.0.13 () {56 vars in 1101 bytes} [Wed Aug 31 09:33:23 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:33:23.400848 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e41da146-0d9f-4d54-a54c-459e6aa5eccc None admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:23.449868 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=100 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:33:23.533094 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e916e1f-e65d-4b29-84bb-e9e29a0fbf27 None None] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:23.575829 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-566e1ccb-1528-429a-bf3e-54d9d71a4c33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:23.576941 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566e1ccb-1528-429a-bf3e-54d9d71a4c33 None admin] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:23.577501 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566e1ccb-1528-429a-bf3e-54d9d71a4c33 None admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:23.577974 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566e1ccb-1528-429a-bf3e-54d9d71a4c33 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:23.578973 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-566e1ccb-1528-429a-bf3e-54d9d71a4c33 None admin] RBAC: Adding query filter params (role.id=6c99f5d9f1f243eaa134d8540f21095d, scope.project.id=1001686acf834f2a8e65b3fc8369f4f6, user.id=1dc7ae39ad83425fb2f7ddb6c98b0a20) {{(pid=88114) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:33:23.580111 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-566e1ccb-1528-429a-bf3e-54d9d71a4c33 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:33:23.582206 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-566e1ccb-1528-429a-bf3e-54d9d71a4c33 None admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:33:23.589171 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e41da146-0d9f-4d54-a54c-459e6aa5eccc None admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:23.589878 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 88/174] 10.222.0.13 () {56 vars in 1374 bytes} [Wed Aug 31 09:33:23 2022] GET /identity/v3/role_assignments?user.id=1dc7ae39ad83425fb2f7ddb6c98b0a20&scope.project.id=1001686acf834f2a8e65b3fc8369f4f6&role.id=6c99f5d9f1f243eaa134d8540f21095d => generated 608 bytes in 318 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:33:23.590960 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d113ac27-d39a-45d0-9d35-dbf16720fa4c None None] REQUEST_METHOD: `POST` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:23.591243 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d113ac27-d39a-45d0-9d35-dbf16720fa4c None None] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:23.591517 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d113ac27-d39a-45d0-9d35-dbf16720fa4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:23.613727 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=100 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:33:23.616190 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.auth.core [None req-d113ac27-d39a-45d0-9d35-dbf16720fa4c None None] MFA Rules not processed for user `fd836cf590a145df9acf19cf3ae2ac56`. Rule list: `[]` (Enabled: `True`). {{(pid=88114) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:33:23.617546 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.fernet_utils [None req-d113ac27-d39a-45d0-9d35-dbf16720fa4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88114) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:33:23.668217 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e866e53-0d22-498f-8b9e-664e29e1f584 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:23.668806 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e866e53-0d22-498f-8b9e-664e29e1f584 None admin] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:23.669080 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e866e53-0d22-498f-8b9e-664e29e1f584 None admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:23.669352 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e866e53-0d22-498f-8b9e-664e29e1f584 None admin] PATH_INFO: `/v3/users/1dc7ae39ad83425fb2f7ddb6c98b0a20` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:23.670752 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e866e53-0d22-498f-8b9e-664e29e1f584 None admin] RBAC: Authorizing `identity:get_user(user_id=1dc7ae39ad83425fb2f7ddb6c98b0a20)` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:33:23.672283 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e866e53-0d22-498f-8b9e-664e29e1f584 None admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:33:23.678318 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 87/175] 10.222.0.13 () {56 vars in 1177 bytes} [Wed Aug 31 09:33:23 2022] GET /identity/v3/users/1dc7ae39ad83425fb2f7ddb6c98b0a20 => generated 275 bytes in 278 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:33:23.679209 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e866e53-0d22-498f-8b9e-664e29e1f584 None admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:23.727268 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 89/176] 10.222.0.13 () {58 vars in 1024 bytes} [Wed Aug 31 09:33:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 09:33:23.728578 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e494da2f-08f9-4e50-a12d-9f0a854e85f6 None None] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:23.728862 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e494da2f-08f9-4e50-a12d-9f0a854e85f6 None None] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:23.729133 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e494da2f-08f9-4e50-a12d-9f0a854e85f6 None None] PATH_INFO: `/` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:23.731016 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 90/177] 10.222.0.13 () {54 vars in 927 bytes} [Wed Aug 31 09:33:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:33:23.731813 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e494da2f-08f9-4e50-a12d-9f0a854e85f6 None None] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:23.808678 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=100 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:33:23.835037 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e866e53-0d22-498f-8b9e-664e29e1f584 None admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:23.889781 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d88f84b9-38e8-4e4f-a328-be86838612ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:23.890434 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88f84b9-38e8-4e4f-a328-be86838612ba None admin] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:23.890711 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88f84b9-38e8-4e4f-a328-be86838612ba None admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:23.890981 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88f84b9-38e8-4e4f-a328-be86838612ba None admin] PATH_INFO: `/v3/users/32b40e2909174cf086806d3a85cd5cc1` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:23.893100 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d88f84b9-38e8-4e4f-a328-be86838612ba None admin] RBAC: Authorizing `identity:get_user(user_id=32b40e2909174cf086806d3a85cd5cc1)` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:33:23.896093 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d88f84b9-38e8-4e4f-a328-be86838612ba None admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:33:23.899326 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 88/178] 10.222.0.13 () {56 vars in 1177 bytes} [Wed Aug 31 09:33:23 2022] GET /identity/v3/users/32b40e2909174cf086806d3a85cd5cc1 => generated 289 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:33:23.899592 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e494da2f-08f9-4e50-a12d-9f0a854e85f6 None None] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:23.906365 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9d8187-53e4-4a0a-8909-8764f38541d5 None None] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:23.906664 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9d8187-53e4-4a0a-8909-8764f38541d5 None None] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:23.906910 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9d8187-53e4-4a0a-8909-8764f38541d5 None None] PATH_INFO: `/` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:23.909419 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 89/179] 10.222.0.13 () {54 vars in 927 bytes} [Wed Aug 31 09:33:23 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:33:23.910987 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cedb1121-3599-4d19-a4a2-ebd8fbae8637 None None] REQUEST_METHOD: `POST` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:23.911268 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cedb1121-3599-4d19-a4a2-ebd8fbae8637 None None] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:23.911536 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cedb1121-3599-4d19-a4a2-ebd8fbae8637 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:23.934276 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.auth.core [None req-cedb1121-3599-4d19-a4a2-ebd8fbae8637 None None] MFA Rules not processed for user `fd836cf590a145df9acf19cf3ae2ac56`. Rule list: `[]` (Enabled: `True`). {{(pid=88113) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:33:23.935480 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.fernet_utils [None req-cedb1121-3599-4d19-a4a2-ebd8fbae8637 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:33:23.951900 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c231e25c-d957-4e6b-b867-116b8c50d2e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:23.952523 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c231e25c-d957-4e6b-b867-116b8c50d2e2 None admin] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:23.952800 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c231e25c-d957-4e6b-b867-116b8c50d2e2 None admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:23.953072 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c231e25c-d957-4e6b-b867-116b8c50d2e2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:23.989148 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=100 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:33:24.037683 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c231e25c-d957-4e6b-b867-116b8c50d2e2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:33:24.039181 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c231e25c-d957-4e6b-b867-116b8c50d2e2 None admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:33:24.049410 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 90/180] 10.222.0.13 () {58 vars in 1024 bytes} [Wed Aug 31 09:33:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 09:33:24.049410 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cedb1121-3599-4d19-a4a2-ebd8fbae8637 None None] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:24.102129 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=100 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:33:24.169400 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 91/181] 10.222.0.13 () {58 vars in 1282 bytes} [Wed Aug 31 09:33:23 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 433 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 31 09:33:24.169400 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8c35e6-4789-40ba-b6ad-98db615f6ef3 None None] REQUEST_METHOD: `POST` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:24.169400 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8c35e6-4789-40ba-b6ad-98db615f6ef3 None None] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:24.169400 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8c35e6-4789-40ba-b6ad-98db615f6ef3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:24.186330 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.auth.core [None req-5e8c35e6-4789-40ba-b6ad-98db615f6ef3 None None] MFA Rules not processed for user `fd836cf590a145df9acf19cf3ae2ac56`. Rule list: `[]` (Enabled: `True`). {{(pid=88114) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:33:24.187752 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.fernet_utils [None req-5e8c35e6-4789-40ba-b6ad-98db615f6ef3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88114) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:33:24.190355 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cedb1121-3599-4d19-a4a2-ebd8fbae8637 None None] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:24.228479 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a60c852-8425-4293-af86-7da565e7a162 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:24.229589 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a60c852-8425-4293-af86-7da565e7a162 None admin] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:24.229865 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a60c852-8425-4293-af86-7da565e7a162 None admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:24.230138 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a60c852-8425-4293-af86-7da565e7a162 None admin] PATH_INFO: `/v3/projects/1001686acf834f2a8e65b3fc8369f4f6` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:24.231369 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a60c852-8425-4293-af86-7da565e7a162 None admin] RBAC: Authorizing `identity:get_project(project_id=1001686acf834f2a8e65b3fc8369f4f6)` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:33:24.232816 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a60c852-8425-4293-af86-7da565e7a162 None admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:33:24.240427 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 91/182] 10.222.0.13 () {56 vars in 1186 bytes} [Wed Aug 31 09:33:24 2022] GET /identity/v3/projects/1001686acf834f2a8e65b3fc8369f4f6 => generated 298 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:33:24.240427 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a60c852-8425-4293-af86-7da565e7a162 None admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:24.281421 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 92/183] 10.222.0.13 () {58 vars in 1024 bytes} [Wed Aug 31 09:33:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 09:33:24.281421 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e8c35e6-4789-40ba-b6ad-98db615f6ef3 None None] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:24.301697 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=100 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:33:24.372395 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a60c852-8425-4293-af86-7da565e7a162 None admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:24.390570 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=100 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:33:24.428296 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e8c35e6-4789-40ba-b6ad-98db615f6ef3 None None] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:24.434659 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84698343-0d76-4ae5-be10-60e1e460c549 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:24.435864 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84698343-0d76-4ae5-be10-60e1e460c549 None admin] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:24.436486 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84698343-0d76-4ae5-be10-60e1e460c549 None admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:24.437033 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84698343-0d76-4ae5-be10-60e1e460c549 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:24.459525 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84698343-0d76-4ae5-be10-60e1e460c549 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 09:33:24.471723 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application [None req-84698343-0d76-4ae5-be10-60e1e460c549 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 09:33:24.471723 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:33:24.471723 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:33:24.471723 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:33:24.471723 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:33:24.471723 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:33:24.471723 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:33:24.471723 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:33:24.471723 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:33:24.471723 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:33:24.471723 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:33:24.471723 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:33:24.471723 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 09:33:24.471723 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 09:33:24.471723 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 09:33:24.471723 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 09:33:24.472661 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 09:33:24.472661 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 09:33:24.472661 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 09:33:24.472661 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 09:33:24.472661 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:33:24.472661 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:33:24.472661 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:33:24.472661 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:33:24.472661 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:33:24.472661 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:33:24.472661 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:33:24.472661 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application with Lock( Aug 31 09:33:24.472661 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:33:24.472661 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:33:24.472661 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:33:24.472661 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:33:24.473475 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:33:24.473475 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:33:24.473475 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:33:24.473475 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:33:24.473475 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 09:33:24.473475 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 09:33:24.473475 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 09:33:24.473475 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 09:33:24.473475 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 09:33:24.473475 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 09:33:24.473475 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 09:33:24.473475 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application Aug 31 09:33:24.474553 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 92/184] 10.222.0.13 () {56 vars in 1096 bytes} [Wed Aug 31 09:33:24 2022] GET /identity/v3/roles/admin => generated 83 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:33:24.476057 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84698343-0d76-4ae5-be10-60e1e460c549 None admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:24.486400 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19cca16f-c8d7-4a7f-8bb0-8e0df8e11f0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:24.487063 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19cca16f-c8d7-4a7f-8bb0-8e0df8e11f0f None admin] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:24.487343 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19cca16f-c8d7-4a7f-8bb0-8e0df8e11f0f None admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:24.487617 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19cca16f-c8d7-4a7f-8bb0-8e0df8e11f0f None admin] PATH_INFO: `/v3/projects/9f41d93b9d96487f867195c8bdd23cbc` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:24.488807 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19cca16f-c8d7-4a7f-8bb0-8e0df8e11f0f None admin] RBAC: Authorizing `identity:get_project(project_id=9f41d93b9d96487f867195c8bdd23cbc)` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:33:24.490845 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19cca16f-c8d7-4a7f-8bb0-8e0df8e11f0f None admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:33:24.493367 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 93/185] 10.222.0.13 () {56 vars in 1186 bytes} [Wed Aug 31 09:33:24 2022] GET /identity/v3/projects/9f41d93b9d96487f867195c8bdd23cbc => generated 312 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:33:24.494062 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19cca16f-c8d7-4a7f-8bb0-8e0df8e11f0f None admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:24.591050 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84698343-0d76-4ae5-be10-60e1e460c549 None admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:24.592754 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=100 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:33:24.636824 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f337d34f-8276-48ed-b340-bc86ec93d421 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:24.637442 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f337d34f-8276-48ed-b340-bc86ec93d421 None admin] REQUEST_METHOD: `PUT` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:24.637724 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f337d34f-8276-48ed-b340-bc86ec93d421 None admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:24.638000 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f337d34f-8276-48ed-b340-bc86ec93d421 None admin] PATH_INFO: `/v3/projects/1001686acf834f2a8e65b3fc8369f4f6/users/32b40e2909174cf086806d3a85cd5cc1/roles/9814fca799974e3e87ffc04473b6510e` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:24.639945 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f337d34f-8276-48ed-b340-bc86ec93d421 None admin] RBAC: Authorizing `identity:create_grant(project_id=1001686acf834f2a8e65b3fc8369f4f6, user_id=32b40e2909174cf086806d3a85cd5cc1, role_id=9814fca799974e3e87ffc04473b6510e)` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:33:24.641418 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f337d34f-8276-48ed-b340-bc86ec93d421 None admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:33:24.655985 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 93/186] 10.222.0.13 () {58 vars in 1439 bytes} [Wed Aug 31 09:33:24 2022] PUT /identity/v3/projects/1001686acf834f2a8e65b3fc8369f4f6/users/32b40e2909174cf086806d3a85cd5cc1/roles/9814fca799974e3e87ffc04473b6510e => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:33:24.656820 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f337d34f-8276-48ed-b340-bc86ec93d421 None admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:24.683780 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19cca16f-c8d7-4a7f-8bb0-8e0df8e11f0f None admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:24.744726 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c974c749-a66c-4294-ac88-c2065a8f4b7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:24.745775 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c974c749-a66c-4294-ac88-c2065a8f4b7d None admin] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:24.745775 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c974c749-a66c-4294-ac88-c2065a8f4b7d None admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:24.746007 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c974c749-a66c-4294-ac88-c2065a8f4b7d None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:24.761888 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c974c749-a66c-4294-ac88-c2065a8f4b7d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 09:33:24.772854 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application [None req-c974c749-a66c-4294-ac88-c2065a8f4b7d None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 31 09:33:24.772854 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:33:24.772854 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:33:24.772854 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:33:24.772854 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:33:24.772854 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:33:24.772854 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:33:24.772854 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:33:24.772854 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:33:24.772854 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:33:24.772854 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:33:24.772854 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:33:24.772854 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 09:33:24.772854 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 09:33:24.772854 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 09:33:24.772854 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 09:33:24.774808 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 09:33:24.774808 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 09:33:24.774808 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 09:33:24.774808 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 09:33:24.774808 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:33:24.774808 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:33:24.774808 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:33:24.774808 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:33:24.774808 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:33:24.774808 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:33:24.774808 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:33:24.774808 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application with Lock( Aug 31 09:33:24.774808 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:33:24.774808 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:33:24.774808 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:33:24.774808 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:33:24.776349 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:33:24.776349 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:33:24.776349 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:33:24.776349 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:33:24.776349 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 09:33:24.776349 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 09:33:24.776349 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 09:33:24.776349 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 09:33:24.776349 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 09:33:24.776349 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 09:33:24.776349 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 31 09:33:24.776349 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application Aug 31 09:33:24.776349 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 94/187] 10.222.0.13 () {56 vars in 1114 bytes} [Wed Aug 31 09:33:24 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 281 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:33:24.776349 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c974c749-a66c-4294-ac88-c2065a8f4b7d None admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:24.790168 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=100 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:33:24.797817 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f337d34f-8276-48ed-b340-bc86ec93d421 None admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:24.853090 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e37a24b-76b7-4555-a913-9b6b82173409 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:24.853742 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e37a24b-76b7-4555-a913-9b6b82173409 None admin] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:24.854026 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e37a24b-76b7-4555-a913-9b6b82173409 None admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:24.854302 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e37a24b-76b7-4555-a913-9b6b82173409 None admin] PATH_INFO: `/v3/roles` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:24.854995 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e37a24b-76b7-4555-a913-9b6b82173409 None admin] RBAC: Adding query filter params (name=admin) {{(pid=88113) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:33:24.855309 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e37a24b-76b7-4555-a913-9b6b82173409 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:33:24.856703 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e37a24b-76b7-4555-a913-9b6b82173409 None admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:33:24.867360 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 94/188] 10.222.0.13 () {56 vars in 1099 bytes} [Wed Aug 31 09:33:24 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:33:24.869274 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e37a24b-76b7-4555-a913-9b6b82173409 None admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:24.914387 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:33:24.937560 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c974c749-a66c-4294-ac88-c2065a8f4b7d None admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:24.991767 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c81a8ca-517a-45d7-b132-ba53f74bc228 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:24.992696 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c81a8ca-517a-45d7-b132-ba53f74bc228 None admin] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:24.993263 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c81a8ca-517a-45d7-b132-ba53f74bc228 None admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:24.993750 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c81a8ca-517a-45d7-b132-ba53f74bc228 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:24.995093 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c81a8ca-517a-45d7-b132-ba53f74bc228 None admin] RBAC: Adding query filter params (role.id=6c99f5d9f1f243eaa134d8540f21095d, scope.project.id=9f41d93b9d96487f867195c8bdd23cbc, user.id=1dc7ae39ad83425fb2f7ddb6c98b0a20) {{(pid=88114) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:33:24.995741 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c81a8ca-517a-45d7-b132-ba53f74bc228 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:33:24.997696 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c81a8ca-517a-45d7-b132-ba53f74bc228 None admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:33:25.015050 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 95/189] 10.222.0.13 () {56 vars in 1374 bytes} [Wed Aug 31 09:33:24 2022] GET /identity/v3/role_assignments?user.id=1dc7ae39ad83425fb2f7ddb6c98b0a20&scope.project.id=9f41d93b9d96487f867195c8bdd23cbc&role.id=6c99f5d9f1f243eaa134d8540f21095d => generated 608 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:33:25.015309 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c81a8ca-517a-45d7-b132-ba53f74bc228 None admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:25.028726 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e37a24b-76b7-4555-a913-9b6b82173409 None admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:25.102398 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dad2dff-80e4-41e0-b34b-ec7d8184531a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:25.103779 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dad2dff-80e4-41e0-b34b-ec7d8184531a None admin] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:25.104290 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dad2dff-80e4-41e0-b34b-ec7d8184531a None admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:25.104711 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dad2dff-80e4-41e0-b34b-ec7d8184531a None admin] PATH_INFO: `/v3/domains/default` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:25.107411 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dad2dff-80e4-41e0-b34b-ec7d8184531a None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:33:25.111894 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dad2dff-80e4-41e0-b34b-ec7d8184531a None admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:33:25.115911 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 95/190] 10.222.0.13 () {56 vars in 1108 bytes} [Wed Aug 31 09:33:24 2022] GET /identity/v3/domains/default => generated 199 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:33:25.123158 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dad2dff-80e4-41e0-b34b-ec7d8184531a None admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:25.166601 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=100 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:33:25.213723 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c81a8ca-517a-45d7-b132-ba53f74bc228 None admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:25.276863 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ca53d9-a791-428f-a387-85b82af35721 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:25.277529 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ca53d9-a791-428f-a387-85b82af35721 None admin] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:25.277814 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ca53d9-a791-428f-a387-85b82af35721 None admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:25.278090 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ca53d9-a791-428f-a387-85b82af35721 None admin] PATH_INFO: `/v3/roles` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:25.278761 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29ca53d9-a791-428f-a387-85b82af35721 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=88114) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:33:25.279228 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29ca53d9-a791-428f-a387-85b82af35721 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:33:25.280523 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29ca53d9-a791-428f-a387-85b82af35721 None admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:33:25.288227 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=100 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:33:25.290951 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 96/191] 10.222.0.13 () {56 vars in 1111 bytes} [Wed Aug 31 09:33:25 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 277 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:33:25.291893 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ca53d9-a791-428f-a387-85b82af35721 None admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:25.299595 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dad2dff-80e4-41e0-b34b-ec7d8184531a None admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:25.349977 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e278a1c-e4ce-4d16-a9b2-8f6816b02d46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:25.350688 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e278a1c-e4ce-4d16-a9b2-8f6816b02d46 None admin] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:25.350966 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e278a1c-e4ce-4d16-a9b2-8f6816b02d46 None admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:25.351228 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e278a1c-e4ce-4d16-a9b2-8f6816b02d46 None admin] PATH_INFO: `/v3/users/6dbf545ccbb14b2fb85b5e32ba1726fd` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:25.369988 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e278a1c-e4ce-4d16-a9b2-8f6816b02d46 None admin] RBAC: Authorizing `identity:get_user(user_id=6dbf545ccbb14b2fb85b5e32ba1726fd)` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:33:25.371481 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e278a1c-e4ce-4d16-a9b2-8f6816b02d46 None admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:33:25.373213 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 96/192] 10.222.0.13 () {56 vars in 1177 bytes} [Wed Aug 31 09:33:25 2022] GET /identity/v3/users/6dbf545ccbb14b2fb85b5e32ba1726fd => generated 283 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:33:25.374047 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e278a1c-e4ce-4d16-a9b2-8f6816b02d46 None admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:25.423053 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ca53d9-a791-428f-a387-85b82af35721 None admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:25.461168 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58faaab5-79d0-4136-b5ca-83bbfae9d073 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:25.462243 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58faaab5-79d0-4136-b5ca-83bbfae9d073 None admin] REQUEST_METHOD: `POST` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:25.462657 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58faaab5-79d0-4136-b5ca-83bbfae9d073 None admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:25.462829 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58faaab5-79d0-4136-b5ca-83bbfae9d073 None admin] PATH_INFO: `/v3/users` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:25.463454 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58faaab5-79d0-4136-b5ca-83bbfae9d073 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:33:25.465616 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58faaab5-79d0-4136-b5ca-83bbfae9d073 None admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:33:25.477977 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=100 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:33:25.489492 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e278a1c-e4ce-4d16-a9b2-8f6816b02d46 None admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:25.509415 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 97/193] 10.222.0.13 () {60 vars in 1132 bytes} [Wed Aug 31 09:33:25 2022] POST /identity/v3/users => generated 297 bytes in 208 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 09:33:25.509415 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58faaab5-79d0-4136-b5ca-83bbfae9d073 None admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:25.529213 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a363dc-129d-4134-bd05-14104642e48f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:25.529937 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a363dc-129d-4134-bd05-14104642e48f None admin] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:25.530214 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a363dc-129d-4134-bd05-14104642e48f None admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:25.530484 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a363dc-129d-4134-bd05-14104642e48f None admin] PATH_INFO: `/v3/users/6dbf545ccbb14b2fb85b5e32ba1726fd` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:25.531704 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4a363dc-129d-4134-bd05-14104642e48f None admin] RBAC: Authorizing `identity:get_user(user_id=6dbf545ccbb14b2fb85b5e32ba1726fd)` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:33:25.533291 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4a363dc-129d-4134-bd05-14104642e48f None admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:33:25.540095 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 97/194] 10.222.0.13 () {56 vars in 1177 bytes} [Wed Aug 31 09:33:25 2022] GET /identity/v3/users/6dbf545ccbb14b2fb85b5e32ba1726fd => generated 283 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:33:25.541013 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a363dc-129d-4134-bd05-14104642e48f None admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:25.563407 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=100 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:33:25.674948 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58faaab5-79d0-4136-b5ca-83bbfae9d073 None admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:25.680569 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a363dc-129d-4134-bd05-14104642e48f None admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:25.713845 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f7d9122-10de-4604-ad0b-5dba9913fe7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:25.714461 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7d9122-10de-4604-ad0b-5dba9913fe7b None admin] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:25.714735 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7d9122-10de-4604-ad0b-5dba9913fe7b None admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:25.715003 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7d9122-10de-4604-ad0b-5dba9913fe7b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:25.730846 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9796a706-ee84-4683-8a4c-2ecadb8223fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:25.731494 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9796a706-ee84-4683-8a4c-2ecadb8223fa None admin] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:25.731779 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9796a706-ee84-4683-8a4c-2ecadb8223fa None admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:25.732989 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9796a706-ee84-4683-8a4c-2ecadb8223fa None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:25.797482 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:33:25.821911 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f7d9122-10de-4604-ad0b-5dba9913fe7b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:33:25.823388 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f7d9122-10de-4604-ad0b-5dba9913fe7b None admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:33:25.837655 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9796a706-ee84-4683-8a4c-2ecadb8223fa None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:33:25.839141 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9796a706-ee84-4683-8a4c-2ecadb8223fa None admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:33:25.870953 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=100 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:33:25.964227 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 98/195] 10.222.0.13 () {58 vars in 1282 bytes} [Wed Aug 31 09:33:25 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 464 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 31 09:33:25.970357 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f7d9122-10de-4604-ad0b-5dba9913fe7b None admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:25.984262 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 98/196] 10.222.0.13 () {58 vars in 1282 bytes} [Wed Aug 31 09:33:25 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 444 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 31 09:33:25.992450 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9796a706-ee84-4683-8a4c-2ecadb8223fa None admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:26.120652 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9796a706-ee84-4683-8a4c-2ecadb8223fa None admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:26.122050 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=100 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:33:26.145013 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=100 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:33:26.151587 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f7d9122-10de-4604-ad0b-5dba9913fe7b None admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:26.179991 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9da259a-ea5c-415d-912b-6dbfcdb1a89d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:26.180645 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9da259a-ea5c-415d-912b-6dbfcdb1a89d None admin] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:26.180931 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9da259a-ea5c-415d-912b-6dbfcdb1a89d None admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:26.181202 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9da259a-ea5c-415d-912b-6dbfcdb1a89d None admin] PATH_INFO: `/v3/projects/5258cb5ffb9c49b1ba3cfb741aef69f8` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:26.183515 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9da259a-ea5c-415d-912b-6dbfcdb1a89d None admin] RBAC: Authorizing `identity:get_project(project_id=5258cb5ffb9c49b1ba3cfb741aef69f8)` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:33:26.185060 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9da259a-ea5c-415d-912b-6dbfcdb1a89d None admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:33:26.186884 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 99/197] 10.222.0.13 () {56 vars in 1186 bytes} [Wed Aug 31 09:33:25 2022] GET /identity/v3/projects/5258cb5ffb9c49b1ba3cfb741aef69f8 => generated 302 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 09:33:26.195293 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9da259a-ea5c-415d-912b-6dbfcdb1a89d None admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:26.227475 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16cda176-5f18-4789-a7fc-df2b64580dd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:26.228118 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16cda176-5f18-4789-a7fc-df2b64580dd0 None admin] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:26.228437 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16cda176-5f18-4789-a7fc-df2b64580dd0 None admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:26.228711 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16cda176-5f18-4789-a7fc-df2b64580dd0 None admin] PATH_INFO: `/v3/projects/5258cb5ffb9c49b1ba3cfb741aef69f8` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:26.230846 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16cda176-5f18-4789-a7fc-df2b64580dd0 None admin] RBAC: Authorizing `identity:get_project(project_id=5258cb5ffb9c49b1ba3cfb741aef69f8)` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:33:26.233289 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16cda176-5f18-4789-a7fc-df2b64580dd0 None admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:33:26.238527 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 99/198] 10.222.0.13 () {56 vars in 1186 bytes} [Wed Aug 31 09:33:25 2022] GET /identity/v3/projects/5258cb5ffb9c49b1ba3cfb741aef69f8 => generated 302 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:33:26.242169 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16cda176-5f18-4789-a7fc-df2b64580dd0 None admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:26.365344 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9da259a-ea5c-415d-912b-6dbfcdb1a89d None admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:26.438154 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce7acec-7e71-40bf-8e55-aa5edfd25392 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:26.439269 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16cda176-5f18-4789-a7fc-df2b64580dd0 None admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:26.439651 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce7acec-7e71-40bf-8e55-aa5edfd25392 None admin] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:26.440129 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce7acec-7e71-40bf-8e55-aa5edfd25392 None admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:26.440539 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce7acec-7e71-40bf-8e55-aa5edfd25392 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:26.441293 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ce7acec-7e71-40bf-8e55-aa5edfd25392 None admin] RBAC: Adding query filter params (role.id=889857b056624d47b572d4a80ecc3f8e, scope.project.id=5258cb5ffb9c49b1ba3cfb741aef69f8, user.id=6dbf545ccbb14b2fb85b5e32ba1726fd) {{(pid=88113) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:33:26.441647 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ce7acec-7e71-40bf-8e55-aa5edfd25392 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:33:26.443129 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ce7acec-7e71-40bf-8e55-aa5edfd25392 None admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:33:26.459797 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 100/199] 10.222.0.13 () {56 vars in 1374 bytes} [Wed Aug 31 09:33:26 2022] GET /identity/v3/role_assignments?user.id=6dbf545ccbb14b2fb85b5e32ba1726fd&scope.project.id=5258cb5ffb9c49b1ba3cfb741aef69f8&role.id=889857b056624d47b572d4a80ecc3f8e => generated 259 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:33:26.494572 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ffd1ce-6cec-44fe-b6e7-c8c04b1a8ca1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:26.495124 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ffd1ce-6cec-44fe-b6e7-c8c04b1a8ca1 None admin] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:26.495456 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ffd1ce-6cec-44fe-b6e7-c8c04b1a8ca1 None admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:26.495734 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ffd1ce-6cec-44fe-b6e7-c8c04b1a8ca1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:26.496510 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4ffd1ce-6cec-44fe-b6e7-c8c04b1a8ca1 None admin] RBAC: Adding query filter params (role.id=5d4ee63a3add47fc9a6b8b16d39e0a1f, scope.project.id=5258cb5ffb9c49b1ba3cfb741aef69f8, user.id=6dbf545ccbb14b2fb85b5e32ba1726fd) {{(pid=88114) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:33:26.496819 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4ffd1ce-6cec-44fe-b6e7-c8c04b1a8ca1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:33:26.498414 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4ffd1ce-6cec-44fe-b6e7-c8c04b1a8ca1 None admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:33:26.510433 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 100/200] 10.222.0.13 () {56 vars in 1374 bytes} [Wed Aug 31 09:33:26 2022] GET /identity/v3/role_assignments?user.id=6dbf545ccbb14b2fb85b5e32ba1726fd&scope.project.id=5258cb5ffb9c49b1ba3cfb741aef69f8&role.id=5d4ee63a3add47fc9a6b8b16d39e0a1f => generated 259 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 09:33:26.893079 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c56320c-9b0c-420e-8202-2eb1d07310cc None None] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:26.893536 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c56320c-9b0c-420e-8202-2eb1d07310cc None None] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:26.893882 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c56320c-9b0c-420e-8202-2eb1d07310cc None None] PATH_INFO: `/` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:26.895969 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 101/201] 10.222.0.13 () {54 vars in 927 bytes} [Wed Aug 31 09:33:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:33:26.902157 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87d45ab-9baf-4dd6-be4a-fd1c5fbcfd0f None None] REQUEST_METHOD: `POST` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:26.902675 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87d45ab-9baf-4dd6-be4a-fd1c5fbcfd0f None None] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:26.902675 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87d45ab-9baf-4dd6-be4a-fd1c5fbcfd0f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:26.922648 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.auth.core [None req-d87d45ab-9baf-4dd6-be4a-fd1c5fbcfd0f None None] MFA Rules not processed for user `fd836cf590a145df9acf19cf3ae2ac56`. Rule list: `[]` (Enabled: `True`). {{(pid=88114) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:33:26.924734 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.fernet_utils [None req-d87d45ab-9baf-4dd6-be4a-fd1c5fbcfd0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88114) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:33:26.944487 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=100 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:33:27.017616 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 101/202] 10.222.0.13 () {58 vars in 1024 bytes} [Wed Aug 31 09:33:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 09:33:27.435921 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c56320c-9b0c-420e-8202-2eb1d07310cc None None] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:27.508926 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=100 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:33:27.572225 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c56320c-9b0c-420e-8202-2eb1d07310cc None None] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:27.608575 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0daaa63a-28e6-4603-b16f-744e54de45fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:27.609211 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0daaa63a-28e6-4603-b16f-744e54de45fa None admin] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:27.609505 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0daaa63a-28e6-4603-b16f-744e54de45fa None admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:27.609780 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0daaa63a-28e6-4603-b16f-744e54de45fa None admin] PATH_INFO: `/v3/roles/reader` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:27.621588 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0daaa63a-28e6-4603-b16f-744e54de45fa None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 09:33:27.627918 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application [None req-0daaa63a-28e6-4603-b16f-744e54de45fa None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 31 09:33:27.627918 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:33:27.627918 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:33:27.627918 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:33:27.627918 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:33:27.627918 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:33:27.627918 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:33:27.627918 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:33:27.627918 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:33:27.627918 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:33:27.627918 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:33:27.627918 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:33:27.627918 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 09:33:27.627918 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 09:33:27.627918 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 09:33:27.627918 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 09:33:27.628801 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 09:33:27.628801 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 09:33:27.628801 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 09:33:27.628801 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 09:33:27.628801 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:33:27.628801 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:33:27.628801 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:33:27.628801 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:33:27.628801 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:33:27.628801 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:33:27.628801 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:33:27.628801 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application with Lock( Aug 31 09:33:27.628801 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:33:27.628801 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:33:27.628801 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:33:27.628801 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:33:27.629609 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:33:27.629609 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:33:27.629609 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:33:27.629609 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:33:27.629609 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 09:33:27.629609 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 09:33:27.629609 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 09:33:27.629609 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 09:33:27.629609 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 09:33:27.629609 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 09:33:27.629609 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 31 09:33:27.629609 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application Aug 31 09:33:27.629609 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 102/203] 10.222.0.13 () {56 vars in 1099 bytes} [Wed Aug 31 09:33:27 2022] GET /identity/v3/roles/reader => generated 84 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:33:27.639062 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d87d45ab-9baf-4dd6-be4a-fd1c5fbcfd0f None None] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:27.790169 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d87d45ab-9baf-4dd6-be4a-fd1c5fbcfd0f None None] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:27.831788 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f766bab1-bfa7-408d-b7b2-c1d1e1417021 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:27.832499 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f766bab1-bfa7-408d-b7b2-c1d1e1417021 None admin] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:27.832784 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f766bab1-bfa7-408d-b7b2-c1d1e1417021 None admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:27.833056 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f766bab1-bfa7-408d-b7b2-c1d1e1417021 None admin] PATH_INFO: `/v3/roles` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:27.834167 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f766bab1-bfa7-408d-b7b2-c1d1e1417021 None admin] RBAC: Adding query filter params (name=reader) {{(pid=88114) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:33:27.834292 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f766bab1-bfa7-408d-b7b2-c1d1e1417021 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:33:27.835916 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f766bab1-bfa7-408d-b7b2-c1d1e1417021 None admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:33:27.847079 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 102/204] 10.222.0.13 () {56 vars in 1101 bytes} [Wed Aug 31 09:33:27 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:33:27.852853 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0daaa63a-28e6-4603-b16f-744e54de45fa None admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:27.951313 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf064ec-02d8-4461-8465-4a9d44e409f4 None None] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:27.951812 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf064ec-02d8-4461-8465-4a9d44e409f4 None None] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:27.951884 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf064ec-02d8-4461-8465-4a9d44e409f4 None None] PATH_INFO: `/` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:27.953195 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 103/205] 10.222.0.13 () {54 vars in 927 bytes} [Wed Aug 31 09:33:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 31 09:33:27.959031 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35d1534-81f0-42ec-9d27-8e63242044b6 None None] REQUEST_METHOD: `POST` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:27.959307 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35d1534-81f0-42ec-9d27-8e63242044b6 None None] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:27.959579 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35d1534-81f0-42ec-9d27-8e63242044b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:27.989479 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.auth.core [None req-e35d1534-81f0-42ec-9d27-8e63242044b6 None None] MFA Rules not processed for user `fd836cf590a145df9acf19cf3ae2ac56`. Rule list: `[]` (Enabled: `True`). {{(pid=88114) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:33:27.990686 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.fernet_utils [None req-e35d1534-81f0-42ec-9d27-8e63242044b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88114) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:33:28.013780 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=100 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:33:28.031565 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0daaa63a-28e6-4603-b16f-744e54de45fa None admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:28.051062 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=100 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:33:28.103417 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 104/206] 10.222.0.13 () {58 vars in 1024 bytes} [Wed Aug 31 09:33:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 363 bytes (3 switches on core 0) Aug 31 09:33:28.138135 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e24abc-e2ec-41a0-a143-920ad5c59c40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:28.138800 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e24abc-e2ec-41a0-a143-920ad5c59c40 None admin] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:28.139084 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e24abc-e2ec-41a0-a143-920ad5c59c40 None admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:28.139362 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e24abc-e2ec-41a0-a143-920ad5c59c40 None admin] PATH_INFO: `/v3/users/32b40e2909174cf086806d3a85cd5cc1` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:28.140607 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5e24abc-e2ec-41a0-a143-920ad5c59c40 None admin] RBAC: Authorizing `identity:get_user(user_id=32b40e2909174cf086806d3a85cd5cc1)` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:33:28.146147 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5e24abc-e2ec-41a0-a143-920ad5c59c40 None admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:33:28.147902 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 103/207] 10.222.0.13 () {56 vars in 1177 bytes} [Wed Aug 31 09:33:27 2022] GET /identity/v3/users/32b40e2909174cf086806d3a85cd5cc1 => generated 289 bytes in 296 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:33:28.153945 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e35d1534-81f0-42ec-9d27-8e63242044b6 None None] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:28.323595 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e35d1534-81f0-42ec-9d27-8e63242044b6 None None] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:28.361114 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf84f6da-d67d-4087-a792-95a79cc42cfb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:28.361761 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf84f6da-d67d-4087-a792-95a79cc42cfb None admin] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:28.362041 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf84f6da-d67d-4087-a792-95a79cc42cfb None admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:28.362355 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf84f6da-d67d-4087-a792-95a79cc42cfb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:28.378039 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=100 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:33:28.445639 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf84f6da-d67d-4087-a792-95a79cc42cfb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:33:28.447423 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf84f6da-d67d-4087-a792-95a79cc42cfb None admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:33:28.484243 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3b8b87-d4a2-400e-84a4-9aebe464e0e1 None None] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:28.485178 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3b8b87-d4a2-400e-84a4-9aebe464e0e1 None None] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:28.485795 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3b8b87-d4a2-400e-84a4-9aebe464e0e1 None None] PATH_INFO: `/` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:28.487336 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 104/208] 10.222.0.13 () {54 vars in 927 bytes} [Wed Aug 31 09:33:28 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 31 09:33:28.492706 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782a0613-475f-4999-a5f9-33b843763036 None None] REQUEST_METHOD: `POST` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:28.493456 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782a0613-475f-4999-a5f9-33b843763036 None None] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:28.493928 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782a0613-475f-4999-a5f9-33b843763036 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:28.516539 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.auth.core [None req-782a0613-475f-4999-a5f9-33b843763036 None None] MFA Rules not processed for user `fd836cf590a145df9acf19cf3ae2ac56`. Rule list: `[]` (Enabled: `True`). {{(pid=88113) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:33:28.518417 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.fernet_utils [None req-782a0613-475f-4999-a5f9-33b843763036 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:33:28.620873 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 105/209] 10.222.0.13 () {58 vars in 1024 bytes} [Wed Aug 31 09:33:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 31 09:33:28.621691 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-782a0613-475f-4999-a5f9-33b843763036 None None] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:28.624577 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 105/210] 10.222.0.13 () {58 vars in 1282 bytes} [Wed Aug 31 09:33:28 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 471 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 31 09:33:28.638466 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf84f6da-d67d-4087-a792-95a79cc42cfb None admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:28.741795 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=100 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:33:28.744176 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-782a0613-475f-4999-a5f9-33b843763036 None None] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:28.755833 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=100 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:33:28.804528 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf84f6da-d67d-4087-a792-95a79cc42cfb None admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:28.825183 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f285cfb8-77ed-40fb-ae22-2cd1987a1a7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:28.826320 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f285cfb8-77ed-40fb-ae22-2cd1987a1a7f None admin] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:28.826825 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f285cfb8-77ed-40fb-ae22-2cd1987a1a7f None admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:28.827299 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f285cfb8-77ed-40fb-ae22-2cd1987a1a7f None admin] PATH_INFO: `/v3/roles/member` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:28.853398 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93310b80-707b-4f29-ad87-8a87086b64c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:28.853850 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93310b80-707b-4f29-ad87-8a87086b64c7 None admin] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:28.854117 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93310b80-707b-4f29-ad87-8a87086b64c7 None admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:28.854387 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93310b80-707b-4f29-ad87-8a87086b64c7 None admin] PATH_INFO: `/v3/projects/1001686acf834f2a8e65b3fc8369f4f6` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:28.856194 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93310b80-707b-4f29-ad87-8a87086b64c7 None admin] RBAC: Authorizing `identity:get_project(project_id=1001686acf834f2a8e65b3fc8369f4f6)` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:33:28.858065 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f285cfb8-77ed-40fb-ae22-2cd1987a1a7f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 09:33:28.863565 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application [None req-f285cfb8-77ed-40fb-ae22-2cd1987a1a7f None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 31 09:33:28.863565 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:33:28.863565 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:33:28.863565 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:33:28.863565 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:33:28.863565 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:33:28.863565 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:33:28.863565 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:33:28.863565 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:33:28.863565 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:33:28.863565 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:33:28.863565 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:33:28.863565 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 09:33:28.863565 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 09:33:28.863565 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 09:33:28.863565 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 09:33:28.864486 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 09:33:28.864486 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 09:33:28.864486 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 09:33:28.864486 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 09:33:28.864486 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:33:28.864486 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:33:28.864486 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:33:28.864486 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:33:28.864486 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:33:28.864486 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:33:28.864486 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:33:28.864486 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application with Lock( Aug 31 09:33:28.864486 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:33:28.864486 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:33:28.864486 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:33:28.864486 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:33:28.865734 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:33:28.865734 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:33:28.865734 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:33:28.865734 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:33:28.865734 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 09:33:28.865734 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 09:33:28.865734 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 09:33:28.865734 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 09:33:28.865734 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 09:33:28.865734 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 09:33:28.865734 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 31 09:33:28.865734 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application Aug 31 09:33:28.865734 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 106/211] 10.222.0.13 () {56 vars in 1099 bytes} [Wed Aug 31 09:33:28 2022] GET /identity/v3/roles/member => generated 84 bytes in 243 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:33:28.866390 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93310b80-707b-4f29-ad87-8a87086b64c7 None admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:33:28.867453 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 106/212] 10.222.0.13 () {56 vars in 1186 bytes} [Wed Aug 31 09:33:28 2022] GET /identity/v3/projects/1001686acf834f2a8e65b3fc8369f4f6 => generated 298 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:33:28.871113 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dfd7cb1-35cc-47d2-b2e5-6f281745261a None None] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:28.871338 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dfd7cb1-35cc-47d2-b2e5-6f281745261a None None] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:28.871579 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dfd7cb1-35cc-47d2-b2e5-6f281745261a None None] PATH_INFO: `/` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:28.873132 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 107/213] 10.222.0.13 () {54 vars in 927 bytes} [Wed Aug 31 09:33:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 31 09:33:28.878611 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8b386d-f27b-4d6b-92f5-c16d0b2ede1a None None] REQUEST_METHOD: `POST` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:28.878892 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8b386d-f27b-4d6b-92f5-c16d0b2ede1a None None] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:28.879165 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8b386d-f27b-4d6b-92f5-c16d0b2ede1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:28.881068 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dfd7cb1-35cc-47d2-b2e5-6f281745261a None None] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:28.898594 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.auth.core [None req-3b8b386d-f27b-4d6b-92f5-c16d0b2ede1a None None] MFA Rules not processed for user `fd836cf590a145df9acf19cf3ae2ac56`. Rule list: `[]` (Enabled: `True`). {{(pid=88114) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:33:28.900071 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.fernet_utils [None req-3b8b386d-f27b-4d6b-92f5-c16d0b2ede1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88114) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:33:28.983266 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 107/214] 10.222.0.13 () {58 vars in 1024 bytes} [Wed Aug 31 09:33:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 31 09:33:28.984317 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b8b386d-f27b-4d6b-92f5-c16d0b2ede1a None None] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:29.013434 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dfd7cb1-35cc-47d2-b2e5-6f281745261a None None] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:29.045221 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=100 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:33:29.056011 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa89ae17-ab82-4f87-85cc-b941f9b4aa64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:29.056943 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa89ae17-ab82-4f87-85cc-b941f9b4aa64 None admin] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:29.057479 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa89ae17-ab82-4f87-85cc-b941f9b4aa64 None admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:29.057969 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa89ae17-ab82-4f87-85cc-b941f9b4aa64 None admin] PATH_INFO: `/v3/roles` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:29.058939 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa89ae17-ab82-4f87-85cc-b941f9b4aa64 None admin] RBAC: Adding query filter params (name=member) {{(pid=88113) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:33:29.059512 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa89ae17-ab82-4f87-85cc-b941f9b4aa64 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:33:29.061122 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa89ae17-ab82-4f87-85cc-b941f9b4aa64 None admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:33:29.075983 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 108/215] 10.222.0.13 () {56 vars in 1101 bytes} [Wed Aug 31 09:33:28 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:33:29.077268 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa89ae17-ab82-4f87-85cc-b941f9b4aa64 None admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:29.105635 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b8b386d-f27b-4d6b-92f5-c16d0b2ede1a None None] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:29.128218 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=100 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:33:29.144500 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af2f291e-897b-49c0-87ff-62d087763318 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:29.145658 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2f291e-897b-49c0-87ff-62d087763318 None admin] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:29.146105 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2f291e-897b-49c0-87ff-62d087763318 None admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:29.146543 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2f291e-897b-49c0-87ff-62d087763318 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:29.147461 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af2f291e-897b-49c0-87ff-62d087763318 None admin] RBAC: Adding query filter params (role.id=9814fca799974e3e87ffc04473b6510e, scope.project.id=1001686acf834f2a8e65b3fc8369f4f6, user.id=32b40e2909174cf086806d3a85cd5cc1) {{(pid=88114) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:33:29.148098 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af2f291e-897b-49c0-87ff-62d087763318 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:33:29.150586 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af2f291e-897b-49c0-87ff-62d087763318 None admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:33:29.158633 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 108/216] 10.222.0.13 () {56 vars in 1374 bytes} [Wed Aug 31 09:33:28 2022] GET /identity/v3/role_assignments?user.id=32b40e2909174cf086806d3a85cd5cc1&scope.project.id=1001686acf834f2a8e65b3fc8369f4f6&role.id=9814fca799974e3e87ffc04473b6510e => generated 608 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:33:29.160026 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af2f291e-897b-49c0-87ff-62d087763318 None admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:29.284367 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa89ae17-ab82-4f87-85cc-b941f9b4aa64 None admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:29.333633 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af2f291e-897b-49c0-87ff-62d087763318 None admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:29.336675 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6cf3ead-9c82-4405-88f3-bd23161af25c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:29.337420 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6cf3ead-9c82-4405-88f3-bd23161af25c None admin] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:29.337864 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6cf3ead-9c82-4405-88f3-bd23161af25c None admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:29.338311 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6cf3ead-9c82-4405-88f3-bd23161af25c None admin] PATH_INFO: `/v3/domains/default` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:29.339697 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6cf3ead-9c82-4405-88f3-bd23161af25c None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:33:29.341477 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6cf3ead-9c82-4405-88f3-bd23161af25c None admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:33:29.343687 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 109/217] 10.222.0.13 () {56 vars in 1108 bytes} [Wed Aug 31 09:33:29 2022] GET /identity/v3/domains/default => generated 199 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:33:29.344774 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6cf3ead-9c82-4405-88f3-bd23161af25c None admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:29.347239 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=100 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:33:29.376220 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c011d6e8-865a-467b-a8a0-3ddfffea1f71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:29.376980 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c011d6e8-865a-467b-a8a0-3ddfffea1f71 None admin] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:29.377256 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c011d6e8-865a-467b-a8a0-3ddfffea1f71 None admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:29.377563 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c011d6e8-865a-467b-a8a0-3ddfffea1f71 None admin] PATH_INFO: `/v3/users/676c06c0cb4141a080531e4762087dea` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:29.393521 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c011d6e8-865a-467b-a8a0-3ddfffea1f71 None admin] RBAC: Authorizing `identity:get_user(user_id=676c06c0cb4141a080531e4762087dea)` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:33:29.394931 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c011d6e8-865a-467b-a8a0-3ddfffea1f71 None admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:33:29.397084 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 109/218] 10.222.0.13 () {56 vars in 1177 bytes} [Wed Aug 31 09:33:29 2022] GET /identity/v3/users/676c06c0cb4141a080531e4762087dea => generated 297 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:33:29.398545 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c011d6e8-865a-467b-a8a0-3ddfffea1f71 None admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:29.468833 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=100 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:33:29.474532 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6cf3ead-9c82-4405-88f3-bd23161af25c None admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:29.510515 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f98fa61-0d0b-4945-8b71-6a42c731990b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:29.511192 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f98fa61-0d0b-4945-8b71-6a42c731990b None admin] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:29.511412 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f98fa61-0d0b-4945-8b71-6a42c731990b None admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:29.511680 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f98fa61-0d0b-4945-8b71-6a42c731990b None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:29.514176 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c011d6e8-865a-467b-a8a0-3ddfffea1f71 None admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:29.522128 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f98fa61-0d0b-4945-8b71-6a42c731990b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 09:33:29.527370 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application [None req-5f98fa61-0d0b-4945-8b71-6a42c731990b None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 31 09:33:29.527370 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:33:29.527370 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:33:29.527370 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:33:29.527370 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:33:29.527370 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:33:29.527370 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:33:29.527370 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:33:29.527370 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:33:29.527370 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:33:29.527370 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:33:29.527370 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:33:29.527370 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 09:33:29.527370 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 09:33:29.527370 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 09:33:29.527370 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 09:33:29.528244 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 09:33:29.528244 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 09:33:29.528244 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 09:33:29.528244 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 09:33:29.528244 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:33:29.528244 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:33:29.528244 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:33:29.528244 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:33:29.528244 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:33:29.528244 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:33:29.528244 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:33:29.528244 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application with Lock( Aug 31 09:33:29.528244 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:33:29.528244 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:33:29.528244 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:33:29.528244 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:33:29.529675 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:33:29.529675 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:33:29.529675 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:33:29.529675 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:33:29.529675 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 09:33:29.529675 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 09:33:29.529675 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 09:33:29.529675 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 09:33:29.529675 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 09:33:29.529675 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 09:33:29.529675 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 31 09:33:29.529675 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application Aug 31 09:33:29.529675 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 110/219] 10.222.0.13 () {56 vars in 1114 bytes} [Wed Aug 31 09:33:29 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:33:29.529675 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f98fa61-0d0b-4945-8b71-6a42c731990b None admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:29.549297 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e799cc3-cd33-42fc-a63f-a80effa8a466 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:29.549926 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e799cc3-cd33-42fc-a63f-a80effa8a466 None admin] REQUEST_METHOD: `POST` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:29.550197 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e799cc3-cd33-42fc-a63f-a80effa8a466 None admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:29.550462 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e799cc3-cd33-42fc-a63f-a80effa8a466 None admin] PATH_INFO: `/v3/users` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:29.551103 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e799cc3-cd33-42fc-a63f-a80effa8a466 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:33:29.552530 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e799cc3-cd33-42fc-a63f-a80effa8a466 None admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:33:29.574281 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 110/220] 10.222.0.13 () {60 vars in 1132 bytes} [Wed Aug 31 09:33:29 2022] POST /identity/v3/users => generated 297 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 09:33:29.575610 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f685dac-4b87-4916-9308-269842f772ed None None] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:29.575904 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f685dac-4b87-4916-9308-269842f772ed None None] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:29.576181 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f685dac-4b87-4916-9308-269842f772ed None None] PATH_INFO: `/` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:29.577613 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 111/221] 10.222.0.13 () {54 vars in 927 bytes} [Wed Aug 31 09:33:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:33:29.578238 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f685dac-4b87-4916-9308-269842f772ed None None] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:29.612251 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:33:29.664374 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f98fa61-0d0b-4945-8b71-6a42c731990b None admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:29.704078 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae395d9-3437-4bda-905c-ebeb658e8a95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:29.704744 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae395d9-3437-4bda-905c-ebeb658e8a95 None admin] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:29.705064 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae395d9-3437-4bda-905c-ebeb658e8a95 None admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:29.705564 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae395d9-3437-4bda-905c-ebeb658e8a95 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:29.711603 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f685dac-4b87-4916-9308-269842f772ed None None] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:29.735316 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=100 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:33:29.755560 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-804ba845-2293-488b-a5db-6dc4261f32a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:29.756190 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804ba845-2293-488b-a5db-6dc4261f32a5 None admin] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:29.756464 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804ba845-2293-488b-a5db-6dc4261f32a5 None admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:29.756732 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804ba845-2293-488b-a5db-6dc4261f32a5 None admin] PATH_INFO: `/v3/roles` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:29.757437 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-804ba845-2293-488b-a5db-6dc4261f32a5 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=88114) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:33:29.757752 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-804ba845-2293-488b-a5db-6dc4261f32a5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:33:29.759129 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-804ba845-2293-488b-a5db-6dc4261f32a5 None admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:33:29.771693 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 112/222] 10.222.0.13 () {56 vars in 1111 bytes} [Wed Aug 31 09:33:29 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:33:29.772773 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8709c9-65cd-4a95-8ca0-761cc9184cc7 None None] REQUEST_METHOD: `POST` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:29.773064 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8709c9-65cd-4a95-8ca0-761cc9184cc7 None None] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:29.773803 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8709c9-65cd-4a95-8ca0-761cc9184cc7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:29.793836 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fae395d9-3437-4bda-905c-ebeb658e8a95 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:33:29.794024 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.auth.core [None req-de8709c9-65cd-4a95-8ca0-761cc9184cc7 None None] MFA Rules not processed for user `fd836cf590a145df9acf19cf3ae2ac56`. Rule list: `[]` (Enabled: `True`). {{(pid=88114) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:33:29.795210 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.fernet_utils [None req-de8709c9-65cd-4a95-8ca0-761cc9184cc7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88114) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:33:29.795338 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fae395d9-3437-4bda-905c-ebeb658e8a95 None admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:33:29.884562 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 113/223] 10.222.0.13 () {58 vars in 1024 bytes} [Wed Aug 31 09:33:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 09:33:29.885968 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de8709c9-65cd-4a95-8ca0-761cc9184cc7 None None] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:29.903225 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=100 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:33:29.931143 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 111/224] 10.222.0.13 () {58 vars in 1282 bytes} [Wed Aug 31 09:33:29 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 401 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 31 09:33:29.936660 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae395d9-3437-4bda-905c-ebeb658e8a95 None admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:30.003713 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=100 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:33:30.006548 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de8709c9-65cd-4a95-8ca0-761cc9184cc7 None None] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:30.044159 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-460312d9-3217-45c1-a2ea-dcfc427f8625 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:30.044996 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460312d9-3217-45c1-a2ea-dcfc427f8625 None admin] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:30.045377 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460312d9-3217-45c1-a2ea-dcfc427f8625 None admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:30.045774 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460312d9-3217-45c1-a2ea-dcfc427f8625 None admin] PATH_INFO: `/v3/users/6dbf545ccbb14b2fb85b5e32ba1726fd` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:30.047070 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-460312d9-3217-45c1-a2ea-dcfc427f8625 None admin] RBAC: Authorizing `identity:get_user(user_id=6dbf545ccbb14b2fb85b5e32ba1726fd)` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:33:30.049277 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-460312d9-3217-45c1-a2ea-dcfc427f8625 None admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:33:30.051200 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 114/225] 10.222.0.13 () {56 vars in 1177 bytes} [Wed Aug 31 09:33:29 2022] GET /identity/v3/users/6dbf545ccbb14b2fb85b5e32ba1726fd => generated 283 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:33:30.052130 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae395d9-3437-4bda-905c-ebeb658e8a95 None admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:30.057023 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-460312d9-3217-45c1-a2ea-dcfc427f8625 None admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:30.092940 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff86d2bb-7413-48a1-be54-968e9357c392 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:30.093619 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff86d2bb-7413-48a1-be54-968e9357c392 None admin] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:30.093899 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff86d2bb-7413-48a1-be54-968e9357c392 None admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:30.094171 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff86d2bb-7413-48a1-be54-968e9357c392 None admin] PATH_INFO: `/v3/projects/5258cb5ffb9c49b1ba3cfb741aef69f8` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:30.095454 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff86d2bb-7413-48a1-be54-968e9357c392 None admin] RBAC: Authorizing `identity:get_project(project_id=5258cb5ffb9c49b1ba3cfb741aef69f8)` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:33:30.097212 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff86d2bb-7413-48a1-be54-968e9357c392 None admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:33:30.103864 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 112/226] 10.222.0.13 () {56 vars in 1186 bytes} [Wed Aug 31 09:33:29 2022] GET /identity/v3/projects/5258cb5ffb9c49b1ba3cfb741aef69f8 => generated 302 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 09:33:30.105227 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff86d2bb-7413-48a1-be54-968e9357c392 None admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:30.146009 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=100 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:33:30.166490 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-460312d9-3217-45c1-a2ea-dcfc427f8625 None admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:30.202497 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb06bce0-3251-43f5-a9ef-1d5d5233e55c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:30.203147 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb06bce0-3251-43f5-a9ef-1d5d5233e55c None admin] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:30.203489 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb06bce0-3251-43f5-a9ef-1d5d5233e55c None admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:30.203827 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb06bce0-3251-43f5-a9ef-1d5d5233e55c None admin] PATH_INFO: `/v3/projects/5258cb5ffb9c49b1ba3cfb741aef69f8` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:30.205830 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb06bce0-3251-43f5-a9ef-1d5d5233e55c None admin] RBAC: Authorizing `identity:get_project(project_id=5258cb5ffb9c49b1ba3cfb741aef69f8)` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:33:30.207350 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb06bce0-3251-43f5-a9ef-1d5d5233e55c None admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:33:30.209071 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 115/227] 10.222.0.13 () {56 vars in 1186 bytes} [Wed Aug 31 09:33:30 2022] GET /identity/v3/projects/5258cb5ffb9c49b1ba3cfb741aef69f8 => generated 302 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 09:33:30.214665 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb06bce0-3251-43f5-a9ef-1d5d5233e55c None admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:30.218922 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff86d2bb-7413-48a1-be54-968e9357c392 None admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:30.259473 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=100 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:33:30.273320 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed057592-cefc-4472-8f55-f927f32a8a8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:30.274518 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed057592-cefc-4472-8f55-f927f32a8a8a None admin] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:30.274947 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed057592-cefc-4472-8f55-f927f32a8a8a None admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:30.275361 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed057592-cefc-4472-8f55-f927f32a8a8a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:30.277013 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed057592-cefc-4472-8f55-f927f32a8a8a None admin] RBAC: Adding query filter params (role.id=6c99f5d9f1f243eaa134d8540f21095d, scope.project.id=5258cb5ffb9c49b1ba3cfb741aef69f8, user.id=676c06c0cb4141a080531e4762087dea) {{(pid=88113) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:33:30.277512 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed057592-cefc-4472-8f55-f927f32a8a8a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:33:30.279633 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed057592-cefc-4472-8f55-f927f32a8a8a None admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:33:30.287832 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 113/228] 10.222.0.13 () {56 vars in 1374 bytes} [Wed Aug 31 09:33:30 2022] GET /identity/v3/role_assignments?user.id=676c06c0cb4141a080531e4762087dea&scope.project.id=5258cb5ffb9c49b1ba3cfb741aef69f8&role.id=6c99f5d9f1f243eaa134d8540f21095d => generated 259 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:33:30.289410 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed057592-cefc-4472-8f55-f927f32a8a8a None admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:30.344974 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb06bce0-3251-43f5-a9ef-1d5d5233e55c None admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:30.390742 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bfc5334-6106-4a05-a3c0-017bb738312a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:30.391423 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bfc5334-6106-4a05-a3c0-017bb738312a None admin] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:30.392280 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bfc5334-6106-4a05-a3c0-017bb738312a None admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:30.392611 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bfc5334-6106-4a05-a3c0-017bb738312a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:30.405605 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bfc5334-6106-4a05-a3c0-017bb738312a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 09:33:30.411697 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application [None req-1bfc5334-6106-4a05-a3c0-017bb738312a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 09:33:30.411697 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:33:30.411697 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:33:30.411697 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:33:30.411697 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:33:30.411697 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:33:30.411697 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:33:30.411697 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:33:30.411697 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:33:30.411697 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:33:30.411697 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:33:30.411697 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:33:30.411697 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 09:33:30.411697 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 09:33:30.411697 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 09:33:30.411697 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 09:33:30.412667 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 09:33:30.412667 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 09:33:30.412667 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 09:33:30.412667 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 09:33:30.412667 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:33:30.412667 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:33:30.412667 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:33:30.412667 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:33:30.412667 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:33:30.412667 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:33:30.412667 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:33:30.412667 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application with Lock( Aug 31 09:33:30.412667 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:33:30.412667 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:33:30.412667 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:33:30.412667 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:33:30.413488 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:33:30.413488 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:33:30.413488 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:33:30.413488 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:33:30.413488 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 09:33:30.413488 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 09:33:30.413488 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 09:33:30.413488 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 09:33:30.413488 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 09:33:30.413488 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 09:33:30.413488 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 09:33:30.413488 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application Aug 31 09:33:30.413488 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 116/229] 10.222.0.13 () {56 vars in 1096 bytes} [Wed Aug 31 09:33:30 2022] GET /identity/v3/roles/admin => generated 83 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:33:30.419006 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bfc5334-6106-4a05-a3c0-017bb738312a None admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:30.436180 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=100 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:33:30.437881 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed057592-cefc-4472-8f55-f927f32a8a8a None admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:30.482685 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d0d068c-8224-4338-a7ad-f05ab4fc0343 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:30.483723 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0d068c-8224-4338-a7ad-f05ab4fc0343 None admin] REQUEST_METHOD: `PUT` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:30.484353 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0d068c-8224-4338-a7ad-f05ab4fc0343 None admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:30.484987 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0d068c-8224-4338-a7ad-f05ab4fc0343 None admin] PATH_INFO: `/v3/projects/5258cb5ffb9c49b1ba3cfb741aef69f8/users/6dbf545ccbb14b2fb85b5e32ba1726fd/roles/889857b056624d47b572d4a80ecc3f8e` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:30.489947 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d0d068c-8224-4338-a7ad-f05ab4fc0343 None admin] RBAC: Authorizing `identity:create_grant(project_id=5258cb5ffb9c49b1ba3cfb741aef69f8, user_id=6dbf545ccbb14b2fb85b5e32ba1726fd, role_id=889857b056624d47b572d4a80ecc3f8e)` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:33:30.491780 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d0d068c-8224-4338-a7ad-f05ab4fc0343 None admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:33:30.511293 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 114/230] 10.222.0.13 () {58 vars in 1439 bytes} [Wed Aug 31 09:33:30 2022] PUT /identity/v3/projects/5258cb5ffb9c49b1ba3cfb741aef69f8/users/6dbf545ccbb14b2fb85b5e32ba1726fd/roles/889857b056624d47b572d4a80ecc3f8e => generated 0 bytes in 222 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:33:30.586159 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bfc5334-6106-4a05-a3c0-017bb738312a None admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:30.629135 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dcad81c-370d-4f54-9da7-4a2daa747757 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:30.630275 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dcad81c-370d-4f54-9da7-4a2daa747757 None admin] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:30.630801 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dcad81c-370d-4f54-9da7-4a2daa747757 None admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:30.631275 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dcad81c-370d-4f54-9da7-4a2daa747757 None admin] PATH_INFO: `/v3/roles` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:30.632290 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dcad81c-370d-4f54-9da7-4a2daa747757 None admin] RBAC: Adding query filter params (name=admin) {{(pid=88114) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:33:30.632830 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dcad81c-370d-4f54-9da7-4a2daa747757 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:33:30.634472 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dcad81c-370d-4f54-9da7-4a2daa747757 None admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:33:30.646261 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 117/231] 10.222.0.13 () {56 vars in 1099 bytes} [Wed Aug 31 09:33:30 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 09:33:30.656897 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d0d068c-8224-4338-a7ad-f05ab4fc0343 None admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:30.730613 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:33:30.789898 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d0d068c-8224-4338-a7ad-f05ab4fc0343 None admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:30.826227 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69935ee8-d2d0-4aca-b068-ae636216767c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:30.826868 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69935ee8-d2d0-4aca-b068-ae636216767c None admin] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:30.827147 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69935ee8-d2d0-4aca-b068-ae636216767c None admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:30.827427 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69935ee8-d2d0-4aca-b068-ae636216767c None admin] PATH_INFO: `/v3/users/6dbf545ccbb14b2fb85b5e32ba1726fd` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:30.828581 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69935ee8-d2d0-4aca-b068-ae636216767c None admin] RBAC: Authorizing `identity:get_user(user_id=6dbf545ccbb14b2fb85b5e32ba1726fd)` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:33:30.830854 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69935ee8-d2d0-4aca-b068-ae636216767c None admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:33:30.832883 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 115/232] 10.222.0.13 () {56 vars in 1177 bytes} [Wed Aug 31 09:33:30 2022] GET /identity/v3/users/6dbf545ccbb14b2fb85b5e32ba1726fd => generated 283 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:33:30.838571 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dcad81c-370d-4f54-9da7-4a2daa747757 None admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:30.962157 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=100 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:33:31.007673 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dcad81c-370d-4f54-9da7-4a2daa747757 None admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:31.065083 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bfcaf40-81af-4637-9a29-3c2abc343be5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:31.065596 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bfcaf40-81af-4637-9a29-3c2abc343be5 None admin] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:31.065873 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bfcaf40-81af-4637-9a29-3c2abc343be5 None admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:31.066140 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bfcaf40-81af-4637-9a29-3c2abc343be5 None admin] PATH_INFO: `/v3/projects/5258cb5ffb9c49b1ba3cfb741aef69f8` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:31.068064 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bfcaf40-81af-4637-9a29-3c2abc343be5 None admin] RBAC: Authorizing `identity:get_project(project_id=5258cb5ffb9c49b1ba3cfb741aef69f8)` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:33:31.069605 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bfcaf40-81af-4637-9a29-3c2abc343be5 None admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:33:31.071416 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 118/233] 10.222.0.13 () {56 vars in 1186 bytes} [Wed Aug 31 09:33:30 2022] GET /identity/v3/projects/5258cb5ffb9c49b1ba3cfb741aef69f8 => generated 302 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:33:31.078393 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69935ee8-d2d0-4aca-b068-ae636216767c None admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:31.203363 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69935ee8-d2d0-4aca-b068-ae636216767c None admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:31.232487 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=100 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:33:31.253875 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f814321-fb2b-4bfa-8a1b-74b36d6698c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:31.254547 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f814321-fb2b-4bfa-8a1b-74b36d6698c9 None admin] REQUEST_METHOD: `PUT` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:31.254826 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f814321-fb2b-4bfa-8a1b-74b36d6698c9 None admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:31.255098 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f814321-fb2b-4bfa-8a1b-74b36d6698c9 None admin] PATH_INFO: `/v3/projects/5258cb5ffb9c49b1ba3cfb741aef69f8/users/6dbf545ccbb14b2fb85b5e32ba1726fd/roles/5d4ee63a3add47fc9a6b8b16d39e0a1f` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:31.257011 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f814321-fb2b-4bfa-8a1b-74b36d6698c9 None admin] RBAC: Authorizing `identity:create_grant(project_id=5258cb5ffb9c49b1ba3cfb741aef69f8, user_id=6dbf545ccbb14b2fb85b5e32ba1726fd, role_id=5d4ee63a3add47fc9a6b8b16d39e0a1f)` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:33:31.258499 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f814321-fb2b-4bfa-8a1b-74b36d6698c9 None admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:33:31.275541 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 116/234] 10.222.0.13 () {58 vars in 1439 bytes} [Wed Aug 31 09:33:31 2022] PUT /identity/v3/projects/5258cb5ffb9c49b1ba3cfb741aef69f8/users/6dbf545ccbb14b2fb85b5e32ba1726fd/roles/5d4ee63a3add47fc9a6b8b16d39e0a1f => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:33:32.099414 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9c16c5-305e-4c59-82bc-8dd22a820377 None None] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:32.100262 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9c16c5-305e-4c59-82bc-8dd22a820377 None None] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:32.100691 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9c16c5-305e-4c59-82bc-8dd22a820377 None None] PATH_INFO: `/` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:32.102208 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 119/235] 10.222.0.13 () {54 vars in 927 bytes} [Wed Aug 31 09:33:32 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:33:32.107333 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8965cbbe-7694-48e8-8bbb-499be8cb8b1d None None] REQUEST_METHOD: `POST` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:32.108135 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8965cbbe-7694-48e8-8bbb-499be8cb8b1d None None] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:32.108600 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8965cbbe-7694-48e8-8bbb-499be8cb8b1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:32.132644 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.auth.core [None req-8965cbbe-7694-48e8-8bbb-499be8cb8b1d None None] MFA Rules not processed for user `fd836cf590a145df9acf19cf3ae2ac56`. Rule list: `[]` (Enabled: `True`). {{(pid=88113) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:33:32.134189 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.fernet_utils [None req-8965cbbe-7694-48e8-8bbb-499be8cb8b1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:33:32.203974 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 117/236] 10.222.0.13 () {58 vars in 1024 bytes} [Wed Aug 31 09:33:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 09:33:32.242639 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca51c624-266e-4535-9189-0cec346f4b9a None None] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:32.243246 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca51c624-266e-4535-9189-0cec346f4b9a None None] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:32.243729 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca51c624-266e-4535-9189-0cec346f4b9a None None] PATH_INFO: `/` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:32.245519 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 120/237] 10.222.0.13 () {54 vars in 927 bytes} [Wed Aug 31 09:33:32 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:33:32.251523 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd0104b-4152-44d0-be76-70f92cd23849 None None] REQUEST_METHOD: `POST` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:32.252077 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd0104b-4152-44d0-be76-70f92cd23849 None None] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:32.252544 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd0104b-4152-44d0-be76-70f92cd23849 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:32.278371 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.auth.core [None req-6cd0104b-4152-44d0-be76-70f92cd23849 None None] MFA Rules not processed for user `fd836cf590a145df9acf19cf3ae2ac56`. Rule list: `[]` (Enabled: `True`). {{(pid=88113) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:33:32.279878 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.fernet_utils [None req-6cd0104b-4152-44d0-be76-70f92cd23849 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:33:32.366545 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 118/238] 10.222.0.13 () {58 vars in 1024 bytes} [Wed Aug 31 09:33:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 09:33:32.695040 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca51c624-266e-4535-9189-0cec346f4b9a None None] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:32.713528 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd0104b-4152-44d0-be76-70f92cd23849 None None] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:32.762162 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:33:32.851608 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd0104b-4152-44d0-be76-70f92cd23849 None None] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:32.915104 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca51c624-266e-4535-9189-0cec346f4b9a None None] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:32.922026 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-199594b2-d36c-4df2-b33e-f2839b9b9036 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:32.923085 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199594b2-d36c-4df2-b33e-f2839b9b9036 None admin] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:32.923799 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199594b2-d36c-4df2-b33e-f2839b9b9036 None admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:32.924304 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199594b2-d36c-4df2-b33e-f2839b9b9036 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:32.936582 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-199594b2-d36c-4df2-b33e-f2839b9b9036 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 09:33:32.942679 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application [None req-199594b2-d36c-4df2-b33e-f2839b9b9036 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 31 09:33:32.942679 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:33:32.942679 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:33:32.942679 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:33:32.942679 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:33:32.942679 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:33:32.942679 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:33:32.942679 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:33:32.942679 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:33:32.942679 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:33:32.942679 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:33:32.942679 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:33:32.942679 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 09:33:32.942679 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 09:33:32.942679 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 09:33:32.942679 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 09:33:32.943584 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 09:33:32.943584 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 09:33:32.943584 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 09:33:32.943584 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 09:33:32.943584 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:33:32.943584 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:33:32.943584 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:33:32.943584 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:33:32.943584 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:33:32.943584 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:33:32.943584 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:33:32.943584 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application with Lock( Aug 31 09:33:32.943584 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:33:32.943584 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:33:32.943584 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:33:32.943584 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:33:32.944379 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:33:32.944379 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:33:32.944379 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:33:32.944379 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:33:32.944379 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 09:33:32.944379 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 09:33:32.944379 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 09:33:32.944379 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 09:33:32.944379 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 09:33:32.944379 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 09:33:32.944379 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 31 09:33:32.944379 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application Aug 31 09:33:32.951615 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 119/239] 10.222.0.13 () {56 vars in 1099 bytes} [Wed Aug 31 09:33:32 2022] GET /identity/v3/roles/reader => generated 84 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 09:33:32.953557 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c219a9-a6ff-4295-899d-4c7a2665a7d3 None None] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:32.954168 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c219a9-a6ff-4295-899d-4c7a2665a7d3 None None] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:32.955260 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c219a9-a6ff-4295-899d-4c7a2665a7d3 None None] PATH_INFO: `/` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:32.956639 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 120/240] 10.222.0.13 () {54 vars in 927 bytes} [Wed Aug 31 09:33:32 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:33:32.957374 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c219a9-a6ff-4295-899d-4c7a2665a7d3 None None] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:32.958694 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d2aa51-c7ea-4efd-bb43-e6f48aaeca1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:32.959640 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d2aa51-c7ea-4efd-bb43-e6f48aaeca1a None admin] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:32.960117 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d2aa51-c7ea-4efd-bb43-e6f48aaeca1a None admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:32.960597 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d2aa51-c7ea-4efd-bb43-e6f48aaeca1a None admin] PATH_INFO: `/v3/domains/default` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:32.962353 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3d2aa51-c7ea-4efd-bb43-e6f48aaeca1a None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:33:32.964633 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3d2aa51-c7ea-4efd-bb43-e6f48aaeca1a None admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:33:32.966811 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 121/241] 10.222.0.13 () {56 vars in 1108 bytes} [Wed Aug 31 09:33:32 2022] GET /identity/v3/domains/default => generated 199 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:33:32.968038 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8889d901-ea66-4713-b664-20e7306e1307 None None] REQUEST_METHOD: `POST` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:32.968320 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8889d901-ea66-4713-b664-20e7306e1307 None None] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:32.968591 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8889d901-ea66-4713-b664-20e7306e1307 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:32.981524 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=100 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:33:32.988474 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.auth.core [None req-8889d901-ea66-4713-b664-20e7306e1307 None None] MFA Rules not processed for user `fd836cf590a145df9acf19cf3ae2ac56`. Rule list: `[]` (Enabled: `True`). {{(pid=88114) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:33:32.989882 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.fernet_utils [None req-8889d901-ea66-4713-b664-20e7306e1307 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88114) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:33:33.062489 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 122/242] 10.222.0.13 () {58 vars in 1024 bytes} [Wed Aug 31 09:33:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 09:33:33.070394 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8889d901-ea66-4713-b664-20e7306e1307 None None] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:33.074812 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=100 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:33:33.082190 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c219a9-a6ff-4295-899d-4c7a2665a7d3 None None] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:33.122730 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d120bb7-3e92-4985-8d88-76dd8ad4fd84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:33.124200 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d120bb7-3e92-4985-8d88-76dd8ad4fd84 None admin] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:33.124666 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d120bb7-3e92-4985-8d88-76dd8ad4fd84 None admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:33.125140 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d120bb7-3e92-4985-8d88-76dd8ad4fd84 None admin] PATH_INFO: `/v3/roles` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:33.126267 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d120bb7-3e92-4985-8d88-76dd8ad4fd84 None admin] RBAC: Adding query filter params (name=reader) {{(pid=88113) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:33:33.126751 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d120bb7-3e92-4985-8d88-76dd8ad4fd84 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:33:33.128490 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d120bb7-3e92-4985-8d88-76dd8ad4fd84 None admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:33:33.138280 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 121/243] 10.222.0.13 () {56 vars in 1101 bytes} [Wed Aug 31 09:33:32 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:33:33.145443 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc657dec-e748-4f0e-b8e4-f70a6f1313a6 None None] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:33.145984 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc657dec-e748-4f0e-b8e4-f70a6f1313a6 None None] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:33.146439 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc657dec-e748-4f0e-b8e4-f70a6f1313a6 None None] PATH_INFO: `/` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:33.148412 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 122/244] 10.222.0.13 () {54 vars in 927 bytes} [Wed Aug 31 09:33:33 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:33:33.149573 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc657dec-e748-4f0e-b8e4-f70a6f1313a6 None None] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:33.186253 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8889d901-ea66-4713-b664-20e7306e1307 None None] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:33.226707 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab94b9ed-3747-48c9-9110-282efe37abc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:33.227323 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab94b9ed-3747-48c9-9110-282efe37abc4 None admin] REQUEST_METHOD: `POST` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:33.227686 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab94b9ed-3747-48c9-9110-282efe37abc4 None admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:33.228030 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab94b9ed-3747-48c9-9110-282efe37abc4 None admin] PATH_INFO: `/v3/users` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:33.228829 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab94b9ed-3747-48c9-9110-282efe37abc4 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:33:33.230380 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab94b9ed-3747-48c9-9110-282efe37abc4 None admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:33:33.240117 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:33:33.257456 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 123/245] 10.222.0.13 () {60 vars in 1132 bytes} [Wed Aug 31 09:33:33 2022] POST /identity/v3/users => generated 293 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 09:33:33.259035 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49019cbf-e384-447f-ad74-f648b0cb8f19 None None] REQUEST_METHOD: `POST` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:33.259386 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49019cbf-e384-447f-ad74-f648b0cb8f19 None None] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:33.259827 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49019cbf-e384-447f-ad74-f648b0cb8f19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:33.283055 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc657dec-e748-4f0e-b8e4-f70a6f1313a6 None None] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:33.284620 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.auth.core [None req-49019cbf-e384-447f-ad74-f648b0cb8f19 None None] MFA Rules not processed for user `fd836cf590a145df9acf19cf3ae2ac56`. Rule list: `[]` (Enabled: `True`). {{(pid=88114) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:33:33.286137 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.fernet_utils [None req-49019cbf-e384-447f-ad74-f648b0cb8f19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88114) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:33:33.322585 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f1ce917-c6ee-48ba-b4e9-1b2e7c592dad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:33.323693 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1ce917-c6ee-48ba-b4e9-1b2e7c592dad None admin] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:33.324174 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1ce917-c6ee-48ba-b4e9-1b2e7c592dad None admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:33.324644 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1ce917-c6ee-48ba-b4e9-1b2e7c592dad None admin] PATH_INFO: `/v3/users/08fabeee4f5e4223ac5d7c11ea5d658f` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:33.341695 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f1ce917-c6ee-48ba-b4e9-1b2e7c592dad None admin] RBAC: Authorizing `identity:get_user(user_id=08fabeee4f5e4223ac5d7c11ea5d658f)` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:33:33.343611 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f1ce917-c6ee-48ba-b4e9-1b2e7c592dad None admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:33:33.346290 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 123/246] 10.222.0.13 () {56 vars in 1177 bytes} [Wed Aug 31 09:33:33 2022] GET /identity/v3/users/08fabeee4f5e4223ac5d7c11ea5d658f => generated 297 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:33:33.347459 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f1ce917-c6ee-48ba-b4e9-1b2e7c592dad None admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:33.369979 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=100 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:33:33.371284 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 124/247] 10.222.0.13 () {58 vars in 1024 bytes} [Wed Aug 31 09:33:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 09:33:33.377426 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49019cbf-e384-447f-ad74-f648b0cb8f19 None None] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:33.528179 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f1ce917-c6ee-48ba-b4e9-1b2e7c592dad None admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:33.532010 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49019cbf-e384-447f-ad74-f648b0cb8f19 None None] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:33.568085 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:33:33.578354 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc259ae-3398-4c7b-92f4-17bd713105f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:33.579627 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc259ae-3398-4c7b-92f4-17bd713105f0 None admin] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:33.580241 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc259ae-3398-4c7b-92f4-17bd713105f0 None admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:33.580925 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc259ae-3398-4c7b-92f4-17bd713105f0 None admin] PATH_INFO: `/v3/roles/member` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:33.593707 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d0e028-25f8-47f6-85ff-a40efbf86647 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:33.594444 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d0e028-25f8-47f6-85ff-a40efbf86647 None admin] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:33.595466 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d0e028-25f8-47f6-85ff-a40efbf86647 None admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:33.595767 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d0e028-25f8-47f6-85ff-a40efbf86647 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:33.596686 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecc259ae-3398-4c7b-92f4-17bd713105f0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 09:33:33.603347 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application [None req-ecc259ae-3398-4c7b-92f4-17bd713105f0 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 31 09:33:33.603347 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:33:33.603347 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:33:33.603347 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:33:33.603347 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:33:33.603347 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:33:33.603347 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:33:33.603347 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:33:33.603347 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:33:33.603347 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:33:33.603347 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:33:33.603347 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:33:33.603347 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 09:33:33.603347 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 09:33:33.603347 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 09:33:33.603347 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 09:33:33.604274 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 09:33:33.604274 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 09:33:33.604274 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 09:33:33.604274 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 09:33:33.604274 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:33:33.604274 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:33:33.604274 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:33:33.604274 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:33:33.604274 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:33:33.604274 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:33:33.604274 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:33:33.604274 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application with Lock( Aug 31 09:33:33.604274 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:33:33.604274 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:33:33.604274 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:33:33.604274 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:33:33.605055 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:33:33.605055 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:33:33.605055 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:33:33.605055 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:33:33.605055 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 09:33:33.605055 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 09:33:33.605055 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 09:33:33.605055 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 09:33:33.605055 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 09:33:33.605055 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 09:33:33.605055 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 31 09:33:33.605055 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application Aug 31 09:33:33.608666 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 124/248] 10.222.0.13 () {56 vars in 1099 bytes} [Wed Aug 31 09:33:33 2022] GET /identity/v3/roles/member => generated 84 bytes in 261 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:33:33.615256 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc259ae-3398-4c7b-92f4-17bd713105f0 None admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:33.694466 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93d0e028-25f8-47f6-85ff-a40efbf86647 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:33:33.696043 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93d0e028-25f8-47f6-85ff-a40efbf86647 None admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:33:33.735409 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=100 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:33:33.771785 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc259ae-3398-4c7b-92f4-17bd713105f0 None admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:33.822840 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04794f86-0a72-4841-91bc-7d1d2ff27c98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:33.823439 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04794f86-0a72-4841-91bc-7d1d2ff27c98 None admin] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:33.823799 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04794f86-0a72-4841-91bc-7d1d2ff27c98 None admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:33.824041 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04794f86-0a72-4841-91bc-7d1d2ff27c98 None admin] PATH_INFO: `/v3/roles` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:33.824768 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04794f86-0a72-4841-91bc-7d1d2ff27c98 None admin] RBAC: Adding query filter params (name=member) {{(pid=88113) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:33:33.825122 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04794f86-0a72-4841-91bc-7d1d2ff27c98 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:33:33.826669 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04794f86-0a72-4841-91bc-7d1d2ff27c98 None admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:33:33.837802 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 125/249] 10.222.0.13 () {58 vars in 1282 bytes} [Wed Aug 31 09:33:33 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 466 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 31 09:33:33.841424 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 125/250] 10.222.0.13 () {56 vars in 1101 bytes} [Wed Aug 31 09:33:33 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 09:33:33.841424 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f66795a-18fc-4261-95a8-58406c0e8307 None None] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:33.841424 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f66795a-18fc-4261-95a8-58406c0e8307 None None] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:33.841424 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f66795a-18fc-4261-95a8-58406c0e8307 None None] PATH_INFO: `/` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:33.845468 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d0e028-25f8-47f6-85ff-a40efbf86647 None admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:33.848739 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 126/251] 10.222.0.13 () {54 vars in 927 bytes} [Wed Aug 31 09:33:33 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:33:33.849666 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f66795a-18fc-4261-95a8-58406c0e8307 None None] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:33.876580 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=100 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:33:33.994557 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d0e028-25f8-47f6-85ff-a40efbf86647 None admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:33.998949 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f66795a-18fc-4261-95a8-58406c0e8307 None None] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:34.040713 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c226d9a3-098a-4ef6-9007-474834548b73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:34.041627 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c226d9a3-098a-4ef6-9007-474834548b73 None admin] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:34.041913 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c226d9a3-098a-4ef6-9007-474834548b73 None admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:34.042195 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c226d9a3-098a-4ef6-9007-474834548b73 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:34.049711 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ffeb160-ff5e-46ae-8c7b-4fd598fee8bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:34.050905 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ffeb160-ff5e-46ae-8c7b-4fd598fee8bd None admin] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:34.051628 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ffeb160-ff5e-46ae-8c7b-4fd598fee8bd None admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:34.052168 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ffeb160-ff5e-46ae-8c7b-4fd598fee8bd None admin] PATH_INFO: `/v3/projects/5258cb5ffb9c49b1ba3cfb741aef69f8` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:34.054060 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ffeb160-ff5e-46ae-8c7b-4fd598fee8bd None admin] RBAC: Authorizing `identity:get_project(project_id=5258cb5ffb9c49b1ba3cfb741aef69f8)` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:33:34.055631 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c226d9a3-098a-4ef6-9007-474834548b73 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 09:33:34.056452 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ffeb160-ff5e-46ae-8c7b-4fd598fee8bd None admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:33:34.058616 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 127/252] 10.222.0.13 () {56 vars in 1186 bytes} [Wed Aug 31 09:33:33 2022] GET /identity/v3/projects/5258cb5ffb9c49b1ba3cfb741aef69f8 => generated 302 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:33:34.060157 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80443d2-9808-471d-b4c8-f558331f19dd None None] REQUEST_METHOD: `POST` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:34.060663 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80443d2-9808-471d-b4c8-f558331f19dd None None] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:34.060768 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application [None req-c226d9a3-098a-4ef6-9007-474834548b73 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 31 09:33:34.060768 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:33:34.060768 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:33:34.060768 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:33:34.060768 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:33:34.060768 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:33:34.060768 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:33:34.060768 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:33:34.060768 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:33:34.060768 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:33:34.060768 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:33:34.060768 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:33:34.060768 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 09:33:34.060768 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 09:33:34.060768 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 09:33:34.060768 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 09:33:34.061697 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 09:33:34.061697 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 09:33:34.061697 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 09:33:34.061697 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 09:33:34.061697 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:33:34.061697 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:33:34.061697 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:33:34.061697 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:33:34.061697 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:33:34.061697 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:33:34.061697 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:33:34.061697 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application with Lock( Aug 31 09:33:34.061697 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:33:34.061697 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:33:34.061697 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:33:34.061697 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:33:34.062598 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:33:34.062598 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:33:34.062598 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:33:34.062598 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:33:34.062598 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 09:33:34.062598 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 09:33:34.062598 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 09:33:34.062598 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 09:33:34.062598 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 09:33:34.062598 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 09:33:34.062598 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 31 09:33:34.062598 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application Aug 31 09:33:34.065282 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80443d2-9808-471d-b4c8-f558331f19dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:34.067812 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 126/253] 10.222.0.13 () {56 vars in 1114 bytes} [Wed Aug 31 09:33:33 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:33:34.068803 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c226d9a3-098a-4ef6-9007-474834548b73 None admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:34.093426 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.auth.core [None req-f80443d2-9808-471d-b4c8-f558331f19dd None None] MFA Rules not processed for user `fd836cf590a145df9acf19cf3ae2ac56`. Rule list: `[]` (Enabled: `True`). {{(pid=88113) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:33:34.095876 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.fernet_utils [None req-f80443d2-9808-471d-b4c8-f558331f19dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:33:34.099006 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=100 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:33:34.189598 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=100 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:33:34.190413 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 128/254] 10.222.0.13 () {58 vars in 1024 bytes} [Wed Aug 31 09:33:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 09:33:34.195955 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f80443d2-9808-471d-b4c8-f558331f19dd None None] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:34.222684 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c226d9a3-098a-4ef6-9007-474834548b73 None admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:34.290184 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d870d9e-c613-46ed-af1f-9d8ddd954683 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:34.291207 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d870d9e-c613-46ed-af1f-9d8ddd954683 None admin] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:34.291834 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d870d9e-c613-46ed-af1f-9d8ddd954683 None admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:34.292311 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d870d9e-c613-46ed-af1f-9d8ddd954683 None admin] PATH_INFO: `/v3/users/676c06c0cb4141a080531e4762087dea` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:34.293874 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d870d9e-c613-46ed-af1f-9d8ddd954683 None admin] RBAC: Authorizing `identity:get_user(user_id=676c06c0cb4141a080531e4762087dea)` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:33:34.295592 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d870d9e-c613-46ed-af1f-9d8ddd954683 None admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:33:34.297679 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 127/255] 10.222.0.13 () {56 vars in 1177 bytes} [Wed Aug 31 09:33:34 2022] GET /identity/v3/users/676c06c0cb4141a080531e4762087dea => generated 297 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:33:34.299459 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d870d9e-c613-46ed-af1f-9d8ddd954683 None admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:34.395739 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f80443d2-9808-471d-b4c8-f558331f19dd None None] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:34.421366 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d870d9e-c613-46ed-af1f-9d8ddd954683 None admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:34.458501 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-899e76ef-a10b-4c23-86b1-5562a9606593 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:34.459127 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899e76ef-a10b-4c23-86b1-5562a9606593 None admin] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:34.459403 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899e76ef-a10b-4c23-86b1-5562a9606593 None admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:34.459672 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899e76ef-a10b-4c23-86b1-5562a9606593 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:34.460466 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-899e76ef-a10b-4c23-86b1-5562a9606593 None admin] RBAC: Adding query filter params (role.id=9814fca799974e3e87ffc04473b6510e, scope.project.id=5258cb5ffb9c49b1ba3cfb741aef69f8, user.id=08fabeee4f5e4223ac5d7c11ea5d658f) {{(pid=88113) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:33:34.460806 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-899e76ef-a10b-4c23-86b1-5562a9606593 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:33:34.462287 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-899e76ef-a10b-4c23-86b1-5562a9606593 None admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:33:34.470617 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 129/256] 10.222.0.13 () {56 vars in 1374 bytes} [Wed Aug 31 09:33:34 2022] GET /identity/v3/role_assignments?user.id=08fabeee4f5e4223ac5d7c11ea5d658f&scope.project.id=5258cb5ffb9c49b1ba3cfb741aef69f8&role.id=9814fca799974e3e87ffc04473b6510e => generated 259 bytes in 277 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:33:34.470617 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-899e76ef-a10b-4c23-86b1-5562a9606593 None admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:34.493403 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=100 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:33:34.499847 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d93daa2-48cc-4f7c-98b1-a878d9747ad7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:34.500418 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d93daa2-48cc-4f7c-98b1-a878d9747ad7 None admin] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:34.500676 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d93daa2-48cc-4f7c-98b1-a878d9747ad7 None admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:34.501480 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d93daa2-48cc-4f7c-98b1-a878d9747ad7 None admin] PATH_INFO: `/v3/roles` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:34.503886 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d93daa2-48cc-4f7c-98b1-a878d9747ad7 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=88114) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:33:34.503886 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d93daa2-48cc-4f7c-98b1-a878d9747ad7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:33:34.504206 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d93daa2-48cc-4f7c-98b1-a878d9747ad7 None admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:33:34.533764 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 128/257] 10.222.0.13 () {56 vars in 1111 bytes} [Wed Aug 31 09:33:34 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:33:34.540056 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d93daa2-48cc-4f7c-98b1-a878d9747ad7 None admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:34.567844 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=100 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:33:34.669205 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-899e76ef-a10b-4c23-86b1-5562a9606593 None admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:34.718783 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d93daa2-48cc-4f7c-98b1-a878d9747ad7 None admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:34.759830 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df55659c-3426-4344-a625-8a44d0ec15e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:34.760457 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df55659c-3426-4344-a625-8a44d0ec15e5 None admin] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:34.760739 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df55659c-3426-4344-a625-8a44d0ec15e5 None admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:34.761007 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df55659c-3426-4344-a625-8a44d0ec15e5 None admin] PATH_INFO: `/v3/users/6dbf545ccbb14b2fb85b5e32ba1726fd` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:34.762315 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df55659c-3426-4344-a625-8a44d0ec15e5 None admin] RBAC: Authorizing `identity:get_user(user_id=6dbf545ccbb14b2fb85b5e32ba1726fd)` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:33:34.763844 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df55659c-3426-4344-a625-8a44d0ec15e5 None admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:33:34.765681 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 129/258] 10.222.0.13 () {56 vars in 1177 bytes} [Wed Aug 31 09:33:34 2022] GET /identity/v3/users/6dbf545ccbb14b2fb85b5e32ba1726fd => generated 283 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:33:34.766453 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df55659c-3426-4344-a625-8a44d0ec15e5 None admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:34.792512 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b7bf43-d1f5-4128-8d1a-0e390d4168fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:34.795971 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b7bf43-d1f5-4128-8d1a-0e390d4168fc None admin] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:34.796280 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b7bf43-d1f5-4128-8d1a-0e390d4168fc None admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:34.796549 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b7bf43-d1f5-4128-8d1a-0e390d4168fc None admin] PATH_INFO: `/v3/projects/5258cb5ffb9c49b1ba3cfb741aef69f8` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:34.797856 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07b7bf43-d1f5-4128-8d1a-0e390d4168fc None admin] RBAC: Authorizing `identity:get_project(project_id=5258cb5ffb9c49b1ba3cfb741aef69f8)` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:33:34.799541 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07b7bf43-d1f5-4128-8d1a-0e390d4168fc None admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:33:34.801417 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 130/259] 10.222.0.13 () {56 vars in 1186 bytes} [Wed Aug 31 09:33:34 2022] GET /identity/v3/projects/5258cb5ffb9c49b1ba3cfb741aef69f8 => generated 302 bytes in 333 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:33:34.802207 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b7bf43-d1f5-4128-8d1a-0e390d4168fc None admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:34.879965 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=100 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:33:34.917118 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b7bf43-d1f5-4128-8d1a-0e390d4168fc None admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:34.953666 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef05287c-79f1-467a-8e2d-1088806957fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:34.954834 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef05287c-79f1-467a-8e2d-1088806957fc None admin] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:34.955159 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef05287c-79f1-467a-8e2d-1088806957fc None admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:34.955395 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef05287c-79f1-467a-8e2d-1088806957fc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:35.003698 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=100 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:33:35.030042 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef05287c-79f1-467a-8e2d-1088806957fc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:33:35.031524 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef05287c-79f1-467a-8e2d-1088806957fc None admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:33:35.091362 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df55659c-3426-4344-a625-8a44d0ec15e5 None admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:35.109817 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=100 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:33:35.160881 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-417af195-cbdd-4b42-985f-e3bc4ae8b4e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:35.161557 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417af195-cbdd-4b42-985f-e3bc4ae8b4e1 None admin] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:35.161835 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417af195-cbdd-4b42-985f-e3bc4ae8b4e1 None admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:35.162107 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417af195-cbdd-4b42-985f-e3bc4ae8b4e1 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:35.166318 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 131/260] 10.222.0.13 () {58 vars in 1282 bytes} [Wed Aug 31 09:33:34 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 363 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 31 09:33:35.167165 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef05287c-79f1-467a-8e2d-1088806957fc None admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:35.188214 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-417af195-cbdd-4b42-985f-e3bc4ae8b4e1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 09:33:35.193954 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application [None req-417af195-cbdd-4b42-985f-e3bc4ae8b4e1 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 09:33:35.193954 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:33:35.193954 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:33:35.193954 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:33:35.193954 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:33:35.193954 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:33:35.193954 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:33:35.193954 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:33:35.193954 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:33:35.193954 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:33:35.193954 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:33:35.193954 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:33:35.193954 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 09:33:35.193954 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 09:33:35.193954 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 09:33:35.193954 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 09:33:35.195343 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 09:33:35.195343 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 09:33:35.195343 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 09:33:35.195343 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 09:33:35.195343 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:33:35.195343 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:33:35.195343 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:33:35.195343 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:33:35.195343 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:33:35.195343 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:33:35.195343 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:33:35.195343 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application with Lock( Aug 31 09:33:35.195343 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:33:35.195343 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:33:35.195343 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:33:35.195343 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:33:35.196131 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:33:35.196131 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:33:35.196131 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:33:35.196131 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:33:35.196131 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 09:33:35.196131 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 09:33:35.196131 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 09:33:35.196131 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 09:33:35.196131 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 09:33:35.196131 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 09:33:35.196131 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 09:33:35.196131 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application Aug 31 09:33:35.203955 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 130/261] 10.222.0.13 () {56 vars in 1096 bytes} [Wed Aug 31 09:33:34 2022] GET /identity/v3/roles/admin => generated 83 bytes in 437 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:33:35.205006 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-417af195-cbdd-4b42-985f-e3bc4ae8b4e1 None admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:35.316557 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef05287c-79f1-467a-8e2d-1088806957fc None admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:35.353474 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-210de538-7199-49b0-ac4c-4750260efe2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:35.354556 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-210de538-7199-49b0-ac4c-4750260efe2f None admin] REQUEST_METHOD: `PUT` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:35.355066 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-210de538-7199-49b0-ac4c-4750260efe2f None admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:35.355539 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-210de538-7199-49b0-ac4c-4750260efe2f None admin] PATH_INFO: `/v3/projects/5258cb5ffb9c49b1ba3cfb741aef69f8/users/676c06c0cb4141a080531e4762087dea/roles/6c99f5d9f1f243eaa134d8540f21095d` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:35.357907 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-210de538-7199-49b0-ac4c-4750260efe2f None admin] RBAC: Authorizing `identity:create_grant(project_id=5258cb5ffb9c49b1ba3cfb741aef69f8, user_id=676c06c0cb4141a080531e4762087dea, role_id=6c99f5d9f1f243eaa134d8540f21095d)` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:33:35.359704 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-210de538-7199-49b0-ac4c-4750260efe2f None admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:33:35.375372 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 132/262] 10.222.0.13 () {58 vars in 1439 bytes} [Wed Aug 31 09:33:35 2022] PUT /identity/v3/projects/5258cb5ffb9c49b1ba3cfb741aef69f8/users/676c06c0cb4141a080531e4762087dea/roles/6c99f5d9f1f243eaa134d8540f21095d => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:33:35.377130 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-210de538-7199-49b0-ac4c-4750260efe2f None admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:35.382156 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-417af195-cbdd-4b42-985f-e3bc4ae8b4e1 None admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:35.414312 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:33:35.447170 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac85e3b-f386-4b90-9734-a6765315f8ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:35.447827 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac85e3b-f386-4b90-9734-a6765315f8ce None admin] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:35.448103 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac85e3b-f386-4b90-9734-a6765315f8ce None admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:35.448371 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac85e3b-f386-4b90-9734-a6765315f8ce None admin] PATH_INFO: `/v3/projects/5258cb5ffb9c49b1ba3cfb741aef69f8` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:35.450246 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ac85e3b-f386-4b90-9734-a6765315f8ce None admin] RBAC: Authorizing `identity:get_project(project_id=5258cb5ffb9c49b1ba3cfb741aef69f8)` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:33:35.451688 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ac85e3b-f386-4b90-9734-a6765315f8ce None admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:33:35.453332 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 131/263] 10.222.0.13 () {56 vars in 1186 bytes} [Wed Aug 31 09:33:35 2022] GET /identity/v3/projects/5258cb5ffb9c49b1ba3cfb741aef69f8 => generated 302 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:33:35.459370 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac85e3b-f386-4b90-9734-a6765315f8ce None admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:35.490420 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=100 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:33:35.508727 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-210de538-7199-49b0-ac4c-4750260efe2f None admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:35.581892 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf911b40-1d01-441e-8362-07cda291356f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:35.582526 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf911b40-1d01-441e-8362-07cda291356f None admin] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:35.582808 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf911b40-1d01-441e-8362-07cda291356f None admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:35.583082 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf911b40-1d01-441e-8362-07cda291356f None admin] PATH_INFO: `/v3/roles` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:35.583766 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf911b40-1d01-441e-8362-07cda291356f None admin] RBAC: Adding query filter params (name=admin) {{(pid=88113) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:33:35.584069 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf911b40-1d01-441e-8362-07cda291356f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:33:35.585487 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf911b40-1d01-441e-8362-07cda291356f None admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:33:35.594802 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 133/264] 10.222.0.13 () {56 vars in 1099 bytes} [Wed Aug 31 09:33:35 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:33:35.605754 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf911b40-1d01-441e-8362-07cda291356f None admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:35.644685 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac85e3b-f386-4b90-9734-a6765315f8ce None admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:35.690278 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9dfc937-e235-4163-8bb5-b537ea76c2c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:35.692433 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9dfc937-e235-4163-8bb5-b537ea76c2c0 None admin] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:35.694072 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9dfc937-e235-4163-8bb5-b537ea76c2c0 None admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:35.694555 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9dfc937-e235-4163-8bb5-b537ea76c2c0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:35.695503 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9dfc937-e235-4163-8bb5-b537ea76c2c0 None admin] RBAC: Adding query filter params (role.id=889857b056624d47b572d4a80ecc3f8e, scope.project.id=5258cb5ffb9c49b1ba3cfb741aef69f8, user.id=6dbf545ccbb14b2fb85b5e32ba1726fd) {{(pid=88114) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:33:35.697243 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9dfc937-e235-4163-8bb5-b537ea76c2c0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:33:35.700136 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9dfc937-e235-4163-8bb5-b537ea76c2c0 None admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:33:35.715892 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 132/265] 10.222.0.13 () {56 vars in 1374 bytes} [Wed Aug 31 09:33:35 2022] GET /identity/v3/role_assignments?user.id=6dbf545ccbb14b2fb85b5e32ba1726fd&scope.project.id=5258cb5ffb9c49b1ba3cfb741aef69f8&role.id=889857b056624d47b572d4a80ecc3f8e => generated 608 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 09:33:35.777600 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=100 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:33:35.788489 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf911b40-1d01-441e-8362-07cda291356f None admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:35.833541 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-523f6e57-1601-4616-a82b-c04006c6c05f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:35.834669 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523f6e57-1601-4616-a82b-c04006c6c05f None admin] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:35.835183 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523f6e57-1601-4616-a82b-c04006c6c05f None admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:35.835728 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523f6e57-1601-4616-a82b-c04006c6c05f None admin] PATH_INFO: `/v3/users/6dbf545ccbb14b2fb85b5e32ba1726fd` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:35.838233 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-523f6e57-1601-4616-a82b-c04006c6c05f None admin] RBAC: Authorizing `identity:get_user(user_id=6dbf545ccbb14b2fb85b5e32ba1726fd)` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:33:35.840046 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-523f6e57-1601-4616-a82b-c04006c6c05f None admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:33:35.842461 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 134/266] 10.222.0.13 () {56 vars in 1177 bytes} [Wed Aug 31 09:33:35 2022] GET /identity/v3/users/6dbf545ccbb14b2fb85b5e32ba1726fd => generated 283 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:33:35.848483 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9dfc937-e235-4163-8bb5-b537ea76c2c0 None admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:35.908485 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5b8060-4dd2-4dfa-8ce0-5655344cfadb None None] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:35.909161 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5b8060-4dd2-4dfa-8ce0-5655344cfadb None None] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:35.909683 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5b8060-4dd2-4dfa-8ce0-5655344cfadb None None] PATH_INFO: `/` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:35.911790 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 135/267] 10.222.0.13 () {54 vars in 927 bytes} [Wed Aug 31 09:33:35 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:33:35.916939 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb286ea9-4df7-42a2-9af4-ed7f277fa7f2 None None] REQUEST_METHOD: `POST` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:35.917569 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb286ea9-4df7-42a2-9af4-ed7f277fa7f2 None None] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:35.918044 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb286ea9-4df7-42a2-9af4-ed7f277fa7f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:35.956913 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.auth.core [None req-fb286ea9-4df7-42a2-9af4-ed7f277fa7f2 None None] MFA Rules not processed for user `fd836cf590a145df9acf19cf3ae2ac56`. Rule list: `[]` (Enabled: `True`). {{(pid=88113) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:33:35.960976 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.fernet_utils [None req-fb286ea9-4df7-42a2-9af4-ed7f277fa7f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:33:35.984244 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=100 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:33:36.019381 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9dfc937-e235-4163-8bb5-b537ea76c2c0 None admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:36.061040 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7dd709d-1834-43f7-8ba6-f98d159c5ced None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:36.061040 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7dd709d-1834-43f7-8ba6-f98d159c5ced None admin] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:36.061040 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7dd709d-1834-43f7-8ba6-f98d159c5ced None admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:36.061040 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7dd709d-1834-43f7-8ba6-f98d159c5ced None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:36.081022 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 136/268] 10.222.0.13 () {58 vars in 1024 bytes} [Wed Aug 31 09:33:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 31 09:33:36.185039 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7dd709d-1834-43f7-8ba6-f98d159c5ced None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:33:36.186541 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7dd709d-1834-43f7-8ba6-f98d159c5ced None admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:33:36.221668 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b0fc6c-ccd1-445e-9099-b18bcfbbdcbc None None] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:36.222361 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b0fc6c-ccd1-445e-9099-b18bcfbbdcbc None None] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:36.223489 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b0fc6c-ccd1-445e-9099-b18bcfbbdcbc None None] PATH_INFO: `/` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:36.228589 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 137/269] 10.222.0.13 () {54 vars in 927 bytes} [Wed Aug 31 09:33:36 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:33:36.232105 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf971e06-688c-4e29-841f-f55a248704b6 None None] REQUEST_METHOD: `POST` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:36.232731 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf971e06-688c-4e29-841f-f55a248704b6 None None] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:36.233279 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf971e06-688c-4e29-841f-f55a248704b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:36.267752 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.auth.core [None req-cf971e06-688c-4e29-841f-f55a248704b6 None None] MFA Rules not processed for user `fd836cf590a145df9acf19cf3ae2ac56`. Rule list: `[]` (Enabled: `True`). {{(pid=88113) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:33:36.269074 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.fernet_utils [None req-cf971e06-688c-4e29-841f-f55a248704b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:33:36.278499 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=100 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:33:36.316699 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 133/270] 10.222.0.13 () {58 vars in 1282 bytes} [Wed Aug 31 09:33:35 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 469 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 31 09:33:36.321516 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7dd709d-1834-43f7-8ba6-f98d159c5ced None admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:36.403921 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 138/271] 10.222.0.13 () {58 vars in 1024 bytes} [Wed Aug 31 09:33:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 31 09:33:36.475161 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7dd709d-1834-43f7-8ba6-f98d159c5ced None admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:36.523530 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c108ea-8591-49f0-aa57-20c344bbac24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:36.524103 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c108ea-8591-49f0-aa57-20c344bbac24 None admin] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:36.524378 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c108ea-8591-49f0-aa57-20c344bbac24 None admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:36.524647 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c108ea-8591-49f0-aa57-20c344bbac24 None admin] PATH_INFO: `/v3/projects/5258cb5ffb9c49b1ba3cfb741aef69f8` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:36.526111 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5c108ea-8591-49f0-aa57-20c344bbac24 None admin] RBAC: Authorizing `identity:get_project(project_id=5258cb5ffb9c49b1ba3cfb741aef69f8)` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:33:36.527575 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5c108ea-8591-49f0-aa57-20c344bbac24 None admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:33:36.529216 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 134/272] 10.222.0.13 () {56 vars in 1186 bytes} [Wed Aug 31 09:33:36 2022] GET /identity/v3/projects/5258cb5ffb9c49b1ba3cfb741aef69f8 => generated 302 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:33:36.535621 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf971e06-688c-4e29-841f-f55a248704b6 None None] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:36.566650 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=100 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:33:36.604927 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c108ea-8591-49f0-aa57-20c344bbac24 None admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:36.686719 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=100 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:33:36.729244 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf971e06-688c-4e29-841f-f55a248704b6 None None] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:36.789038 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38148c33-8b76-4cde-b9e8-2752de5308f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:36.789609 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38148c33-8b76-4cde-b9e8-2752de5308f1 None admin] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:36.789816 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38148c33-8b76-4cde-b9e8-2752de5308f1 None admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:36.790090 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38148c33-8b76-4cde-b9e8-2752de5308f1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:36.790819 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38148c33-8b76-4cde-b9e8-2752de5308f1 None admin] RBAC: Adding query filter params (role.id=5d4ee63a3add47fc9a6b8b16d39e0a1f, scope.project.id=5258cb5ffb9c49b1ba3cfb741aef69f8, user.id=6dbf545ccbb14b2fb85b5e32ba1726fd) {{(pid=88113) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:33:36.791123 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38148c33-8b76-4cde-b9e8-2752de5308f1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:33:36.792774 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38148c33-8b76-4cde-b9e8-2752de5308f1 None admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:33:36.800202 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 139/273] 10.222.0.13 () {56 vars in 1374 bytes} [Wed Aug 31 09:33:36 2022] GET /identity/v3/role_assignments?user.id=6dbf545ccbb14b2fb85b5e32ba1726fd&scope.project.id=5258cb5ffb9c49b1ba3cfb741aef69f8&role.id=5d4ee63a3add47fc9a6b8b16d39e0a1f => generated 608 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:33:36.800829 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c108ea-8591-49f0-aa57-20c344bbac24 None admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:36.801131 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38148c33-8b76-4cde-b9e8-2752de5308f1 None admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:36.862265 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8eeb058-06c3-46fd-a38e-0cb0ad992002 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:36.863349 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8eeb058-06c3-46fd-a38e-0cb0ad992002 None admin] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:36.863853 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8eeb058-06c3-46fd-a38e-0cb0ad992002 None admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:36.864328 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8eeb058-06c3-46fd-a38e-0cb0ad992002 None admin] PATH_INFO: `/v3/domains/default` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:36.865963 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8eeb058-06c3-46fd-a38e-0cb0ad992002 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:33:36.868476 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8eeb058-06c3-46fd-a38e-0cb0ad992002 None admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:33:36.871056 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 135/274] 10.222.0.13 () {56 vars in 1108 bytes} [Wed Aug 31 09:33:36 2022] GET /identity/v3/domains/default => generated 199 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 09:33:36.877589 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8eeb058-06c3-46fd-a38e-0cb0ad992002 None admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:36.911640 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=100 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:33:36.932883 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38148c33-8b76-4cde-b9e8-2752de5308f1 None admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:36.973331 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4eb5fcb-4585-4a81-826b-ccc5b873e942 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:36.974032 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4eb5fcb-4585-4a81-826b-ccc5b873e942 None admin] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:36.974321 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4eb5fcb-4585-4a81-826b-ccc5b873e942 None admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:36.974593 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4eb5fcb-4585-4a81-826b-ccc5b873e942 None admin] PATH_INFO: `/v3/roles/member` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:36.991601 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8eeb058-06c3-46fd-a38e-0cb0ad992002 None admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:36.999840 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4eb5fcb-4585-4a81-826b-ccc5b873e942 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 09:33:37.009914 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=100 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:33:37.013841 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application [None req-b4eb5fcb-4585-4a81-826b-ccc5b873e942 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 31 09:33:37.013841 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:33:37.013841 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:33:37.013841 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:33:37.013841 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:33:37.013841 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:33:37.013841 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:33:37.013841 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:33:37.013841 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:33:37.013841 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:33:37.013841 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:33:37.013841 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:33:37.013841 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 09:33:37.013841 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 09:33:37.013841 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 09:33:37.013841 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 09:33:37.014716 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 09:33:37.014716 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 09:33:37.014716 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 09:33:37.014716 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 09:33:37.014716 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:33:37.014716 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:33:37.014716 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:33:37.014716 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:33:37.014716 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:33:37.014716 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:33:37.014716 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:33:37.014716 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application with Lock( Aug 31 09:33:37.014716 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:33:37.014716 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:33:37.014716 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:33:37.014716 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:33:37.015507 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:33:37.015507 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:33:37.015507 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:33:37.015507 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:33:37.015507 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 09:33:37.015507 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 09:33:37.015507 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 09:33:37.015507 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 09:33:37.015507 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 09:33:37.015507 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 09:33:37.015507 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 31 09:33:37.015507 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application Aug 31 09:33:37.017045 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 140/275] 10.222.0.13 () {56 vars in 1099 bytes} [Wed Aug 31 09:33:36 2022] GET /identity/v3/roles/member => generated 84 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:33:37.025654 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4eb5fcb-4585-4a81-826b-ccc5b873e942 None admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:37.042409 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad6df678-fe04-4733-90f1-e1c458d57bab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:37.043196 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6df678-fe04-4733-90f1-e1c458d57bab None admin] REQUEST_METHOD: `POST` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:37.043578 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6df678-fe04-4733-90f1-e1c458d57bab None admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:37.043940 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6df678-fe04-4733-90f1-e1c458d57bab None admin] PATH_INFO: `/v3/users` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:37.044700 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad6df678-fe04-4733-90f1-e1c458d57bab None admin] RBAC: Authorizing `identity:create_user()` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:33:37.046278 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad6df678-fe04-4733-90f1-e1c458d57bab None admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:33:37.075608 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 136/276] 10.222.0.13 () {60 vars in 1132 bytes} [Wed Aug 31 09:33:36 2022] POST /identity/v3/users => generated 293 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 09:33:37.147294 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f937167-6549-43df-a4e1-0649b7652d15 None None] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:37.147578 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f937167-6549-43df-a4e1-0649b7652d15 None None] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:37.147866 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f937167-6549-43df-a4e1-0649b7652d15 None None] PATH_INFO: `/` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:37.149373 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 137/277] 10.222.0.13 () {54 vars in 927 bytes} [Wed Aug 31 09:33:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 31 09:33:37.156000 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe36086-90a1-4125-9f01-ed56888d8cf8 None None] REQUEST_METHOD: `POST` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:37.156283 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe36086-90a1-4125-9f01-ed56888d8cf8 None None] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:37.156546 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe36086-90a1-4125-9f01-ed56888d8cf8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:37.168882 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4eb5fcb-4585-4a81-826b-ccc5b873e942 None admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:37.182905 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.auth.core [None req-fbe36086-90a1-4125-9f01-ed56888d8cf8 None None] MFA Rules not processed for user `fd836cf590a145df9acf19cf3ae2ac56`. Rule list: `[]` (Enabled: `True`). {{(pid=88114) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:33:37.185448 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.fernet_utils [None req-fbe36086-90a1-4125-9f01-ed56888d8cf8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88114) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:33:37.211899 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad14b78-8c93-41b5-acac-d9b066de79d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:37.212638 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad14b78-8c93-41b5-acac-d9b066de79d0 None admin] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:37.212915 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad14b78-8c93-41b5-acac-d9b066de79d0 None admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:37.213187 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad14b78-8c93-41b5-acac-d9b066de79d0 None admin] PATH_INFO: `/v3/roles` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:37.213917 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ad14b78-8c93-41b5-acac-d9b066de79d0 None admin] RBAC: Adding query filter params (name=member) {{(pid=88113) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:33:37.214258 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ad14b78-8c93-41b5-acac-d9b066de79d0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:33:37.215859 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ad14b78-8c93-41b5-acac-d9b066de79d0 None admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:33:37.227481 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 141/278] 10.222.0.13 () {56 vars in 1101 bytes} [Wed Aug 31 09:33:37 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 09:33:37.234727 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad14b78-8c93-41b5-acac-d9b066de79d0 None admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:37.255538 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=100 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:33:37.310359 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 138/279] 10.222.0.13 () {58 vars in 1024 bytes} [Wed Aug 31 09:33:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 31 09:33:37.374847 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad14b78-8c93-41b5-acac-d9b066de79d0 None admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:37.412617 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-831a8fbe-21ea-4b5f-90c2-54954369c629 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:37.413937 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831a8fbe-21ea-4b5f-90c2-54954369c629 None admin] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:37.414374 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831a8fbe-21ea-4b5f-90c2-54954369c629 None admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:37.414781 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831a8fbe-21ea-4b5f-90c2-54954369c629 None admin] PATH_INFO: `/v3/users/22d585550661496e956a728556c3d90d` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:37.430844 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-831a8fbe-21ea-4b5f-90c2-54954369c629 None admin] RBAC: Authorizing `identity:get_user(user_id=22d585550661496e956a728556c3d90d)` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:33:37.432521 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-831a8fbe-21ea-4b5f-90c2-54954369c629 None admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:33:37.434713 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 142/280] 10.222.0.13 () {56 vars in 1177 bytes} [Wed Aug 31 09:33:37 2022] GET /identity/v3/users/22d585550661496e956a728556c3d90d => generated 293 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:33:37.441797 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe36086-90a1-4125-9f01-ed56888d8cf8 None None] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:37.554788 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:33:37.563262 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe36086-90a1-4125-9f01-ed56888d8cf8 None None] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:37.584933 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-831a8fbe-21ea-4b5f-90c2-54954369c629 None admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:37.609935 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ece175-9341-476d-ace5-81478665286a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:37.611309 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ece175-9341-476d-ace5-81478665286a None admin] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:37.611592 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ece175-9341-476d-ace5-81478665286a None admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:37.611864 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ece175-9341-476d-ace5-81478665286a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:37.612603 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45ece175-9341-476d-ace5-81478665286a None admin] RBAC: Adding query filter params (role.id=6c99f5d9f1f243eaa134d8540f21095d, user.id=22d585550661496e956a728556c3d90d, scope.system=all) {{(pid=88114) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:33:37.612914 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45ece175-9341-476d-ace5-81478665286a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:33:37.614395 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45ece175-9341-476d-ace5-81478665286a None admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:33:37.627582 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 139/281] 10.222.0.13 () {56 vars in 1308 bytes} [Wed Aug 31 09:33:37 2022] GET /identity/v3/role_assignments?user.id=22d585550661496e956a728556c3d90d&scope.system=all&role.id=6c99f5d9f1f243eaa134d8540f21095d => generated 226 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:33:37.700292 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=100 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:33:37.757678 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-831a8fbe-21ea-4b5f-90c2-54954369c629 None admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:37.810181 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-308bbc88-fb4f-4b46-82c8-61766f854170 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:37.810844 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308bbc88-fb4f-4b46-82c8-61766f854170 None admin] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:37.811117 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308bbc88-fb4f-4b46-82c8-61766f854170 None admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:37.811381 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308bbc88-fb4f-4b46-82c8-61766f854170 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:37.832139 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-308bbc88-fb4f-4b46-82c8-61766f854170 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 09:33:37.837215 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application [None req-308bbc88-fb4f-4b46-82c8-61766f854170 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 31 09:33:37.837215 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:33:37.837215 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:33:37.837215 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:33:37.837215 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:33:37.837215 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:33:37.837215 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:33:37.837215 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:33:37.837215 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:33:37.837215 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:33:37.837215 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:33:37.837215 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:33:37.837215 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 09:33:37.837215 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 09:33:37.837215 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 09:33:37.837215 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 09:33:37.838147 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 09:33:37.838147 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 09:33:37.838147 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 09:33:37.838147 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 09:33:37.838147 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:33:37.838147 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:33:37.838147 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:33:37.838147 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:33:37.838147 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:33:37.838147 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:33:37.838147 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:33:37.838147 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application with Lock( Aug 31 09:33:37.838147 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:33:37.838147 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:33:37.838147 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:33:37.838147 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:33:37.838927 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:33:37.838927 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:33:37.838927 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:33:37.838927 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:33:37.838927 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 09:33:37.838927 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 09:33:37.838927 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 09:33:37.838927 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 09:33:37.838927 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 09:33:37.838927 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 09:33:37.838927 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 31 09:33:37.838927 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application Aug 31 09:33:37.838927 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 143/282] 10.222.0.13 () {56 vars in 1099 bytes} [Wed Aug 31 09:33:37 2022] GET /identity/v3/roles/reader => generated 84 bytes in 254 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:33:37.846772 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ece175-9341-476d-ace5-81478665286a None admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:37.875217 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df41d5fa-2c42-4bc0-8134-9af69bf85f78 None None] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:37.875540 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df41d5fa-2c42-4bc0-8134-9af69bf85f78 None None] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:37.876852 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df41d5fa-2c42-4bc0-8134-9af69bf85f78 None None] PATH_INFO: `/` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:37.879213 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 144/283] 10.222.0.13 () {54 vars in 927 bytes} [Wed Aug 31 09:33:37 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 31 09:33:37.886613 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509e02e9-9606-42df-921a-584b43edd7d1 None None] REQUEST_METHOD: `POST` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:37.886892 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509e02e9-9606-42df-921a-584b43edd7d1 None None] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:37.887161 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509e02e9-9606-42df-921a-584b43edd7d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:37.920910 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.auth.core [None req-509e02e9-9606-42df-921a-584b43edd7d1 None None] MFA Rules not processed for user `fd836cf590a145df9acf19cf3ae2ac56`. Rule list: `[]` (Enabled: `True`). {{(pid=88113) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:33:37.922123 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.fernet_utils [None req-509e02e9-9606-42df-921a-584b43edd7d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:33:38.039858 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 145/284] 10.222.0.13 () {58 vars in 1024 bytes} [Wed Aug 31 09:33:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 09:33:38.057608 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ece175-9341-476d-ace5-81478665286a None admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:38.104656 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e38bb0ee-136d-404e-8718-e251b0589d62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:38.105197 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38bb0ee-136d-404e-8718-e251b0589d62 None admin] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:38.105499 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38bb0ee-136d-404e-8718-e251b0589d62 None admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:38.105776 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38bb0ee-136d-404e-8718-e251b0589d62 None admin] PATH_INFO: `/v3/roles` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:38.106451 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e38bb0ee-136d-404e-8718-e251b0589d62 None admin] RBAC: Adding query filter params (name=reader) {{(pid=88114) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:33:38.106757 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e38bb0ee-136d-404e-8718-e251b0589d62 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:33:38.108207 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e38bb0ee-136d-404e-8718-e251b0589d62 None admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:33:38.120104 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 140/285] 10.222.0.13 () {56 vars in 1101 bytes} [Wed Aug 31 09:33:37 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 274 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:33:38.125574 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-509e02e9-9606-42df-921a-584b43edd7d1 None None] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:38.198944 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=100 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:33:38.265481 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-509e02e9-9606-42df-921a-584b43edd7d1 None None] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:38.311925 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da50bd6e-2bf0-4f06-9893-3742f9633c60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:38.312945 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da50bd6e-2bf0-4f06-9893-3742f9633c60 None admin] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:38.313993 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da50bd6e-2bf0-4f06-9893-3742f9633c60 None admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:38.314487 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da50bd6e-2bf0-4f06-9893-3742f9633c60 None admin] PATH_INFO: `/v3/users/08fabeee4f5e4223ac5d7c11ea5d658f` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:38.315991 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da50bd6e-2bf0-4f06-9893-3742f9633c60 None admin] RBAC: Authorizing `identity:get_user(user_id=08fabeee4f5e4223ac5d7c11ea5d658f)` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:33:38.317668 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da50bd6e-2bf0-4f06-9893-3742f9633c60 None admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:33:38.319782 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 146/286] 10.222.0.13 () {56 vars in 1177 bytes} [Wed Aug 31 09:33:38 2022] GET /identity/v3/users/08fabeee4f5e4223ac5d7c11ea5d658f => generated 297 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:33:38.325071 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e38bb0ee-136d-404e-8718-e251b0589d62 None admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:38.362684 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=100 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:33:38.414895 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da50bd6e-2bf0-4f06-9893-3742f9633c60 None admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:38.458696 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e38bb0ee-136d-404e-8718-e251b0589d62 None admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:38.519355 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37fdc5e9-0107-437d-b9da-a569d6c82381 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:38.520260 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37fdc5e9-0107-437d-b9da-a569d6c82381 None admin] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:38.520535 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37fdc5e9-0107-437d-b9da-a569d6c82381 None admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:38.520798 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37fdc5e9-0107-437d-b9da-a569d6c82381 None admin] PATH_INFO: `/v3/projects/5258cb5ffb9c49b1ba3cfb741aef69f8` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:38.521976 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37fdc5e9-0107-437d-b9da-a569d6c82381 None admin] RBAC: Authorizing `identity:get_project(project_id=5258cb5ffb9c49b1ba3cfb741aef69f8)` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:33:38.523403 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37fdc5e9-0107-437d-b9da-a569d6c82381 None admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:33:38.528189 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 141/287] 10.222.0.13 () {56 vars in 1186 bytes} [Wed Aug 31 09:33:38 2022] GET /identity/v3/projects/5258cb5ffb9c49b1ba3cfb741aef69f8 => generated 302 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:33:38.532049 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37fdc5e9-0107-437d-b9da-a569d6c82381 None admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:38.605971 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da50bd6e-2bf0-4f06-9893-3742f9633c60 None admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:38.646904 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=100 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:33:38.668792 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6349e00f-2210-4964-8075-d513ddd93ebd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:38.669462 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6349e00f-2210-4964-8075-d513ddd93ebd None admin] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:38.669738 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6349e00f-2210-4964-8075-d513ddd93ebd None admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:38.670934 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6349e00f-2210-4964-8075-d513ddd93ebd None admin] PATH_INFO: `/v3/roles/member` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:38.694604 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6349e00f-2210-4964-8075-d513ddd93ebd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 09:33:38.700074 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application [None req-6349e00f-2210-4964-8075-d513ddd93ebd None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 31 09:33:38.700074 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:33:38.700074 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:33:38.700074 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:33:38.700074 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:33:38.700074 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:33:38.700074 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:33:38.700074 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:33:38.700074 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:33:38.700074 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:33:38.700074 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:33:38.700074 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:33:38.700074 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 09:33:38.700074 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 09:33:38.700074 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 09:33:38.700074 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 09:33:38.700999 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 09:33:38.700999 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 09:33:38.700999 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 09:33:38.700999 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 09:33:38.700999 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:33:38.700999 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:33:38.700999 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:33:38.700999 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:33:38.700999 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:33:38.700999 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:33:38.700999 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:33:38.700999 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application with Lock( Aug 31 09:33:38.700999 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:33:38.700999 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:33:38.700999 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:33:38.700999 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:33:38.701879 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:33:38.701879 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:33:38.701879 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:33:38.701879 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:33:38.701879 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 09:33:38.701879 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 09:33:38.701879 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 09:33:38.701879 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 09:33:38.701879 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 09:33:38.701879 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 09:33:38.701879 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 31 09:33:38.701879 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application Aug 31 09:33:38.701879 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 147/288] 10.222.0.13 () {56 vars in 1099 bytes} [Wed Aug 31 09:33:38 2022] GET /identity/v3/roles/member => generated 84 bytes in 287 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 09:33:38.710426 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6349e00f-2210-4964-8075-d513ddd93ebd None admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:38.712760 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=100 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:33:38.731784 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37fdc5e9-0107-437d-b9da-a569d6c82381 None admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:38.804197 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b83bfc27-502b-40b8-8862-50b27e5b54f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:38.804897 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83bfc27-502b-40b8-8862-50b27e5b54f7 None admin] REQUEST_METHOD: `PUT` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:38.805179 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83bfc27-502b-40b8-8862-50b27e5b54f7 None admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:38.805471 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83bfc27-502b-40b8-8862-50b27e5b54f7 None admin] PATH_INFO: `/v3/projects/5258cb5ffb9c49b1ba3cfb741aef69f8/users/08fabeee4f5e4223ac5d7c11ea5d658f/roles/9814fca799974e3e87ffc04473b6510e` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:38.809738 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b83bfc27-502b-40b8-8862-50b27e5b54f7 None admin] RBAC: Authorizing `identity:create_grant(project_id=5258cb5ffb9c49b1ba3cfb741aef69f8, user_id=08fabeee4f5e4223ac5d7c11ea5d658f, role_id=9814fca799974e3e87ffc04473b6510e)` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:33:38.814455 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b83bfc27-502b-40b8-8862-50b27e5b54f7 None admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:33:38.838110 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 142/289] 10.222.0.13 () {58 vars in 1439 bytes} [Wed Aug 31 09:33:38 2022] PUT /identity/v3/projects/5258cb5ffb9c49b1ba3cfb741aef69f8/users/08fabeee4f5e4223ac5d7c11ea5d658f/roles/9814fca799974e3e87ffc04473b6510e => generated 0 bytes in 306 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:33:38.907476 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6349e00f-2210-4964-8075-d513ddd93ebd None admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:38.951868 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a41d78-4e99-4ab2-b922-aefeefcf0ed5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:38.952485 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a41d78-4e99-4ab2-b922-aefeefcf0ed5 None admin] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:38.952763 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a41d78-4e99-4ab2-b922-aefeefcf0ed5 None admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:38.953029 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a41d78-4e99-4ab2-b922-aefeefcf0ed5 None admin] PATH_INFO: `/v3/roles` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:38.953805 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2a41d78-4e99-4ab2-b922-aefeefcf0ed5 None admin] RBAC: Adding query filter params (name=member) {{(pid=88113) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:33:38.954344 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2a41d78-4e99-4ab2-b922-aefeefcf0ed5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:33:38.955922 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2a41d78-4e99-4ab2-b922-aefeefcf0ed5 None admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:33:38.979956 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 148/290] 10.222.0.13 () {56 vars in 1101 bytes} [Wed Aug 31 09:33:38 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 09:33:38.988360 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b83bfc27-502b-40b8-8862-50b27e5b54f7 None admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:39.098720 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b83bfc27-502b-40b8-8862-50b27e5b54f7 None admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:39.134834 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68071ef4-10a5-4dec-825b-d2d426e9a4dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:39.135478 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68071ef4-10a5-4dec-825b-d2d426e9a4dc None admin] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:39.135756 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68071ef4-10a5-4dec-825b-d2d426e9a4dc None admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:39.136027 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68071ef4-10a5-4dec-825b-d2d426e9a4dc None admin] PATH_INFO: `/v3/users/676c06c0cb4141a080531e4762087dea` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:39.137173 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68071ef4-10a5-4dec-825b-d2d426e9a4dc None admin] RBAC: Authorizing `identity:get_user(user_id=676c06c0cb4141a080531e4762087dea)` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:33:39.138660 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68071ef4-10a5-4dec-825b-d2d426e9a4dc None admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:33:39.140555 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 143/291] 10.222.0.13 () {56 vars in 1177 bytes} [Wed Aug 31 09:33:38 2022] GET /identity/v3/users/676c06c0cb4141a080531e4762087dea => generated 297 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:33:39.147260 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a41d78-4e99-4ab2-b922-aefeefcf0ed5 None admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:39.215252 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=100 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:33:39.279656 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a41d78-4e99-4ab2-b922-aefeefcf0ed5 None admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:39.321948 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d35f29-4089-40b4-8641-a49a35e31d2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:39.322579 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d35f29-4089-40b4-8641-a49a35e31d2b None admin] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:39.322862 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d35f29-4089-40b4-8641-a49a35e31d2b None admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:39.323141 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d35f29-4089-40b4-8641-a49a35e31d2b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:39.397953 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97d35f29-4089-40b4-8641-a49a35e31d2b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:33:39.399420 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97d35f29-4089-40b4-8641-a49a35e31d2b None admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:33:39.446097 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=100 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:33:39.505296 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 149/292] 10.222.0.13 () {58 vars in 1282 bytes} [Wed Aug 31 09:33:39 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 358 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 31 09:33:39.510805 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68071ef4-10a5-4dec-825b-d2d426e9a4dc None admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:39.559096 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:33:39.704868 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68071ef4-10a5-4dec-825b-d2d426e9a4dc None admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:39.761790 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6858edd-2a03-4c5a-b645-72968c908449 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:39.762357 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6858edd-2a03-4c5a-b645-72968c908449 None admin] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:39.762638 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6858edd-2a03-4c5a-b645-72968c908449 None admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:39.762929 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6858edd-2a03-4c5a-b645-72968c908449 None admin] PATH_INFO: `/v3/projects/5258cb5ffb9c49b1ba3cfb741aef69f8` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:39.770423 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6858edd-2a03-4c5a-b645-72968c908449 None admin] RBAC: Authorizing `identity:get_project(project_id=5258cb5ffb9c49b1ba3cfb741aef69f8)` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:33:39.771881 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6858edd-2a03-4c5a-b645-72968c908449 None admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:33:39.773613 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 144/293] 10.222.0.13 () {56 vars in 1186 bytes} [Wed Aug 31 09:33:39 2022] GET /identity/v3/projects/5258cb5ffb9c49b1ba3cfb741aef69f8 => generated 302 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:33:39.779737 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d35f29-4089-40b4-8641-a49a35e31d2b None admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:39.795894 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0daca09-ca43-4a0b-9a00-68db4e2cdef3 None None] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:39.796199 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0daca09-ca43-4a0b-9a00-68db4e2cdef3 None None] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:39.796452 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0daca09-ca43-4a0b-9a00-68db4e2cdef3 None None] PATH_INFO: `/` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:39.798160 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 145/294] 10.222.0.13 () {54 vars in 927 bytes} [Wed Aug 31 09:33:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:33:39.803356 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd77dcec-fb82-4b4e-9afc-a3b36b10825e None None] REQUEST_METHOD: `POST` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:39.803642 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd77dcec-fb82-4b4e-9afc-a3b36b10825e None None] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:39.803917 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd77dcec-fb82-4b4e-9afc-a3b36b10825e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:39.837144 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.auth.core [None req-dd77dcec-fb82-4b4e-9afc-a3b36b10825e None None] MFA Rules not processed for user `fd836cf590a145df9acf19cf3ae2ac56`. Rule list: `[]` (Enabled: `True`). {{(pid=88114) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:33:39.838412 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.fernet_utils [None req-dd77dcec-fb82-4b4e-9afc-a3b36b10825e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88114) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:33:39.955705 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d35f29-4089-40b4-8641-a49a35e31d2b None admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:39.962730 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 146/295] 10.222.0.13 () {58 vars in 1024 bytes} [Wed Aug 31 09:33:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 09:33:39.964575 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3272d19-0780-4dad-8ec4-d432e7210a0f None None] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:39.964848 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3272d19-0780-4dad-8ec4-d432e7210a0f None None] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:39.965121 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3272d19-0780-4dad-8ec4-d432e7210a0f None None] PATH_INFO: `/` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:39.971376 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 147/296] 10.222.0.13 () {54 vars in 927 bytes} [Wed Aug 31 09:33:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:33:39.971376 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8d70c6-db82-4c5c-89e8-a3f9b1305ade None None] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:39.971376 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8d70c6-db82-4c5c-89e8-a3f9b1305ade None None] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:39.971376 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8d70c6-db82-4c5c-89e8-a3f9b1305ade None None] PATH_INFO: `/` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:39.971376 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 148/297] 10.222.0.13 () {54 vars in 927 bytes} [Wed Aug 31 09:33:39 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:33:39.978879 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4fc2b2a-c8e0-4801-a0f3-fac6c61a294a None None] REQUEST_METHOD: `POST` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:39.979163 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4fc2b2a-c8e0-4801-a0f3-fac6c61a294a None None] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:39.979437 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4fc2b2a-c8e0-4801-a0f3-fac6c61a294a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:39.999054 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.auth.core [None req-e4fc2b2a-c8e0-4801-a0f3-fac6c61a294a None None] MFA Rules not processed for user `fd836cf590a145df9acf19cf3ae2ac56`. Rule list: `[]` (Enabled: `True`). {{(pid=88114) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:33:40.000249 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.fernet_utils [None req-e4fc2b2a-c8e0-4801-a0f3-fac6c61a294a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88114) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:33:40.011172 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=100 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:33:40.014444 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-902e1d52-0825-4cfb-b19b-714df1055c4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:40.015038 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902e1d52-0825-4cfb-b19b-714df1055c4d None admin] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:40.015519 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902e1d52-0825-4cfb-b19b-714df1055c4d None admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:40.016196 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902e1d52-0825-4cfb-b19b-714df1055c4d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:40.017909 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-902e1d52-0825-4cfb-b19b-714df1055c4d None admin] RBAC: Adding query filter params (role.id=6c99f5d9f1f243eaa134d8540f21095d, scope.project.id=5258cb5ffb9c49b1ba3cfb741aef69f8, user.id=676c06c0cb4141a080531e4762087dea) {{(pid=88113) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:33:40.018678 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-902e1d52-0825-4cfb-b19b-714df1055c4d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:33:40.020823 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-902e1d52-0825-4cfb-b19b-714df1055c4d None admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:33:40.029628 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 150/298] 10.222.0.13 () {56 vars in 1374 bytes} [Wed Aug 31 09:33:39 2022] GET /identity/v3/role_assignments?user.id=676c06c0cb4141a080531e4762087dea&scope.project.id=5258cb5ffb9c49b1ba3cfb741aef69f8&role.id=6c99f5d9f1f243eaa134d8540f21095d => generated 608 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:33:40.031169 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9305e06-be68-4f2c-8da6-f512e3a78b47 None None] REQUEST_METHOD: `POST` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:40.031621 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9305e06-be68-4f2c-8da6-f512e3a78b47 None None] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:40.032598 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9305e06-be68-4f2c-8da6-f512e3a78b47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:40.054793 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.auth.core [None req-a9305e06-be68-4f2c-8da6-f512e3a78b47 None None] MFA Rules not processed for user `fd836cf590a145df9acf19cf3ae2ac56`. Rule list: `[]` (Enabled: `True`). {{(pid=88113) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:33:40.057531 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.fernet_utils [None req-a9305e06-be68-4f2c-8da6-f512e3a78b47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:33:40.060306 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=100 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:33:40.138795 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 149/299] 10.222.0.13 () {58 vars in 1024 bytes} [Wed Aug 31 09:33:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 09:33:40.175075 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 151/300] 10.222.0.13 () {58 vars in 1024 bytes} [Wed Aug 31 09:33:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 09:33:40.268652 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4fc2b2a-c8e0-4801-a0f3-fac6c61a294a None None] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:40.410840 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4fc2b2a-c8e0-4801-a0f3-fac6c61a294a None None] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:40.448569 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f551dbc-6d22-4c50-8133-5fa8f8993036 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:40.449210 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f551dbc-6d22-4c50-8133-5fa8f8993036 None admin] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:40.449518 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f551dbc-6d22-4c50-8133-5fa8f8993036 None admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:40.449793 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f551dbc-6d22-4c50-8133-5fa8f8993036 None admin] PATH_INFO: `/v3/domains/default` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:40.450925 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f551dbc-6d22-4c50-8133-5fa8f8993036 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:33:40.452402 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f551dbc-6d22-4c50-8133-5fa8f8993036 None admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:33:40.454134 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 150/301] 10.222.0.13 () {56 vars in 1108 bytes} [Wed Aug 31 09:33:40 2022] GET /identity/v3/domains/default => generated 199 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:33:40.460156 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9305e06-be68-4f2c-8da6-f512e3a78b47 None None] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:40.474008 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f551dbc-6d22-4c50-8133-5fa8f8993036 None admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:40.491409 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=100 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:33:40.590445 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9305e06-be68-4f2c-8da6-f512e3a78b47 None None] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:40.613411 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f551dbc-6d22-4c50-8133-5fa8f8993036 None admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:40.635936 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c3e2d3d-b8d3-4254-9306-8569b15cd539 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:40.637023 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3e2d3d-b8d3-4254-9306-8569b15cd539 None admin] REQUEST_METHOD: `POST` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:40.637306 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3e2d3d-b8d3-4254-9306-8569b15cd539 None admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:40.637756 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3e2d3d-b8d3-4254-9306-8569b15cd539 None admin] PATH_INFO: `/v3/groups` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:40.638401 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c3e2d3d-b8d3-4254-9306-8569b15cd539 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:33:40.639867 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c3e2d3d-b8d3-4254-9306-8569b15cd539 None admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:33:40.654663 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa1da55-a5a5-44bc-b654-9a3f1fd3ccf2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:40.655310 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa1da55-a5a5-44bc-b654-9a3f1fd3ccf2 None admin] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:40.655587 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa1da55-a5a5-44bc-b654-9a3f1fd3ccf2 None admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:40.655895 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa1da55-a5a5-44bc-b654-9a3f1fd3ccf2 None admin] PATH_INFO: `/v3/roles/member` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:40.658297 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:33:40.664656 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 152/302] 10.222.0.13 () {60 vars in 1134 bytes} [Wed Aug 31 09:33:40 2022] POST /identity/v3/groups => generated 229 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 09:33:40.666057 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c3e2d3d-b8d3-4254-9306-8569b15cd539 None admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:40.679626 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aa1da55-a5a5-44bc-b654-9a3f1fd3ccf2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 09:33:40.688029 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application [None req-8aa1da55-a5a5-44bc-b654-9a3f1fd3ccf2 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 31 09:33:40.688029 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:33:40.688029 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:33:40.688029 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:33:40.688029 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:33:40.688029 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:33:40.688029 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:33:40.688029 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:33:40.688029 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:33:40.688029 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:33:40.688029 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:33:40.688029 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:33:40.688029 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 09:33:40.688029 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 09:33:40.688029 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 09:33:40.688029 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 09:33:40.689573 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 09:33:40.689573 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 09:33:40.689573 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 09:33:40.689573 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 09:33:40.689573 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:33:40.689573 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:33:40.689573 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:33:40.689573 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:33:40.689573 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:33:40.689573 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:33:40.689573 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:33:40.689573 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application with Lock( Aug 31 09:33:40.689573 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:33:40.689573 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:33:40.689573 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:33:40.689573 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:33:40.690400 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:33:40.690400 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:33:40.690400 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:33:40.690400 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:33:40.690400 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 09:33:40.690400 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 09:33:40.690400 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 09:33:40.690400 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 09:33:40.690400 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 09:33:40.690400 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 09:33:40.690400 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 31 09:33:40.690400 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application Aug 31 09:33:40.698797 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 151/303] 10.222.0.13 () {56 vars in 1099 bytes} [Wed Aug 31 09:33:40 2022] GET /identity/v3/roles/member => generated 84 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 09:33:40.699371 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa1da55-a5a5-44bc-b654-9a3f1fd3ccf2 None admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:40.809118 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=100 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:33:40.817612 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c3e2d3d-b8d3-4254-9306-8569b15cd539 None admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:40.849243 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa1da55-a5a5-44bc-b654-9a3f1fd3ccf2 None admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:40.870347 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e6ee491-d38a-4749-9b3c-7c95fda5afd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:40.871023 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6ee491-d38a-4749-9b3c-7c95fda5afd4 None admin] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:40.871300 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6ee491-d38a-4749-9b3c-7c95fda5afd4 None admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:40.871566 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6ee491-d38a-4749-9b3c-7c95fda5afd4 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:40.884685 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e6ee491-d38a-4749-9b3c-7c95fda5afd4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 09:33:40.890900 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application [None req-4e6ee491-d38a-4749-9b3c-7c95fda5afd4 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 31 09:33:40.890900 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:33:40.890900 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:33:40.890900 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:33:40.890900 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:33:40.890900 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:33:40.890900 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:33:40.890900 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:33:40.890900 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:33:40.890900 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:33:40.890900 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:33:40.890900 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:33:40.890900 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 09:33:40.890900 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 09:33:40.890900 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 09:33:40.890900 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 09:33:40.892529 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 09:33:40.892529 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 09:33:40.892529 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 09:33:40.892529 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 09:33:40.892529 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:33:40.892529 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:33:40.892529 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:33:40.892529 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:33:40.892529 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:33:40.892529 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:33:40.892529 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:33:40.892529 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application with Lock( Aug 31 09:33:40.892529 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:33:40.892529 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:33:40.892529 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:33:40.892529 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:33:40.894102 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:33:40.894102 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:33:40.894102 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:33:40.894102 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:33:40.894102 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 09:33:40.894102 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 09:33:40.894102 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 09:33:40.894102 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 09:33:40.894102 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 09:33:40.894102 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 09:33:40.894102 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 31 09:33:40.894102 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application Aug 31 09:33:40.894102 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 153/304] 10.222.0.13 () {56 vars in 1099 bytes} [Wed Aug 31 09:33:40 2022] GET /identity/v3/roles/reader => generated 84 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:33:40.898235 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e6ee491-d38a-4749-9b3c-7c95fda5afd4 None admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:40.908764 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c38fa689-e932-4a0c-ba96-1c82391c1002 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:40.911050 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38fa689-e932-4a0c-ba96-1c82391c1002 None admin] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:40.911338 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38fa689-e932-4a0c-ba96-1c82391c1002 None admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:40.911712 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38fa689-e932-4a0c-ba96-1c82391c1002 None admin] PATH_INFO: `/v3/roles` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:40.912670 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c38fa689-e932-4a0c-ba96-1c82391c1002 None admin] RBAC: Adding query filter params (name=member) {{(pid=88114) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:33:40.913268 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c38fa689-e932-4a0c-ba96-1c82391c1002 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:33:40.914457 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c38fa689-e932-4a0c-ba96-1c82391c1002 None admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:33:40.924148 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 152/305] 10.222.0.13 () {56 vars in 1101 bytes} [Wed Aug 31 09:33:40 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:33:40.935260 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c38fa689-e932-4a0c-ba96-1c82391c1002 None admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:40.970377 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=100 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:33:41.019190 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e6ee491-d38a-4749-9b3c-7c95fda5afd4 None admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:41.051547 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c38fa689-e932-4a0c-ba96-1c82391c1002 None admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:41.056843 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c037a1fb-f238-4f6e-80b3-4043924dd575 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:41.057478 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c037a1fb-f238-4f6e-80b3-4043924dd575 None admin] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:41.057756 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c037a1fb-f238-4f6e-80b3-4043924dd575 None admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:41.058022 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c037a1fb-f238-4f6e-80b3-4043924dd575 None admin] PATH_INFO: `/v3/roles` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:41.058696 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c037a1fb-f238-4f6e-80b3-4043924dd575 None admin] RBAC: Adding query filter params (name=reader) {{(pid=88113) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:33:41.058996 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c037a1fb-f238-4f6e-80b3-4043924dd575 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:33:41.060394 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c037a1fb-f238-4f6e-80b3-4043924dd575 None admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:33:41.075810 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 154/306] 10.222.0.13 () {56 vars in 1101 bytes} [Wed Aug 31 09:33:40 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 09:33:41.077765 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c037a1fb-f238-4f6e-80b3-4043924dd575 None admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:41.106685 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73485ff9-5403-46d6-b120-5a11d31ec07f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:41.107954 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73485ff9-5403-46d6-b120-5a11d31ec07f None admin] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:41.108439 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73485ff9-5403-46d6-b120-5a11d31ec07f None admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:41.109230 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73485ff9-5403-46d6-b120-5a11d31ec07f None admin] PATH_INFO: `/v3/users/22d585550661496e956a728556c3d90d` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:41.111723 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73485ff9-5403-46d6-b120-5a11d31ec07f None admin] RBAC: Authorizing `identity:get_user(user_id=22d585550661496e956a728556c3d90d)` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:33:41.114422 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73485ff9-5403-46d6-b120-5a11d31ec07f None admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:33:41.121730 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 153/307] 10.222.0.13 () {56 vars in 1177 bytes} [Wed Aug 31 09:33:40 2022] GET /identity/v3/users/22d585550661496e956a728556c3d90d => generated 293 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:33:41.122798 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73485ff9-5403-46d6-b120-5a11d31ec07f None admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:41.129537 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=100 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:33:41.210144 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c037a1fb-f238-4f6e-80b3-4043924dd575 None admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:41.232627 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=100 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:33:41.248677 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3bdac13-8686-43df-a741-7d2794b93af3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:41.249323 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3bdac13-8686-43df-a741-7d2794b93af3 None admin] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:41.249624 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3bdac13-8686-43df-a741-7d2794b93af3 None admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:41.249898 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3bdac13-8686-43df-a741-7d2794b93af3 None admin] PATH_INFO: `/v3/users/12cfd2823ac141b0ac63a7c988146d85` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:41.260256 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73485ff9-5403-46d6-b120-5a11d31ec07f None admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:41.265997 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3bdac13-8686-43df-a741-7d2794b93af3 None admin] RBAC: Authorizing `identity:get_user(user_id=12cfd2823ac141b0ac63a7c988146d85)` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:33:41.268359 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3bdac13-8686-43df-a741-7d2794b93af3 None admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:33:41.270395 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 155/308] 10.222.0.13 () {56 vars in 1177 bytes} [Wed Aug 31 09:33:41 2022] GET /identity/v3/users/12cfd2823ac141b0ac63a7c988146d85 => generated 293 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:33:41.275568 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31c4f8c-3686-4df7-9a32-ae72944dab7f None None] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:41.275849 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31c4f8c-3686-4df7-9a32-ae72944dab7f None None] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:41.276121 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31c4f8c-3686-4df7-9a32-ae72944dab7f None None] PATH_INFO: `/` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:41.278464 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 156/309] 10.222.0.13 () {54 vars in 927 bytes} [Wed Aug 31 09:33:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:33:41.279056 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f31c4f8c-3686-4df7-9a32-ae72944dab7f None None] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:41.305910 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41257c47-a337-4325-bb77-5fe31632a94c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:41.306582 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41257c47-a337-4325-bb77-5fe31632a94c None admin] REQUEST_METHOD: `PUT` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:41.306860 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41257c47-a337-4325-bb77-5fe31632a94c None admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:41.307133 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41257c47-a337-4325-bb77-5fe31632a94c None admin] PATH_INFO: `/v3/system/users/22d585550661496e956a728556c3d90d/roles/6c99f5d9f1f243eaa134d8540f21095d` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:41.308581 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41257c47-a337-4325-bb77-5fe31632a94c None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=22d585550661496e956a728556c3d90d, role_id=6c99f5d9f1f243eaa134d8540f21095d)` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:33:41.310004 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41257c47-a337-4325-bb77-5fe31632a94c None admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:33:41.321933 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 154/310] 10.222.0.13 () {58 vars in 1334 bytes} [Wed Aug 31 09:33:41 2022] PUT /identity/v3/system/users/22d585550661496e956a728556c3d90d/roles/6c99f5d9f1f243eaa134d8540f21095d => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:33:41.323923 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d61d062-60ac-47d3-93ee-4cc71a18079e None None] REQUEST_METHOD: `POST` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:41.324416 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d61d062-60ac-47d3-93ee-4cc71a18079e None None] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:41.324904 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d61d062-60ac-47d3-93ee-4cc71a18079e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:41.363796 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.auth.core [None req-3d61d062-60ac-47d3-93ee-4cc71a18079e None None] MFA Rules not processed for user `fd836cf590a145df9acf19cf3ae2ac56`. Rule list: `[]` (Enabled: `True`). {{(pid=88114) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:33:41.365327 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.fernet_utils [None req-3d61d062-60ac-47d3-93ee-4cc71a18079e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88114) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:33:41.417185 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f31c4f8c-3686-4df7-9a32-ae72944dab7f None None] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:41.460139 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b8210b9-7783-46cb-8c39-7d56f9409b3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:41.460875 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8210b9-7783-46cb-8c39-7d56f9409b3e None admin] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:41.461440 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8210b9-7783-46cb-8c39-7d56f9409b3e None admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:41.461627 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:33:41.462588 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8210b9-7783-46cb-8c39-7d56f9409b3e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:41.463814 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b8210b9-7783-46cb-8c39-7d56f9409b3e None admin] RBAC: Adding query filter params (role.id=9814fca799974e3e87ffc04473b6510e, user.id=12cfd2823ac141b0ac63a7c988146d85, scope.system=all) {{(pid=88113) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:33:41.464276 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b8210b9-7783-46cb-8c39-7d56f9409b3e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:33:41.466209 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b8210b9-7783-46cb-8c39-7d56f9409b3e None admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:33:41.474171 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 157/311] 10.222.0.13 () {56 vars in 1308 bytes} [Wed Aug 31 09:33:41 2022] GET /identity/v3/role_assignments?user.id=12cfd2823ac141b0ac63a7c988146d85&scope.system=all&role.id=9814fca799974e3e87ffc04473b6510e => generated 226 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:33:41.475241 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 155/312] 10.222.0.13 () {58 vars in 1024 bytes} [Wed Aug 31 09:33:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 09:33:41.756410 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b8210b9-7783-46cb-8c39-7d56f9409b3e None admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:41.807560 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=100 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:33:41.872298 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b8210b9-7783-46cb-8c39-7d56f9409b3e None admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:41.911882 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-121782bc-f195-4a47-ad32-0b022f67ca3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:41.912547 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121782bc-f195-4a47-ad32-0b022f67ca3b None admin] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:41.912823 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121782bc-f195-4a47-ad32-0b022f67ca3b None admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:41.913089 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121782bc-f195-4a47-ad32-0b022f67ca3b None admin] PATH_INFO: `/v3/roles/reader` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:41.927010 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-121782bc-f195-4a47-ad32-0b022f67ca3b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 09:33:41.931928 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application [None req-121782bc-f195-4a47-ad32-0b022f67ca3b None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 31 09:33:41.931928 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:33:41.931928 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:33:41.931928 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:33:41.931928 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:33:41.931928 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:33:41.931928 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:33:41.931928 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:33:41.931928 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:33:41.931928 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:33:41.931928 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:33:41.931928 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:33:41.931928 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 09:33:41.931928 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 09:33:41.931928 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 09:33:41.931928 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 09:33:41.932811 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 09:33:41.932811 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 09:33:41.932811 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 09:33:41.932811 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 09:33:41.932811 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:33:41.932811 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:33:41.932811 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:33:41.932811 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:33:41.932811 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:33:41.932811 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:33:41.932811 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:33:41.932811 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application with Lock( Aug 31 09:33:41.932811 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:33:41.932811 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:33:41.932811 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:33:41.932811 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:33:41.933648 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:33:41.933648 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:33:41.933648 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:33:41.933648 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:33:41.933648 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 09:33:41.933648 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 09:33:41.933648 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 09:33:41.933648 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 09:33:41.933648 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 09:33:41.933648 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 09:33:41.933648 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 31 09:33:41.933648 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application Aug 31 09:33:41.933648 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 158/313] 10.222.0.13 () {56 vars in 1099 bytes} [Wed Aug 31 09:33:41 2022] GET /identity/v3/roles/reader => generated 84 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:33:41.940328 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d61d062-60ac-47d3-93ee-4cc71a18079e None None] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:42.053083 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d61d062-60ac-47d3-93ee-4cc71a18079e None None] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:42.100168 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2574030-d65b-433c-8a61-4ed270feb1cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:42.100782 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2574030-d65b-433c-8a61-4ed270feb1cf None admin] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:42.101063 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2574030-d65b-433c-8a61-4ed270feb1cf None admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:42.101337 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2574030-d65b-433c-8a61-4ed270feb1cf None admin] PATH_INFO: `/v3/roles` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:42.102096 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2574030-d65b-433c-8a61-4ed270feb1cf None admin] RBAC: Adding query filter params (name=reader) {{(pid=88114) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:33:42.102403 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2574030-d65b-433c-8a61-4ed270feb1cf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:33:42.103814 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2574030-d65b-433c-8a61-4ed270feb1cf None admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:33:42.113669 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 156/314] 10.222.0.13 () {56 vars in 1101 bytes} [Wed Aug 31 09:33:41 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:33:42.119333 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-121782bc-f195-4a47-ad32-0b022f67ca3b None admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:42.232299 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-121782bc-f195-4a47-ad32-0b022f67ca3b None admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:42.234066 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=100 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:33:42.278034 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c53395-738b-47ab-a207-7b211f7438c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:42.278839 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c53395-738b-47ab-a207-7b211f7438c0 None admin] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:42.279205 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c53395-738b-47ab-a207-7b211f7438c0 None admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:42.279503 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c53395-738b-47ab-a207-7b211f7438c0 None admin] PATH_INFO: `/v3/users/08fabeee4f5e4223ac5d7c11ea5d658f` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:42.280768 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53c53395-738b-47ab-a207-7b211f7438c0 None admin] RBAC: Authorizing `identity:get_user(user_id=08fabeee4f5e4223ac5d7c11ea5d658f)` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:33:42.282770 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53c53395-738b-47ab-a207-7b211f7438c0 None admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:33:42.285169 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 159/315] 10.222.0.13 () {56 vars in 1177 bytes} [Wed Aug 31 09:33:42 2022] GET /identity/v3/users/08fabeee4f5e4223ac5d7c11ea5d658f => generated 297 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:33:42.291000 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2574030-d65b-433c-8a61-4ed270feb1cf None admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:42.358587 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=100 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:33:42.405220 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2574030-d65b-433c-8a61-4ed270feb1cf None admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:42.445089 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a70f62ca-f22b-4af8-b4a1-b6bfcddd5053 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:42.446113 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70f62ca-f22b-4af8-b4a1-b6bfcddd5053 None admin] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:42.446614 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70f62ca-f22b-4af8-b4a1-b6bfcddd5053 None admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:42.446985 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70f62ca-f22b-4af8-b4a1-b6bfcddd5053 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:42.522804 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a70f62ca-f22b-4af8-b4a1-b6bfcddd5053 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:33:42.524377 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a70f62ca-f22b-4af8-b4a1-b6bfcddd5053 None admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:33:42.590801 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=100 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:33:42.636968 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 157/316] 10.222.0.13 () {58 vars in 1282 bytes} [Wed Aug 31 09:33:42 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 346 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 31 09:33:42.643810 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c53395-738b-47ab-a207-7b211f7438c0 None admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:42.764153 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c53395-738b-47ab-a207-7b211f7438c0 None admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:42.803363 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e1faee2-9b27-411f-bfd5-6134d6a41169 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:42.804375 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1faee2-9b27-411f-bfd5-6134d6a41169 None admin] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:42.804856 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1faee2-9b27-411f-bfd5-6134d6a41169 None admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:42.805347 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1faee2-9b27-411f-bfd5-6134d6a41169 None admin] PATH_INFO: `/v3/projects/5258cb5ffb9c49b1ba3cfb741aef69f8` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:42.806771 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e1faee2-9b27-411f-bfd5-6134d6a41169 None admin] RBAC: Authorizing `identity:get_project(project_id=5258cb5ffb9c49b1ba3cfb741aef69f8)` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:33:42.808447 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e1faee2-9b27-411f-bfd5-6134d6a41169 None admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:33:42.810695 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 160/317] 10.222.0.13 () {56 vars in 1186 bytes} [Wed Aug 31 09:33:42 2022] GET /identity/v3/projects/5258cb5ffb9c49b1ba3cfb741aef69f8 => generated 302 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:33:42.818048 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a70f62ca-f22b-4af8-b4a1-b6bfcddd5053 None admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:42.882638 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d87016b-9189-4a13-91a9-0ce925ccc936 None None] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:42.882934 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d87016b-9189-4a13-91a9-0ce925ccc936 None None] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:42.883357 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d87016b-9189-4a13-91a9-0ce925ccc936 None None] PATH_INFO: `/` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:42.884750 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 161/318] 10.222.0.13 () {54 vars in 927 bytes} [Wed Aug 31 09:33:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 31 09:33:42.890275 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a9eb49-1e3e-4c19-a56c-ad5fed68eb0d None None] REQUEST_METHOD: `POST` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:42.890474 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a9eb49-1e3e-4c19-a56c-ad5fed68eb0d None None] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:42.890902 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a9eb49-1e3e-4c19-a56c-ad5fed68eb0d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:42.911588 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.auth.core [None req-18a9eb49-1e3e-4c19-a56c-ad5fed68eb0d None None] MFA Rules not processed for user `fd836cf590a145df9acf19cf3ae2ac56`. Rule list: `[]` (Enabled: `True`). {{(pid=88113) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:33:42.912924 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.fernet_utils [None req-18a9eb49-1e3e-4c19-a56c-ad5fed68eb0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:33:42.954027 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=100 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:33:42.968805 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a70f62ca-f22b-4af8-b4a1-b6bfcddd5053 None admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:43.016274 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 162/319] 10.222.0.13 () {58 vars in 1024 bytes} [Wed Aug 31 09:33:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 31 09:33:43.025045 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab4aaa6b-8305-4b9b-9196-9ebf7477935e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:43.026003 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4aaa6b-8305-4b9b-9196-9ebf7477935e None admin] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:43.026617 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4aaa6b-8305-4b9b-9196-9ebf7477935e None admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:43.027474 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4aaa6b-8305-4b9b-9196-9ebf7477935e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:43.029117 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab4aaa6b-8305-4b9b-9196-9ebf7477935e None admin] RBAC: Adding query filter params (role.id=9814fca799974e3e87ffc04473b6510e, scope.project.id=5258cb5ffb9c49b1ba3cfb741aef69f8, user.id=08fabeee4f5e4223ac5d7c11ea5d658f) {{(pid=88114) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:33:43.029840 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab4aaa6b-8305-4b9b-9196-9ebf7477935e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:33:43.033714 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab4aaa6b-8305-4b9b-9196-9ebf7477935e None admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:33:43.045157 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 158/320] 10.222.0.13 () {56 vars in 1374 bytes} [Wed Aug 31 09:33:42 2022] GET /identity/v3/role_assignments?user.id=08fabeee4f5e4223ac5d7c11ea5d658f&scope.project.id=5258cb5ffb9c49b1ba3cfb741aef69f8&role.id=9814fca799974e3e87ffc04473b6510e => generated 608 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:33:43.434750 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5cc24a-c62e-4ade-9d95-f44377ea4ef6 None None] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:43.435126 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5cc24a-c62e-4ade-9d95-f44377ea4ef6 None None] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:43.435302 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5cc24a-c62e-4ade-9d95-f44377ea4ef6 None None] PATH_INFO: `/` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:43.436807 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 163/321] 10.222.0.13 () {54 vars in 927 bytes} [Wed Aug 31 09:33:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:33:43.442574 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76cfa38b-96fa-4496-b776-8b1314e9ddc3 None None] REQUEST_METHOD: `POST` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:43.443013 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76cfa38b-96fa-4496-b776-8b1314e9ddc3 None None] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:43.443296 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76cfa38b-96fa-4496-b776-8b1314e9ddc3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:43.462646 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.auth.core [None req-76cfa38b-96fa-4496-b776-8b1314e9ddc3 None None] MFA Rules not processed for user `fd836cf590a145df9acf19cf3ae2ac56`. Rule list: `[]` (Enabled: `True`). {{(pid=88114) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:33:43.464042 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.fernet_utils [None req-76cfa38b-96fa-4496-b776-8b1314e9ddc3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88114) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:33:43.471361 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8279a054-847d-44c4-9bd3-689548714a2b None None] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:43.471661 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8279a054-847d-44c4-9bd3-689548714a2b None None] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:43.471900 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8279a054-847d-44c4-9bd3-689548714a2b None None] PATH_INFO: `/` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:43.473440 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 164/322] 10.222.0.13 () {54 vars in 927 bytes} [Wed Aug 31 09:33:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 31 09:33:43.478704 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10225e5-3ce9-46e4-8582-b09d808504c4 None None] REQUEST_METHOD: `POST` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:43.478990 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10225e5-3ce9-46e4-8582-b09d808504c4 None None] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:43.479262 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10225e5-3ce9-46e4-8582-b09d808504c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:43.484911 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=100 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:33:43.499238 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.auth.core [None req-b10225e5-3ce9-46e4-8582-b09d808504c4 None None] MFA Rules not processed for user `fd836cf590a145df9acf19cf3ae2ac56`. Rule list: `[]` (Enabled: `True`). {{(pid=88113) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:33:43.500364 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.fernet_utils [None req-b10225e5-3ce9-46e4-8582-b09d808504c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:33:43.565471 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 159/323] 10.222.0.13 () {58 vars in 1024 bytes} [Wed Aug 31 09:33:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 09:33:43.565471 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76cfa38b-96fa-4496-b776-8b1314e9ddc3 None None] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:43.600461 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 165/324] 10.222.0.13 () {58 vars in 1024 bytes} [Wed Aug 31 09:33:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 31 09:33:43.693101 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76cfa38b-96fa-4496-b776-8b1314e9ddc3 None None] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:43.734206 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab0a8acf-7cc4-4a40-b758-896b3a00cd79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:43.734873 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0a8acf-7cc4-4a40-b758-896b3a00cd79 None admin] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:43.735212 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0a8acf-7cc4-4a40-b758-896b3a00cd79 None admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:43.735531 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0a8acf-7cc4-4a40-b758-896b3a00cd79 None admin] PATH_INFO: `/v3/domains/default` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:43.736922 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab0a8acf-7cc4-4a40-b758-896b3a00cd79 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:33:43.738446 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab0a8acf-7cc4-4a40-b758-896b3a00cd79 None admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:33:43.740302 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 160/325] 10.222.0.13 () {56 vars in 1108 bytes} [Wed Aug 31 09:33:43 2022] GET /identity/v3/domains/default => generated 199 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:33:43.745933 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b10225e5-3ce9-46e4-8582-b09d808504c4 None None] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:43.829677 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab0a8acf-7cc4-4a40-b758-896b3a00cd79 None admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:43.865168 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=100 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:33:43.868366 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=100 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:33:43.873786 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b10225e5-3ce9-46e4-8582-b09d808504c4 None None] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:43.916160 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-362f3916-f155-4ee4-b843-eaac5c96de92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:43.916785 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362f3916-f155-4ee4-b843-eaac5c96de92 None admin] REQUEST_METHOD: `POST` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:43.917061 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362f3916-f155-4ee4-b843-eaac5c96de92 None admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:43.917330 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362f3916-f155-4ee4-b843-eaac5c96de92 None admin] PATH_INFO: `/v3/groups` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:43.918023 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-362f3916-f155-4ee4-b843-eaac5c96de92 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:33:43.919497 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-362f3916-f155-4ee4-b843-eaac5c96de92 None admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:33:43.947115 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 166/326] 10.222.0.13 () {60 vars in 1134 bytes} [Wed Aug 31 09:33:43 2022] POST /identity/v3/groups => generated 226 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 09:33:43.947115 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-362f3916-f155-4ee4-b843-eaac5c96de92 None admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:43.990215 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab0a8acf-7cc4-4a40-b758-896b3a00cd79 None admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:44.042812 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48733c24-8c18-4bcd-8db6-a7112e740bec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:44.043538 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48733c24-8c18-4bcd-8db6-a7112e740bec None admin] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:44.043902 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48733c24-8c18-4bcd-8db6-a7112e740bec None admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:44.044356 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48733c24-8c18-4bcd-8db6-a7112e740bec None admin] PATH_INFO: `/v3/roles/reader` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:44.060989 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48733c24-8c18-4bcd-8db6-a7112e740bec None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 09:33:44.070306 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application [None req-48733c24-8c18-4bcd-8db6-a7112e740bec None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 31 09:33:44.070306 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:33:44.070306 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:33:44.070306 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:33:44.070306 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:33:44.070306 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:33:44.070306 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:33:44.070306 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:33:44.070306 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:33:44.070306 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:33:44.070306 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:33:44.070306 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:33:44.070306 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 09:33:44.070306 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 09:33:44.070306 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 09:33:44.070306 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 09:33:44.071960 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 09:33:44.071960 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 09:33:44.071960 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 09:33:44.071960 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 09:33:44.071960 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:33:44.071960 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:33:44.071960 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:33:44.071960 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:33:44.071960 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:33:44.071960 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:33:44.071960 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:33:44.071960 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application with Lock( Aug 31 09:33:44.071960 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:33:44.071960 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:33:44.071960 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:33:44.071960 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:33:44.073344 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:33:44.073344 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:33:44.073344 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:33:44.073344 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:33:44.073344 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 09:33:44.073344 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 09:33:44.073344 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 09:33:44.073344 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 09:33:44.073344 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 09:33:44.073344 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 09:33:44.073344 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 31 09:33:44.073344 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application Aug 31 09:33:44.073344 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 161/327] 10.222.0.13 () {56 vars in 1099 bytes} [Wed Aug 31 09:33:43 2022] GET /identity/v3/roles/reader => generated 84 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 09:33:44.079592 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48733c24-8c18-4bcd-8db6-a7112e740bec None admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:44.094339 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-362f3916-f155-4ee4-b843-eaac5c96de92 None admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:44.154853 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04db16c3-3e2e-4804-9921-16988d688cdb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:44.155859 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04db16c3-3e2e-4804-9921-16988d688cdb None admin] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:44.156312 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04db16c3-3e2e-4804-9921-16988d688cdb None admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:44.156741 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04db16c3-3e2e-4804-9921-16988d688cdb None admin] PATH_INFO: `/v3/roles/member` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:44.172785 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04db16c3-3e2e-4804-9921-16988d688cdb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 09:33:44.179385 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application [None req-04db16c3-3e2e-4804-9921-16988d688cdb None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 31 09:33:44.179385 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:33:44.179385 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:33:44.179385 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:33:44.179385 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:33:44.179385 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:33:44.179385 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:33:44.179385 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:33:44.179385 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:33:44.179385 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:33:44.179385 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:33:44.179385 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:33:44.179385 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 09:33:44.179385 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 09:33:44.179385 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 09:33:44.179385 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 09:33:44.180474 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 09:33:44.180474 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 09:33:44.180474 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 09:33:44.180474 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 09:33:44.180474 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:33:44.180474 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:33:44.180474 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:33:44.180474 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:33:44.180474 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:33:44.180474 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:33:44.180474 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:33:44.180474 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application with Lock( Aug 31 09:33:44.180474 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:33:44.180474 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:33:44.180474 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:33:44.180474 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:33:44.181362 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:33:44.181362 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:33:44.181362 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:33:44.181362 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:33:44.181362 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 09:33:44.181362 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 09:33:44.181362 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 09:33:44.181362 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 09:33:44.181362 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 09:33:44.181362 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 09:33:44.181362 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 31 09:33:44.181362 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application Aug 31 09:33:44.183387 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 167/328] 10.222.0.13 () {56 vars in 1099 bytes} [Wed Aug 31 09:33:43 2022] GET /identity/v3/roles/member => generated 84 bytes in 245 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:33:44.190632 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04db16c3-3e2e-4804-9921-16988d688cdb None admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:44.226525 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=100 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:33:44.226691 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:33:44.271524 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48733c24-8c18-4bcd-8db6-a7112e740bec None admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:44.348983 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ea68bb-6707-4a9e-a654-f2a4e02720aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:44.350508 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ea68bb-6707-4a9e-a654-f2a4e02720aa None admin] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:44.351245 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ea68bb-6707-4a9e-a654-f2a4e02720aa None admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:44.351983 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ea68bb-6707-4a9e-a654-f2a4e02720aa None admin] PATH_INFO: `/v3/roles` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:44.353470 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63ea68bb-6707-4a9e-a654-f2a4e02720aa None admin] RBAC: Adding query filter params (name=reader) {{(pid=88114) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:33:44.354298 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63ea68bb-6707-4a9e-a654-f2a4e02720aa None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:33:44.357470 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63ea68bb-6707-4a9e-a654-f2a4e02720aa None admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:33:44.376374 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 162/329] 10.222.0.13 () {56 vars in 1101 bytes} [Wed Aug 31 09:33:44 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 298 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 09:33:44.377745 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04db16c3-3e2e-4804-9921-16988d688cdb None admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:44.385017 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ea68bb-6707-4a9e-a654-f2a4e02720aa None admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:44.440870 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b293dc0-d0ca-4111-84c5-9ef26adce25c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:44.441749 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b293dc0-d0ca-4111-84c5-9ef26adce25c None admin] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:44.442166 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b293dc0-d0ca-4111-84c5-9ef26adce25c None admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:44.442599 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b293dc0-d0ca-4111-84c5-9ef26adce25c None admin] PATH_INFO: `/v3/roles` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:44.443717 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b293dc0-d0ca-4111-84c5-9ef26adce25c None admin] RBAC: Adding query filter params (name=member) {{(pid=88113) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:33:44.444229 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b293dc0-d0ca-4111-84c5-9ef26adce25c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:33:44.446663 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b293dc0-d0ca-4111-84c5-9ef26adce25c None admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:33:44.462788 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 168/330] 10.222.0.13 () {56 vars in 1101 bytes} [Wed Aug 31 09:33:44 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:33:44.472522 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b293dc0-d0ca-4111-84c5-9ef26adce25c None admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:44.596907 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=100 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:33:44.614450 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ea68bb-6707-4a9e-a654-f2a4e02720aa None admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:44.614702 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b293dc0-d0ca-4111-84c5-9ef26adce25c None admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:44.667392 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd50ab7-e6ea-4059-b17e-3dc1dde38a24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:44.668691 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd50ab7-e6ea-4059-b17e-3dc1dde38a24 None admin] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:44.670393 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd50ab7-e6ea-4059-b17e-3dc1dde38a24 None admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:44.671041 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd50ab7-e6ea-4059-b17e-3dc1dde38a24 None admin] PATH_INFO: `/v3/users/22d585550661496e956a728556c3d90d` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:44.673281 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cd50ab7-e6ea-4059-b17e-3dc1dde38a24 None admin] RBAC: Authorizing `identity:get_user(user_id=22d585550661496e956a728556c3d90d)` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:33:44.676575 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cd50ab7-e6ea-4059-b17e-3dc1dde38a24 None admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:33:44.679443 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 169/331] 10.222.0.13 () {56 vars in 1177 bytes} [Wed Aug 31 09:33:44 2022] GET /identity/v3/users/22d585550661496e956a728556c3d90d => generated 293 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:33:44.689120 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0753979-426d-4b7f-99ef-85610935c26f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:44.690008 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0753979-426d-4b7f-99ef-85610935c26f None admin] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:44.690432 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0753979-426d-4b7f-99ef-85610935c26f None admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:44.690588 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd50ab7-e6ea-4059-b17e-3dc1dde38a24 None admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:44.690951 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0753979-426d-4b7f-99ef-85610935c26f None admin] PATH_INFO: `/v3/users/12cfd2823ac141b0ac63a7c988146d85` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:44.692736 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0753979-426d-4b7f-99ef-85610935c26f None admin] RBAC: Authorizing `identity:get_user(user_id=12cfd2823ac141b0ac63a7c988146d85)` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:33:44.695549 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0753979-426d-4b7f-99ef-85610935c26f None admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:33:44.699643 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 163/332] 10.222.0.13 () {56 vars in 1177 bytes} [Wed Aug 31 09:33:44 2022] GET /identity/v3/users/12cfd2823ac141b0ac63a7c988146d85 => generated 293 bytes in 316 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:33:44.707011 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0753979-426d-4b7f-99ef-85610935c26f None admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:44.749772 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=100 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:33:44.946292 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd50ab7-e6ea-4059-b17e-3dc1dde38a24 None admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:44.972029 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0753979-426d-4b7f-99ef-85610935c26f None admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:45.027742 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b33cfe-a741-4226-a80d-5181d5acfefb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:45.028754 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b33cfe-a741-4226-a80d-5181d5acfefb None admin] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:45.029234 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b33cfe-a741-4226-a80d-5181d5acfefb None admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:45.030356 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b33cfe-a741-4226-a80d-5181d5acfefb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:45.032833 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16b33cfe-a741-4226-a80d-5181d5acfefb None admin] RBAC: Adding query filter params (role.id=6c99f5d9f1f243eaa134d8540f21095d, user.id=22d585550661496e956a728556c3d90d, scope.system=all) {{(pid=88113) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:33:45.033418 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16b33cfe-a741-4226-a80d-5181d5acfefb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:33:45.036393 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16b33cfe-a741-4226-a80d-5181d5acfefb None admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:33:45.054862 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a18243a5-d693-454e-871e-591691f84681 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:45.056940 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18243a5-d693-454e-871e-591691f84681 None admin] REQUEST_METHOD: `PUT` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:45.057734 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18243a5-d693-454e-871e-591691f84681 None admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:45.059384 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18243a5-d693-454e-871e-591691f84681 None admin] PATH_INFO: `/v3/system/users/12cfd2823ac141b0ac63a7c988146d85/roles/9814fca799974e3e87ffc04473b6510e` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:45.062927 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a18243a5-d693-454e-871e-591691f84681 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=12cfd2823ac141b0ac63a7c988146d85, role_id=9814fca799974e3e87ffc04473b6510e)` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:33:45.066420 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 170/333] 10.222.0.13 () {56 vars in 1308 bytes} [Wed Aug 31 09:33:44 2022] GET /identity/v3/role_assignments?user.id=22d585550661496e956a728556c3d90d&scope.system=all&role.id=6c99f5d9f1f243eaa134d8540f21095d => generated 510 bytes in 376 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:33:45.066984 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a18243a5-d693-454e-871e-591691f84681 None admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:33:45.091214 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 164/334] 10.222.0.13 () {58 vars in 1334 bytes} [Wed Aug 31 09:33:44 2022] PUT /identity/v3/system/users/12cfd2823ac141b0ac63a7c988146d85/roles/9814fca799974e3e87ffc04473b6510e => generated 0 bytes in 385 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:33:47.631993 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b96be87-1ae4-422e-bd6d-f9430902d492 None None] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:47.632875 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b96be87-1ae4-422e-bd6d-f9430902d492 None None] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:47.633017 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b96be87-1ae4-422e-bd6d-f9430902d492 None None] PATH_INFO: `/` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:47.635447 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 171/335] 10.222.0.13 () {54 vars in 927 bytes} [Wed Aug 31 09:33:47 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:33:47.642923 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06953624-3dd4-494b-8c40-7ebca6def0b1 None None] REQUEST_METHOD: `POST` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:47.643676 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06953624-3dd4-494b-8c40-7ebca6def0b1 None None] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:47.643809 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06953624-3dd4-494b-8c40-7ebca6def0b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:47.651075 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f3daeb-edf8-49ee-b267-c0916f5a4d73 None None] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:47.651561 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f3daeb-edf8-49ee-b267-c0916f5a4d73 None None] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:47.652034 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f3daeb-edf8-49ee-b267-c0916f5a4d73 None None] PATH_INFO: `/` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:47.656121 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 172/336] 10.222.0.13 () {54 vars in 927 bytes} [Wed Aug 31 09:33:47 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:33:47.665768 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7e9133-7aea-45f8-ac11-d30a356367c4 None None] REQUEST_METHOD: `POST` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:47.666253 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7e9133-7aea-45f8-ac11-d30a356367c4 None None] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:47.666731 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7e9133-7aea-45f8-ac11-d30a356367c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:47.713368 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.auth.core [None req-06953624-3dd4-494b-8c40-7ebca6def0b1 None None] MFA Rules not processed for user `fd836cf590a145df9acf19cf3ae2ac56`. Rule list: `[]` (Enabled: `True`). {{(pid=88114) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:33:47.717407 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.fernet_utils [None req-06953624-3dd4-494b-8c40-7ebca6def0b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88114) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:33:47.721112 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=100 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:33:47.738161 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.auth.core [None req-ec7e9133-7aea-45f8-ac11-d30a356367c4 None None] MFA Rules not processed for user `fd836cf590a145df9acf19cf3ae2ac56`. Rule list: `[]` (Enabled: `True`). {{(pid=88113) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:33:47.740142 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.fernet_utils [None req-ec7e9133-7aea-45f8-ac11-d30a356367c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:33:47.956974 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 173/337] 10.222.0.13 () {58 vars in 1024 bytes} [Wed Aug 31 09:33:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 293 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 09:33:47.970327 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 165/338] 10.222.0.13 () {58 vars in 1024 bytes} [Wed Aug 31 09:33:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 330 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 09:33:48.334899 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec7e9133-7aea-45f8-ac11-d30a356367c4 None None] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:48.555117 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec7e9133-7aea-45f8-ac11-d30a356367c4 None None] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:48.603736 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-463f24f5-3676-4e4b-ad4d-8c054a94c10c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:48.604320 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463f24f5-3676-4e4b-ad4d-8c054a94c10c None admin] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:48.604601 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463f24f5-3676-4e4b-ad4d-8c054a94c10c None admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:48.604878 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463f24f5-3676-4e4b-ad4d-8c054a94c10c None admin] PATH_INFO: `/v3/roles/member` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:48.612609 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=100 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:33:48.630472 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-463f24f5-3676-4e4b-ad4d-8c054a94c10c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 09:33:48.635597 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application [None req-463f24f5-3676-4e4b-ad4d-8c054a94c10c None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 31 09:33:48.635597 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:33:48.635597 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:33:48.635597 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:33:48.635597 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:33:48.635597 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:33:48.635597 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:33:48.635597 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:33:48.635597 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:33:48.635597 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:33:48.635597 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:33:48.635597 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:33:48.635597 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 09:33:48.635597 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 09:33:48.635597 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 09:33:48.635597 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 09:33:48.638640 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 09:33:48.638640 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 09:33:48.638640 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 09:33:48.638640 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 09:33:48.638640 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:33:48.638640 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:33:48.638640 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:33:48.638640 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:33:48.638640 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:33:48.638640 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:33:48.638640 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:33:48.638640 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application with Lock( Aug 31 09:33:48.638640 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:33:48.638640 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:33:48.638640 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:33:48.638640 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:33:48.642293 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:33:48.642293 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:33:48.642293 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:33:48.642293 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:33:48.642293 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 09:33:48.642293 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 09:33:48.642293 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 09:33:48.642293 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 09:33:48.642293 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 09:33:48.642293 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 09:33:48.642293 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 31 09:33:48.642293 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application Aug 31 09:33:48.642293 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 174/339] 10.222.0.13 () {56 vars in 1099 bytes} [Wed Aug 31 09:33:48 2022] GET /identity/v3/roles/member => generated 84 bytes in 303 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:33:48.649793 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06953624-3dd4-494b-8c40-7ebca6def0b1 None None] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:48.651251 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-463f24f5-3676-4e4b-ad4d-8c054a94c10c None admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:48.663037 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:33:48.781466 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-463f24f5-3676-4e4b-ad4d-8c054a94c10c None admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:48.825253 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d682870-aaf2-4830-a078-164f76b890ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:48.826150 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d682870-aaf2-4830-a078-164f76b890ef None admin] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:48.826434 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d682870-aaf2-4830-a078-164f76b890ef None admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:48.826720 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d682870-aaf2-4830-a078-164f76b890ef None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:48.840489 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d682870-aaf2-4830-a078-164f76b890ef None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 09:33:48.849204 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application [None req-8d682870-aaf2-4830-a078-164f76b890ef None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 31 09:33:48.849204 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:33:48.849204 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:33:48.849204 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:33:48.849204 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:33:48.849204 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:33:48.849204 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:33:48.849204 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:33:48.849204 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:33:48.849204 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:33:48.849204 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:33:48.849204 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:33:48.849204 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 09:33:48.849204 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 09:33:48.849204 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 09:33:48.849204 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 09:33:48.850954 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 09:33:48.850954 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 09:33:48.850954 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 09:33:48.850954 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 09:33:48.850954 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:33:48.850954 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:33:48.850954 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:33:48.850954 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:33:48.850954 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:33:48.850954 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:33:48.850954 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:33:48.850954 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application with Lock( Aug 31 09:33:48.850954 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:33:48.850954 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:33:48.850954 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:33:48.850954 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:33:48.852496 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:33:48.852496 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:33:48.852496 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:33:48.852496 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:33:48.852496 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 09:33:48.852496 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 09:33:48.852496 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 09:33:48.852496 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 09:33:48.852496 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 09:33:48.852496 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 09:33:48.852496 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 31 09:33:48.852496 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application Aug 31 09:33:48.852496 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 175/340] 10.222.0.13 () {56 vars in 1114 bytes} [Wed Aug 31 09:33:48 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 09:33:48.856707 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d682870-aaf2-4830-a078-164f76b890ef None admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:48.940481 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=100 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:33:48.951858 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06953624-3dd4-494b-8c40-7ebca6def0b1 None None] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:48.995835 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d682870-aaf2-4830-a078-164f76b890ef None admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:49.027024 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a5db12-d8fa-4fc7-bc46-00b4d096cd52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:49.027686 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a5db12-d8fa-4fc7-bc46-00b4d096cd52 None admin] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:49.027963 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a5db12-d8fa-4fc7-bc46-00b4d096cd52 None admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:49.028317 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a5db12-d8fa-4fc7-bc46-00b4d096cd52 None admin] PATH_INFO: `/v3/roles` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:49.029287 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3a5db12-d8fa-4fc7-bc46-00b4d096cd52 None admin] RBAC: Adding query filter params (name=member) {{(pid=88114) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:33:49.029622 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3a5db12-d8fa-4fc7-bc46-00b4d096cd52 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:33:49.031329 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3a5db12-d8fa-4fc7-bc46-00b4d096cd52 None admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:33:49.056033 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 166/341] 10.222.0.13 () {56 vars in 1101 bytes} [Wed Aug 31 09:33:48 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 410 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:33:49.062095 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a5db12-d8fa-4fc7-bc46-00b4d096cd52 None admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:49.091014 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c80df2-96ff-45d7-9e81-1a240d6e596d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:49.092047 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c80df2-96ff-45d7-9e81-1a240d6e596d None admin] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:49.092540 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c80df2-96ff-45d7-9e81-1a240d6e596d None admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:49.093094 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c80df2-96ff-45d7-9e81-1a240d6e596d None admin] PATH_INFO: `/v3/roles` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:49.094783 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06c80df2-96ff-45d7-9e81-1a240d6e596d None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=88113) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:33:49.095297 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06c80df2-96ff-45d7-9e81-1a240d6e596d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:33:49.097739 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06c80df2-96ff-45d7-9e81-1a240d6e596d None admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:33:49.117110 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 176/342] 10.222.0.13 () {56 vars in 1111 bytes} [Wed Aug 31 09:33:48 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 09:33:49.119443 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80c3701-c190-42f6-94cb-25378e233497 None None] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:49.119735 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80c3701-c190-42f6-94cb-25378e233497 None None] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:49.120008 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80c3701-c190-42f6-94cb-25378e233497 None None] PATH_INFO: `/` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:49.124202 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 177/343] 10.222.0.13 () {54 vars in 927 bytes} [Wed Aug 31 09:33:49 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:33:49.126305 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b80c3701-c190-42f6-94cb-25378e233497 None None] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:49.189224 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=100 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:33:49.242688 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a5db12-d8fa-4fc7-bc46-00b4d096cd52 None admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:49.288518 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d7b143-f67e-4e27-b68c-19733e22456b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:49.289144 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d7b143-f67e-4e27-b68c-19733e22456b None admin] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:49.289446 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d7b143-f67e-4e27-b68c-19733e22456b None admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:49.289718 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d7b143-f67e-4e27-b68c-19733e22456b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:49.348146 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b80c3701-c190-42f6-94cb-25378e233497 None None] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:49.385946 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=100 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:33:49.409225 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9d7b143-f67e-4e27-b68c-19733e22456b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:33:49.410924 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9d7b143-f67e-4e27-b68c-19733e22456b None admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:33:49.426527 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ea9fb7-4ea8-45fc-b9d0-7a85051605a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:49.427242 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ea9fb7-4ea8-45fc-b9d0-7a85051605a8 None admin] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:49.427713 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ea9fb7-4ea8-45fc-b9d0-7a85051605a8 None admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:49.428003 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ea9fb7-4ea8-45fc-b9d0-7a85051605a8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:49.496881 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=100 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:33:49.527863 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9ea9fb7-4ea8-45fc-b9d0-7a85051605a8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:33:49.529372 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9ea9fb7-4ea8-45fc-b9d0-7a85051605a8 None admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:33:49.543798 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 167/344] 10.222.0.13 () {58 vars in 1282 bytes} [Wed Aug 31 09:33:49 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 483 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 31 09:33:49.545126 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10aa7d29-ee0d-433b-91a9-8b3200f03506 None None] REQUEST_METHOD: `POST` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:49.549452 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10aa7d29-ee0d-433b-91a9-8b3200f03506 None None] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:49.549903 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10aa7d29-ee0d-433b-91a9-8b3200f03506 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:49.571672 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.auth.core [None req-10aa7d29-ee0d-433b-91a9-8b3200f03506 None None] MFA Rules not processed for user `fd836cf590a145df9acf19cf3ae2ac56`. Rule list: `[]` (Enabled: `True`). {{(pid=88114) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:33:49.572912 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.fernet_utils [None req-10aa7d29-ee0d-433b-91a9-8b3200f03506 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88114) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:33:49.653780 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 168/345] 10.222.0.13 () {58 vars in 1024 bytes} [Wed Aug 31 09:33:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 09:33:49.655058 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0b1989-8103-4ec4-8659-1badbb1a98e3 None None] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:49.655342 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0b1989-8103-4ec4-8659-1badbb1a98e3 None None] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:49.655610 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0b1989-8103-4ec4-8659-1badbb1a98e3 None None] PATH_INFO: `/` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:49.658044 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 169/346] 10.222.0.13 () {54 vars in 927 bytes} [Wed Aug 31 09:33:49 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:33:49.658809 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc0b1989-8103-4ec4-8659-1badbb1a98e3 None None] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:49.737553 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 178/347] 10.222.0.13 () {58 vars in 1282 bytes} [Wed Aug 31 09:33:49 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 612 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 31 09:33:49.740360 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bba29fa-1da2-4429-a1b6-d39355a35878 None None] REQUEST_METHOD: `POST` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:49.740838 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bba29fa-1da2-4429-a1b6-d39355a35878 None None] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:49.741306 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bba29fa-1da2-4429-a1b6-d39355a35878 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:49.770935 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=100 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:33:49.777997 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc0b1989-8103-4ec4-8659-1badbb1a98e3 None None] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:49.782682 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.auth.core [None req-8bba29fa-1da2-4429-a1b6-d39355a35878 None None] MFA Rules not processed for user `fd836cf590a145df9acf19cf3ae2ac56`. Rule list: `[]` (Enabled: `True`). {{(pid=88113) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:33:49.784955 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.fernet_utils [None req-8bba29fa-1da2-4429-a1b6-d39355a35878 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:33:49.812381 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=100 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:33:49.816717 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f521942e-bb89-49ef-a7c2-100678f29884 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:49.817362 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f521942e-bb89-49ef-a7c2-100678f29884 None admin] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:49.817665 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f521942e-bb89-49ef-a7c2-100678f29884 None admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:49.817933 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f521942e-bb89-49ef-a7c2-100678f29884 None admin] PATH_INFO: `/v3/projects/1001686acf834f2a8e65b3fc8369f4f6` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:49.819437 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f521942e-bb89-49ef-a7c2-100678f29884 None admin] RBAC: Authorizing `identity:get_project(project_id=1001686acf834f2a8e65b3fc8369f4f6)` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:33:49.820874 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f521942e-bb89-49ef-a7c2-100678f29884 None admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:33:49.822761 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 170/348] 10.222.0.13 () {56 vars in 1186 bytes} [Wed Aug 31 09:33:49 2022] GET /identity/v3/projects/1001686acf834f2a8e65b3fc8369f4f6 => generated 298 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:33:49.823330 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f521942e-bb89-49ef-a7c2-100678f29884 None admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:49.936943 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 179/349] 10.222.0.13 () {58 vars in 1024 bytes} [Wed Aug 31 09:33:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 09:33:49.938946 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bba29fa-1da2-4429-a1b6-d39355a35878 None None] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:49.940380 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f521942e-bb89-49ef-a7c2-100678f29884 None admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:49.978726 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b23d99-a4bb-4b7f-89b1-9d9b9c613c4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:49.979562 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b23d99-a4bb-4b7f-89b1-9d9b9c613c4d None admin] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:49.980014 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b23d99-a4bb-4b7f-89b1-9d9b9c613c4d None admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:49.980434 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b23d99-a4bb-4b7f-89b1-9d9b9c613c4d None admin] PATH_INFO: `/v3/projects/1001686acf834f2a8e65b3fc8369f4f6` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:49.981791 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50b23d99-a4bb-4b7f-89b1-9d9b9c613c4d None admin] RBAC: Authorizing `identity:get_project(project_id=1001686acf834f2a8e65b3fc8369f4f6)` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:33:49.983414 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50b23d99-a4bb-4b7f-89b1-9d9b9c613c4d None admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:33:49.985869 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 171/350] 10.222.0.13 () {56 vars in 1186 bytes} [Wed Aug 31 09:33:49 2022] GET /identity/v3/projects/1001686acf834f2a8e65b3fc8369f4f6 => generated 298 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:33:49.987232 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12024e35-0e57-477f-a2bf-085d615d4cd6 None None] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:49.987672 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12024e35-0e57-477f-a2bf-085d615d4cd6 None None] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:49.988103 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12024e35-0e57-477f-a2bf-085d615d4cd6 None None] PATH_INFO: `/` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:49.989509 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 172/351] 10.222.0.13 () {54 vars in 927 bytes} [Wed Aug 31 09:33:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:33:49.994329 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12024e35-0e57-477f-a2bf-085d615d4cd6 None None] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:50.066910 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=100 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:33:50.145158 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bba29fa-1da2-4429-a1b6-d39355a35878 None None] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:50.207994 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d19f5ab-e9ae-4218-92e5-7d79d507f369 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:50.208817 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d19f5ab-e9ae-4218-92e5-7d79d507f369 None admin] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:50.209207 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d19f5ab-e9ae-4218-92e5-7d79d507f369 None admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:50.209634 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d19f5ab-e9ae-4218-92e5-7d79d507f369 None admin] PATH_INFO: `/v3/groups/468fee363d53431790072ed027f8bb71` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:50.218627 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d19f5ab-e9ae-4218-92e5-7d79d507f369 None admin] RBAC: Authorizing `identity:get_group(group_id=468fee363d53431790072ed027f8bb71)` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:33:50.221596 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d19f5ab-e9ae-4218-92e5-7d79d507f369 None admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:33:50.226845 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 180/352] 10.222.0.13 () {56 vars in 1180 bytes} [Wed Aug 31 09:33:49 2022] GET /identity/v3/groups/468fee363d53431790072ed027f8bb71 => generated 226 bytes in 287 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:33:50.227422 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4052b2d-2321-4e84-aa82-1cab5651d9de None None] REQUEST_METHOD: `POST` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:50.227917 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4052b2d-2321-4e84-aa82-1cab5651d9de None None] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:50.228493 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4052b2d-2321-4e84-aa82-1cab5651d9de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:50.242742 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12024e35-0e57-477f-a2bf-085d615d4cd6 None None] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:50.268473 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.auth.core [None req-a4052b2d-2321-4e84-aa82-1cab5651d9de None None] MFA Rules not processed for user `fd836cf590a145df9acf19cf3ae2ac56`. Rule list: `[]` (Enabled: `True`). {{(pid=88113) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:33:50.271136 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.fernet_utils [None req-a4052b2d-2321-4e84-aa82-1cab5651d9de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:33:50.286607 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e974943-37d9-43ba-8f7f-7d1b5efad68e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:50.288393 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=100 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:33:50.288718 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e974943-37d9-43ba-8f7f-7d1b5efad68e None admin] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:50.289630 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e974943-37d9-43ba-8f7f-7d1b5efad68e None admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:50.290118 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e974943-37d9-43ba-8f7f-7d1b5efad68e None admin] PATH_INFO: `/v3/groups/468fee363d53431790072ed027f8bb71` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:50.291742 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e974943-37d9-43ba-8f7f-7d1b5efad68e None admin] RBAC: Authorizing `identity:get_group(group_id=468fee363d53431790072ed027f8bb71)` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:33:50.293985 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e974943-37d9-43ba-8f7f-7d1b5efad68e None admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:33:50.296781 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 173/353] 10.222.0.13 () {56 vars in 1180 bytes} [Wed Aug 31 09:33:49 2022] GET /identity/v3/groups/468fee363d53431790072ed027f8bb71 => generated 226 bytes in 303 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:33:50.298952 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e974943-37d9-43ba-8f7f-7d1b5efad68e None admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:50.380259 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 181/354] 10.222.0.13 () {58 vars in 1024 bytes} [Wed Aug 31 09:33:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 09:33:50.381209 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4052b2d-2321-4e84-aa82-1cab5651d9de None None] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:50.448741 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=100 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:33:50.454717 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e974943-37d9-43ba-8f7f-7d1b5efad68e None admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:50.502871 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45714d60-4013-4c73-afe4-63224685597a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:50.505242 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45714d60-4013-4c73-afe4-63224685597a None admin] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:50.505242 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45714d60-4013-4c73-afe4-63224685597a None admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:50.505242 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45714d60-4013-4c73-afe4-63224685597a None admin] PATH_INFO: `/v3/roles/member` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:50.516884 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45714d60-4013-4c73-afe4-63224685597a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 09:33:50.522610 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application [None req-45714d60-4013-4c73-afe4-63224685597a None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 31 09:33:50.522610 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:33:50.522610 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:33:50.522610 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:33:50.522610 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:33:50.522610 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:33:50.522610 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:33:50.522610 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:33:50.522610 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:33:50.522610 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:33:50.522610 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:33:50.522610 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:33:50.522610 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 09:33:50.522610 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 09:33:50.522610 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 09:33:50.522610 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 09:33:50.523519 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 09:33:50.523519 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 09:33:50.523519 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 09:33:50.523519 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 09:33:50.523519 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:33:50.523519 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:33:50.523519 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:33:50.523519 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:33:50.523519 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:33:50.523519 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:33:50.523519 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:33:50.523519 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application with Lock( Aug 31 09:33:50.523519 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:33:50.523519 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:33:50.523519 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:33:50.523519 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:33:50.524317 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:33:50.524317 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:33:50.524317 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:33:50.524317 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:33:50.524317 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 09:33:50.524317 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 09:33:50.524317 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 09:33:50.524317 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 09:33:50.524317 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 09:33:50.524317 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 09:33:50.524317 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 31 09:33:50.524317 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application Aug 31 09:33:50.524317 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 174/355] 10.222.0.13 () {56 vars in 1099 bytes} [Wed Aug 31 09:33:50 2022] GET /identity/v3/roles/member => generated 84 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:33:50.524967 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45714d60-4013-4c73-afe4-63224685597a None admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:50.528061 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4052b2d-2321-4e84-aa82-1cab5651d9de None None] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:50.577462 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc880ae-3c59-4e22-8f08-95f7c789e9a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:50.578354 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc880ae-3c59-4e22-8f08-95f7c789e9a5 None admin] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:50.579382 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc880ae-3c59-4e22-8f08-95f7c789e9a5 None admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:50.579792 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc880ae-3c59-4e22-8f08-95f7c789e9a5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:50.580902 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bc880ae-3c59-4e22-8f08-95f7c789e9a5 None admin] RBAC: Adding query filter params (group.id=468fee363d53431790072ed027f8bb71, role.id=6c99f5d9f1f243eaa134d8540f21095d, scope.project.id=1001686acf834f2a8e65b3fc8369f4f6) {{(pid=88113) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:33:50.581363 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bc880ae-3c59-4e22-8f08-95f7c789e9a5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:33:50.583252 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bc880ae-3c59-4e22-8f08-95f7c789e9a5 None admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:33:50.589964 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 182/356] 10.222.0.13 () {56 vars in 1376 bytes} [Wed Aug 31 09:33:50 2022] GET /identity/v3/role_assignments?group.id=468fee363d53431790072ed027f8bb71&scope.project.id=1001686acf834f2a8e65b3fc8369f4f6&role.id=6c99f5d9f1f243eaa134d8540f21095d => generated 260 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:33:50.590604 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc880ae-3c59-4e22-8f08-95f7c789e9a5 None admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:50.610568 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=100 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:33:50.682227 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45714d60-4013-4c73-afe4-63224685597a None admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:50.736102 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca77d7a5-9275-421d-85ec-e1a7b0a6c970 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:50.736825 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca77d7a5-9275-421d-85ec-e1a7b0a6c970 None admin] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:50.737111 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca77d7a5-9275-421d-85ec-e1a7b0a6c970 None admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:50.737413 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca77d7a5-9275-421d-85ec-e1a7b0a6c970 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:50.750495 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc880ae-3c59-4e22-8f08-95f7c789e9a5 None admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:50.751134 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca77d7a5-9275-421d-85ec-e1a7b0a6c970 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 09:33:50.756038 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application [None req-ca77d7a5-9275-421d-85ec-e1a7b0a6c970 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 31 09:33:50.756038 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:33:50.756038 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:33:50.756038 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:33:50.756038 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:33:50.756038 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:33:50.756038 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:33:50.756038 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:33:50.756038 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:33:50.756038 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:33:50.756038 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:33:50.756038 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:33:50.756038 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 09:33:50.756038 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 09:33:50.756038 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 09:33:50.756038 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 09:33:50.757057 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 09:33:50.757057 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 09:33:50.757057 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 09:33:50.757057 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 09:33:50.757057 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:33:50.757057 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:33:50.757057 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:33:50.757057 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:33:50.757057 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:33:50.757057 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:33:50.757057 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:33:50.757057 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application with Lock( Aug 31 09:33:50.757057 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:33:50.757057 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:33:50.757057 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:33:50.757057 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:33:50.758534 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:33:50.758534 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:33:50.758534 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:33:50.758534 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:33:50.758534 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 09:33:50.758534 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 09:33:50.758534 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 09:33:50.758534 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 09:33:50.758534 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 09:33:50.758534 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 09:33:50.758534 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 31 09:33:50.758534 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application Aug 31 09:33:50.758534 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 175/357] 10.222.0.13 () {56 vars in 1114 bytes} [Wed Aug 31 09:33:50 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:33:50.758534 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1717de-b25f-4ffe-8043-68aa37f5e8b4 None None] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:50.759307 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1717de-b25f-4ffe-8043-68aa37f5e8b4 None None] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:50.759307 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1717de-b25f-4ffe-8043-68aa37f5e8b4 None None] PATH_INFO: `/` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:50.761041 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 176/358] 10.222.0.13 () {54 vars in 927 bytes} [Wed Aug 31 09:33:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:33:50.761820 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a1717de-b25f-4ffe-8043-68aa37f5e8b4 None None] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:50.789712 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=100 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:33:50.800993 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84422e69-453a-4fc0-993e-34c30bf92a24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:50.801905 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84422e69-453a-4fc0-993e-34c30bf92a24 None admin] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:50.802351 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84422e69-453a-4fc0-993e-34c30bf92a24 None admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:50.802781 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84422e69-453a-4fc0-993e-34c30bf92a24 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:50.803528 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84422e69-453a-4fc0-993e-34c30bf92a24 None admin] RBAC: Adding query filter params (group.id=468fee363d53431790072ed027f8bb71, role.id=889857b056624d47b572d4a80ecc3f8e, scope.project.id=1001686acf834f2a8e65b3fc8369f4f6) {{(pid=88113) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:33:50.803840 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84422e69-453a-4fc0-993e-34c30bf92a24 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:33:50.805872 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84422e69-453a-4fc0-993e-34c30bf92a24 None admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:33:50.817414 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 183/359] 10.222.0.13 () {56 vars in 1376 bytes} [Wed Aug 31 09:33:50 2022] GET /identity/v3/role_assignments?group.id=468fee363d53431790072ed027f8bb71&scope.project.id=1001686acf834f2a8e65b3fc8369f4f6&role.id=889857b056624d47b572d4a80ecc3f8e => generated 260 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:33:50.817414 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84422e69-453a-4fc0-993e-34c30bf92a24 None admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:50.918263 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a1717de-b25f-4ffe-8043-68aa37f5e8b4 None None] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:50.941193 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=100 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:33:50.964504 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4214c3cd-29b5-4cf5-8dd4-02ac42027785 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:50.965131 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4214c3cd-29b5-4cf5-8dd4-02ac42027785 None admin] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:50.965434 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4214c3cd-29b5-4cf5-8dd4-02ac42027785 None admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:50.965715 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4214c3cd-29b5-4cf5-8dd4-02ac42027785 None admin] PATH_INFO: `/v3/roles` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:50.966402 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4214c3cd-29b5-4cf5-8dd4-02ac42027785 None admin] RBAC: Adding query filter params (name=member) {{(pid=88114) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:33:50.966706 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4214c3cd-29b5-4cf5-8dd4-02ac42027785 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:33:50.968180 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4214c3cd-29b5-4cf5-8dd4-02ac42027785 None admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:33:50.982194 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 177/360] 10.222.0.13 () {56 vars in 1101 bytes} [Wed Aug 31 09:33:50 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:33:50.983123 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4214c3cd-29b5-4cf5-8dd4-02ac42027785 None admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:50.993375 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84422e69-453a-4fc0-993e-34c30bf92a24 None admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:51.045914 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d644b36-4501-43b2-9277-490d631fa323 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:51.046910 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d644b36-4501-43b2-9277-490d631fa323 None admin] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:51.047450 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d644b36-4501-43b2-9277-490d631fa323 None admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:51.048749 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d644b36-4501-43b2-9277-490d631fa323 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:51.071125 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d644b36-4501-43b2-9277-490d631fa323 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 09:33:51.078255 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application [None req-1d644b36-4501-43b2-9277-490d631fa323 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 31 09:33:51.078255 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:33:51.078255 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:33:51.078255 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:33:51.078255 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:33:51.078255 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:33:51.078255 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:33:51.078255 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:33:51.078255 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:33:51.078255 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:33:51.078255 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:33:51.078255 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:33:51.078255 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 09:33:51.078255 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 09:33:51.078255 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 09:33:51.078255 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 09:33:51.080201 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 09:33:51.080201 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 09:33:51.080201 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 09:33:51.080201 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 09:33:51.080201 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:33:51.080201 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:33:51.080201 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:33:51.080201 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:33:51.080201 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:33:51.080201 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:33:51.080201 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:33:51.080201 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application with Lock( Aug 31 09:33:51.080201 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:33:51.080201 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:33:51.080201 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:33:51.080201 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:33:51.081183 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:33:51.081183 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:33:51.081183 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:33:51.081183 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:33:51.081183 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 09:33:51.081183 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 09:33:51.081183 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 09:33:51.081183 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 09:33:51.081183 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 09:33:51.081183 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 09:33:51.081183 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 31 09:33:51.081183 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application Aug 31 09:33:51.081183 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 184/361] 10.222.0.13 () {56 vars in 1099 bytes} [Wed Aug 31 09:33:50 2022] GET /identity/v3/roles/reader => generated 84 bytes in 268 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:33:51.086383 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70c390f-e69c-4704-b951-58a1ce7bc0fe None None] REQUEST_METHOD: `POST` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:51.089535 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70c390f-e69c-4704-b951-58a1ce7bc0fe None None] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:51.090405 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70c390f-e69c-4704-b951-58a1ce7bc0fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:51.120004 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=100 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:33:51.124444 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.auth.core [None req-b70c390f-e69c-4704-b951-58a1ce7bc0fe None None] MFA Rules not processed for user `fd836cf590a145df9acf19cf3ae2ac56`. Rule list: `[]` (Enabled: `True`). {{(pid=88113) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:33:51.127176 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.fernet_utils [None req-b70c390f-e69c-4704-b951-58a1ce7bc0fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:33:51.144284 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4214c3cd-29b5-4cf5-8dd4-02ac42027785 None admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:51.223803 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b3fb457-a804-45a5-83c0-b1f0aa407026 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:51.224840 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3fb457-a804-45a5-83c0-b1f0aa407026 None admin] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:51.225332 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3fb457-a804-45a5-83c0-b1f0aa407026 None admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:51.225902 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3fb457-a804-45a5-83c0-b1f0aa407026 None admin] PATH_INFO: `/v3/roles` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:51.227284 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b3fb457-a804-45a5-83c0-b1f0aa407026 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=88114) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:33:51.227798 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b3fb457-a804-45a5-83c0-b1f0aa407026 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:33:51.231144 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b3fb457-a804-45a5-83c0-b1f0aa407026 None admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:33:51.251220 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 178/362] 10.222.0.13 () {56 vars in 1111 bytes} [Wed Aug 31 09:33:50 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:33:51.251437 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 185/363] 10.222.0.13 () {58 vars in 1024 bytes} [Wed Aug 31 09:33:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 09:33:51.252282 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b70c390f-e69c-4704-b951-58a1ce7bc0fe None None] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:51.258230 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b3fb457-a804-45a5-83c0-b1f0aa407026 None admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:51.330955 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=100 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:33:51.388899 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b70c390f-e69c-4704-b951-58a1ce7bc0fe None None] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:51.435177 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e2ffb97-659b-4f38-92b3-c8fa77d92aeb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:51.436098 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2ffb97-659b-4f38-92b3-c8fa77d92aeb None admin] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:51.436759 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2ffb97-659b-4f38-92b3-c8fa77d92aeb None admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:51.437175 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2ffb97-659b-4f38-92b3-c8fa77d92aeb None admin] PATH_INFO: `/v3/roles` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:51.438151 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e2ffb97-659b-4f38-92b3-c8fa77d92aeb None admin] RBAC: Adding query filter params (name=reader) {{(pid=88113) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:33:51.438606 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e2ffb97-659b-4f38-92b3-c8fa77d92aeb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:33:51.440202 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e2ffb97-659b-4f38-92b3-c8fa77d92aeb None admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:33:51.452127 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 186/364] 10.222.0.13 () {56 vars in 1101 bytes} [Wed Aug 31 09:33:51 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:33:51.453303 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e2ffb97-659b-4f38-92b3-c8fa77d92aeb None admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:51.518132 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b3fb457-a804-45a5-83c0-b1f0aa407026 None admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:51.595546 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e2ffb97-659b-4f38-92b3-c8fa77d92aeb None admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:51.598161 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5458274-3620-4c2b-a529-d313f5808fa2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:51.599691 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5458274-3620-4c2b-a529-d313f5808fa2 None admin] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:51.599691 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5458274-3620-4c2b-a529-d313f5808fa2 None admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:51.599691 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5458274-3620-4c2b-a529-d313f5808fa2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:51.630902 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=100 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:33:51.649860 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b25416-33ee-4b29-be48-758bf9774d6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:51.651330 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b25416-33ee-4b29-be48-758bf9774d6c None admin] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:51.651797 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b25416-33ee-4b29-be48-758bf9774d6c None admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:51.652480 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b25416-33ee-4b29-be48-758bf9774d6c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:51.669831 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=100 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:33:51.733173 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5458274-3620-4c2b-a529-d313f5808fa2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:33:51.736398 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5458274-3620-4c2b-a529-d313f5808fa2 None admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:33:51.757293 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78b25416-33ee-4b29-be48-758bf9774d6c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:33:51.759309 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78b25416-33ee-4b29-be48-758bf9774d6c None admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:33:51.912816 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 187/365] 10.222.0.13 () {58 vars in 1282 bytes} [Wed Aug 31 09:33:51 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 460 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 31 09:33:51.914063 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b25416-33ee-4b29-be48-758bf9774d6c None admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:51.918813 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 179/366] 10.222.0.13 () {58 vars in 1282 bytes} [Wed Aug 31 09:33:51 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 667 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 31 09:33:51.919687 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5458274-3620-4c2b-a529-d313f5808fa2 None admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:51.968962 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=100 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:33:51.981898 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=100 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:33:52.062117 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5458274-3620-4c2b-a529-d313f5808fa2 None admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:52.089350 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b25416-33ee-4b29-be48-758bf9774d6c None admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:52.112043 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-066ab1b9-f384-4d51-bf21-788c65fbc397 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:52.112871 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066ab1b9-f384-4d51-bf21-788c65fbc397 None admin] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:52.113264 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066ab1b9-f384-4d51-bf21-788c65fbc397 None admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:52.113685 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066ab1b9-f384-4d51-bf21-788c65fbc397 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:52.125580 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-066ab1b9-f384-4d51-bf21-788c65fbc397 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 09:33:52.130983 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application [None req-066ab1b9-f384-4d51-bf21-788c65fbc397 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 09:33:52.130983 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:33:52.130983 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:33:52.130983 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:33:52.130983 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:33:52.130983 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:33:52.130983 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:33:52.130983 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:33:52.130983 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:33:52.130983 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:33:52.130983 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:33:52.130983 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:33:52.130983 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 09:33:52.130983 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 09:33:52.130983 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 09:33:52.130983 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 09:33:52.132011 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 09:33:52.132011 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 09:33:52.132011 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 09:33:52.132011 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 09:33:52.132011 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:33:52.132011 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:33:52.132011 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:33:52.132011 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:33:52.132011 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:33:52.132011 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:33:52.132011 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:33:52.132011 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application with Lock( Aug 31 09:33:52.132011 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:33:52.132011 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:33:52.132011 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:33:52.132011 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:33:52.132800 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:33:52.132800 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:33:52.132800 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:33:52.132800 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:33:52.132800 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 09:33:52.132800 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 09:33:52.132800 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 09:33:52.132800 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 09:33:52.132800 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 09:33:52.132800 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 09:33:52.132800 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 09:33:52.132800 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application Aug 31 09:33:52.132800 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 180/367] 10.222.0.13 () {56 vars in 1096 bytes} [Wed Aug 31 09:33:51 2022] GET /identity/v3/roles/admin => generated 83 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:33:52.133881 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-066ab1b9-f384-4d51-bf21-788c65fbc397 None admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:52.137691 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09350771-83e6-408a-9d38-cd0bba493958 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:52.138533 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09350771-83e6-408a-9d38-cd0bba493958 None admin] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:52.138882 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09350771-83e6-408a-9d38-cd0bba493958 None admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:52.139228 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09350771-83e6-408a-9d38-cd0bba493958 None admin] PATH_INFO: `/v3/users/12cfd2823ac141b0ac63a7c988146d85` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:52.142085 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09350771-83e6-408a-9d38-cd0bba493958 None admin] RBAC: Authorizing `identity:get_user(user_id=12cfd2823ac141b0ac63a7c988146d85)` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:33:52.143769 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09350771-83e6-408a-9d38-cd0bba493958 None admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:33:52.146109 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 188/368] 10.222.0.13 () {56 vars in 1177 bytes} [Wed Aug 31 09:33:51 2022] GET /identity/v3/users/12cfd2823ac141b0ac63a7c988146d85 => generated 293 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:33:52.146753 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09350771-83e6-408a-9d38-cd0bba493958 None admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:52.282065 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-066ab1b9-f384-4d51-bf21-788c65fbc397 None admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:52.285775 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=100 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:33:52.287161 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=100 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:33:52.297307 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09350771-83e6-408a-9d38-cd0bba493958 None admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:52.348168 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-502484c5-f56e-4b14-af82-01c82c090dc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:52.348843 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502484c5-f56e-4b14-af82-01c82c090dc6 None admin] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:52.349117 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502484c5-f56e-4b14-af82-01c82c090dc6 None admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:52.349409 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502484c5-f56e-4b14-af82-01c82c090dc6 None admin] PATH_INFO: `/v3/projects/5258cb5ffb9c49b1ba3cfb741aef69f8` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:52.350540 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6da172e-52a4-4554-8469-6925fa5935fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:52.350756 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-502484c5-f56e-4b14-af82-01c82c090dc6 None admin] RBAC: Authorizing `identity:get_project(project_id=5258cb5ffb9c49b1ba3cfb741aef69f8)` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:33:52.351346 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6da172e-52a4-4554-8469-6925fa5935fa None admin] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:52.351701 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6da172e-52a4-4554-8469-6925fa5935fa None admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:52.352054 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6da172e-52a4-4554-8469-6925fa5935fa None admin] PATH_INFO: `/v3/projects/5258cb5ffb9c49b1ba3cfb741aef69f8` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:52.352367 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-502484c5-f56e-4b14-af82-01c82c090dc6 None admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:33:52.353618 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6da172e-52a4-4554-8469-6925fa5935fa None admin] RBAC: Authorizing `identity:get_project(project_id=5258cb5ffb9c49b1ba3cfb741aef69f8)` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:33:52.354699 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 181/369] 10.222.0.13 () {56 vars in 1186 bytes} [Wed Aug 31 09:33:52 2022] GET /identity/v3/projects/5258cb5ffb9c49b1ba3cfb741aef69f8 => generated 302 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:33:52.355427 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6da172e-52a4-4554-8469-6925fa5935fa None admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:33:52.355616 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-502484c5-f56e-4b14-af82-01c82c090dc6 None admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:52.357686 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 189/370] 10.222.0.13 () {56 vars in 1186 bytes} [Wed Aug 31 09:33:52 2022] GET /identity/v3/projects/5258cb5ffb9c49b1ba3cfb741aef69f8 => generated 302 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:33:52.358709 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6da172e-52a4-4554-8469-6925fa5935fa None admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:52.520791 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-502484c5-f56e-4b14-af82-01c82c090dc6 None admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:52.531017 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6da172e-52a4-4554-8469-6925fa5935fa None admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:52.574623 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1790a619-36f1-4ff6-85e1-dcbf887f30a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:52.575366 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1790a619-36f1-4ff6-85e1-dcbf887f30a0 None admin] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:52.575724 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1790a619-36f1-4ff6-85e1-dcbf887f30a0 None admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:52.575999 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1790a619-36f1-4ff6-85e1-dcbf887f30a0 None admin] PATH_INFO: `/v3/roles` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:52.576753 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1790a619-36f1-4ff6-85e1-dcbf887f30a0 None admin] RBAC: Adding query filter params (name=admin) {{(pid=88114) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:33:52.577133 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1790a619-36f1-4ff6-85e1-dcbf887f30a0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:33:52.578844 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1790a619-36f1-4ff6-85e1-dcbf887f30a0 None admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:33:52.581269 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a1bfc6c-f89f-4f91-890f-503cde0866bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:52.582485 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1bfc6c-f89f-4f91-890f-503cde0866bb None admin] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:52.582761 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1bfc6c-f89f-4f91-890f-503cde0866bb None admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:52.583037 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1bfc6c-f89f-4f91-890f-503cde0866bb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:52.583792 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a1bfc6c-f89f-4f91-890f-503cde0866bb None admin] RBAC: Adding query filter params (role.id=9814fca799974e3e87ffc04473b6510e, user.id=12cfd2823ac141b0ac63a7c988146d85, scope.system=all) {{(pid=88113) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:33:52.584099 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a1bfc6c-f89f-4f91-890f-503cde0866bb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:33:52.585633 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a1bfc6c-f89f-4f91-890f-503cde0866bb None admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:33:52.596206 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 182/371] 10.222.0.13 () {56 vars in 1099 bytes} [Wed Aug 31 09:33:52 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:33:52.597123 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1790a619-36f1-4ff6-85e1-dcbf887f30a0 None admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:52.607210 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 190/372] 10.222.0.13 () {56 vars in 1308 bytes} [Wed Aug 31 09:33:52 2022] GET /identity/v3/role_assignments?user.id=12cfd2823ac141b0ac63a7c988146d85&scope.system=all&role.id=9814fca799974e3e87ffc04473b6510e => generated 510 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:33:52.608136 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a1bfc6c-f89f-4f91-890f-503cde0866bb None admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:52.646386 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=100 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:33:52.669949 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=100 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:33:52.816987 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1790a619-36f1-4ff6-85e1-dcbf887f30a0 None admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:52.832271 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a1bfc6c-f89f-4f91-890f-503cde0866bb None admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:52.885933 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c1c0b4-5501-4b73-b17f-bc6614b59a5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:52.886897 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c1c0b4-5501-4b73-b17f-bc6614b59a5c None admin] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:52.887179 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c1c0b4-5501-4b73-b17f-bc6614b59a5c None admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:52.887455 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c1c0b4-5501-4b73-b17f-bc6614b59a5c None admin] PATH_INFO: `/v3/groups/468fee363d53431790072ed027f8bb71` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:52.889516 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9c1c0b4-5501-4b73-b17f-bc6614b59a5c None admin] RBAC: Authorizing `identity:get_group(group_id=468fee363d53431790072ed027f8bb71)` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:33:52.890979 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9c1c0b4-5501-4b73-b17f-bc6614b59a5c None admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:33:52.892639 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ac7345-c511-4c60-a18e-a6a3c22b9bb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:52.893456 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 191/373] 10.222.0.13 () {56 vars in 1180 bytes} [Wed Aug 31 09:33:52 2022] GET /identity/v3/groups/468fee363d53431790072ed027f8bb71 => generated 226 bytes in 286 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:33:52.893581 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ac7345-c511-4c60-a18e-a6a3c22b9bb0 None admin] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:52.894056 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c1c0b4-5501-4b73-b17f-bc6614b59a5c None admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:52.894225 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ac7345-c511-4c60-a18e-a6a3c22b9bb0 None admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:52.894463 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ac7345-c511-4c60-a18e-a6a3c22b9bb0 None admin] PATH_INFO: `/v3/groups/468fee363d53431790072ed027f8bb71` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:52.896207 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4ac7345-c511-4c60-a18e-a6a3c22b9bb0 None admin] RBAC: Authorizing `identity:get_group(group_id=468fee363d53431790072ed027f8bb71)` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:33:52.897701 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4ac7345-c511-4c60-a18e-a6a3c22b9bb0 None admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:33:52.899690 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 183/374] 10.222.0.13 () {56 vars in 1180 bytes} [Wed Aug 31 09:33:52 2022] GET /identity/v3/groups/468fee363d53431790072ed027f8bb71 => generated 226 bytes in 303 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:33:52.902124 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2429cd60-64ed-4902-840c-57dd8bea034c None None] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:52.902409 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2429cd60-64ed-4902-840c-57dd8bea034c None None] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:52.902687 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2429cd60-64ed-4902-840c-57dd8bea034c None None] PATH_INFO: `/` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:52.903970 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 184/375] 10.222.0.13 () {54 vars in 927 bytes} [Wed Aug 31 09:33:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:33:52.904795 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2429cd60-64ed-4902-840c-57dd8bea034c None None] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:53.018423 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=100 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:33:53.040359 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c1c0b4-5501-4b73-b17f-bc6614b59a5c None admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:53.059105 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2429cd60-64ed-4902-840c-57dd8bea034c None None] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:53.061101 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=100 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:33:53.090687 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1dbfd2c-c260-43c3-bb1e-df15aff7219b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:53.091273 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1dbfd2c-c260-43c3-bb1e-df15aff7219b None admin] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:53.091487 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1dbfd2c-c260-43c3-bb1e-df15aff7219b None admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:53.091755 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1dbfd2c-c260-43c3-bb1e-df15aff7219b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:53.107286 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a14e27-098a-433a-90cb-165593474155 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:53.107885 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a14e27-098a-433a-90cb-165593474155 None admin] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:53.109177 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a14e27-098a-433a-90cb-165593474155 None admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:53.109177 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a14e27-098a-433a-90cb-165593474155 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:53.109447 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65a14e27-098a-433a-90cb-165593474155 None admin] RBAC: Adding query filter params (group.id=468fee363d53431790072ed027f8bb71, role.id=6c99f5d9f1f243eaa134d8540f21095d, scope.project.id=5258cb5ffb9c49b1ba3cfb741aef69f8) {{(pid=88114) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:33:53.109799 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65a14e27-098a-433a-90cb-165593474155 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:33:53.111556 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65a14e27-098a-433a-90cb-165593474155 None admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:33:53.120112 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 185/376] 10.222.0.13 () {56 vars in 1376 bytes} [Wed Aug 31 09:33:52 2022] GET /identity/v3/role_assignments?group.id=468fee363d53431790072ed027f8bb71&scope.project.id=5258cb5ffb9c49b1ba3cfb741aef69f8&role.id=6c99f5d9f1f243eaa134d8540f21095d => generated 260 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:33:53.120948 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a14e27-098a-433a-90cb-165593474155 None admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:53.197717 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1dbfd2c-c260-43c3-bb1e-df15aff7219b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:33:53.200944 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1dbfd2c-c260-43c3-bb1e-df15aff7219b None admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:33:53.255472 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a14e27-098a-433a-90cb-165593474155 None admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:53.302139 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e428668e-705d-44d6-82e6-e4f4e331d9fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:53.302599 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e428668e-705d-44d6-82e6-e4f4e331d9fd None admin] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:53.302876 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e428668e-705d-44d6-82e6-e4f4e331d9fd None admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:53.303145 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e428668e-705d-44d6-82e6-e4f4e331d9fd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:53.304176 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e428668e-705d-44d6-82e6-e4f4e331d9fd None admin] RBAC: Adding query filter params (group.id=468fee363d53431790072ed027f8bb71, role.id=889857b056624d47b572d4a80ecc3f8e, scope.project.id=5258cb5ffb9c49b1ba3cfb741aef69f8) {{(pid=88114) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:33:53.304481 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e428668e-705d-44d6-82e6-e4f4e331d9fd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:33:53.306101 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e428668e-705d-44d6-82e6-e4f4e331d9fd None admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:33:53.310305 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=100 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:33:53.313779 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 186/377] 10.222.0.13 () {56 vars in 1376 bytes} [Wed Aug 31 09:33:53 2022] GET /identity/v3/role_assignments?group.id=468fee363d53431790072ed027f8bb71&scope.project.id=5258cb5ffb9c49b1ba3cfb741aef69f8&role.id=889857b056624d47b572d4a80ecc3f8e => generated 260 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:33:53.315669 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590f4f96-2fb6-4d72-9242-03440dcccc94 None None] REQUEST_METHOD: `POST` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:53.316021 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590f4f96-2fb6-4d72-9242-03440dcccc94 None None] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:53.316517 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590f4f96-2fb6-4d72-9242-03440dcccc94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:53.340607 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.auth.core [None req-590f4f96-2fb6-4d72-9242-03440dcccc94 None None] MFA Rules not processed for user `fd836cf590a145df9acf19cf3ae2ac56`. Rule list: `[]` (Enabled: `True`). {{(pid=88114) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:33:53.342042 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.fernet_utils [None req-590f4f96-2fb6-4d72-9242-03440dcccc94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88114) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:33:53.342431 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 192/378] 10.222.0.13 () {58 vars in 1282 bytes} [Wed Aug 31 09:33:52 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 448 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 31 09:33:53.344870 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c29f84-56d7-4ad5-98a6-8934569a57fd None None] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:53.345645 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c29f84-56d7-4ad5-98a6-8934569a57fd None None] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:53.347095 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c29f84-56d7-4ad5-98a6-8934569a57fd None None] PATH_INFO: `/` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:53.353556 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 193/379] 10.222.0.13 () {54 vars in 927 bytes} [Wed Aug 31 09:33:53 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:33:53.354499 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c29f84-56d7-4ad5-98a6-8934569a57fd None None] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:53.377554 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=100 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:33:53.435623 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 187/380] 10.222.0.13 () {58 vars in 1024 bytes} [Wed Aug 31 09:33:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 09:33:53.437156 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a5a1f4-2bfc-4cb8-82f3-d3606bc25192 None None] REQUEST_METHOD: `POST` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:53.437668 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a5a1f4-2bfc-4cb8-82f3-d3606bc25192 None None] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:53.438220 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a5a1f4-2bfc-4cb8-82f3-d3606bc25192 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:53.467134 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.auth.core [None req-22a5a1f4-2bfc-4cb8-82f3-d3606bc25192 None None] MFA Rules not processed for user `fd836cf590a145df9acf19cf3ae2ac56`. Rule list: `[]` (Enabled: `True`). {{(pid=88114) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:33:53.468630 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.fernet_utils [None req-22a5a1f4-2bfc-4cb8-82f3-d3606bc25192 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88114) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:33:53.526747 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c29f84-56d7-4ad5-98a6-8934569a57fd None None] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:53.559553 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 188/381] 10.222.0.13 () {58 vars in 1024 bytes} [Wed Aug 31 09:33:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 09:33:53.569731 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac0408f-1464-48c3-b24e-0665f54fa035 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:53.570537 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac0408f-1464-48c3-b24e-0665f54fa035 None admin] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:53.570932 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac0408f-1464-48c3-b24e-0665f54fa035 None admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:53.571308 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac0408f-1464-48c3-b24e-0665f54fa035 None admin] PATH_INFO: `/v3/projects/3f187a2ac46b4c229b4eeead01522f05` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:53.572626 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ac0408f-1464-48c3-b24e-0665f54fa035 None admin] RBAC: Authorizing `identity:get_project(project_id=3f187a2ac46b4c229b4eeead01522f05)` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:33:53.574207 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ac0408f-1464-48c3-b24e-0665f54fa035 None admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:33:53.576154 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 194/382] 10.222.0.13 () {56 vars in 1186 bytes} [Wed Aug 31 09:33:53 2022] GET /identity/v3/projects/3f187a2ac46b4c229b4eeead01522f05 => generated 344 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:33:53.582107 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a5a1f4-2bfc-4cb8-82f3-d3606bc25192 None None] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:53.685831 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=100 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:33:53.697449 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a5a1f4-2bfc-4cb8-82f3-d3606bc25192 None None] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:53.735775 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14490a07-7fae-4169-94fb-d1b633c22190 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:53.736417 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14490a07-7fae-4169-94fb-d1b633c22190 None admin] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:53.736699 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14490a07-7fae-4169-94fb-d1b633c22190 None admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:53.736975 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14490a07-7fae-4169-94fb-d1b633c22190 None admin] PATH_INFO: `/v3/groups/fee4c6ea28cd42bcb3a4c3c16446e493` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:53.743669 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14490a07-7fae-4169-94fb-d1b633c22190 None admin] RBAC: Authorizing `identity:get_group(group_id=fee4c6ea28cd42bcb3a4c3c16446e493)` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:33:53.745132 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14490a07-7fae-4169-94fb-d1b633c22190 None admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:33:53.747180 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 189/383] 10.222.0.13 () {56 vars in 1180 bytes} [Wed Aug 31 09:33:53 2022] GET /identity/v3/groups/fee4c6ea28cd42bcb3a4c3c16446e493 => generated 229 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 09:33:53.752732 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac0408f-1464-48c3-b24e-0665f54fa035 None admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:53.817715 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=100 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:33:53.866879 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14490a07-7fae-4169-94fb-d1b633c22190 None admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:53.886666 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac0408f-1464-48c3-b24e-0665f54fa035 None admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:53.952264 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd5a4c7-617b-41e9-9dab-7a1c24778538 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:53.952796 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd5a4c7-617b-41e9-9dab-7a1c24778538 None admin] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:53.953075 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd5a4c7-617b-41e9-9dab-7a1c24778538 None admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:53.953414 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd5a4c7-617b-41e9-9dab-7a1c24778538 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:53.954361 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dd5a4c7-617b-41e9-9dab-7a1c24778538 None admin] RBAC: Adding query filter params (group.id=fee4c6ea28cd42bcb3a4c3c16446e493, role.id=5d4ee63a3add47fc9a6b8b16d39e0a1f, scope.project.id=3f187a2ac46b4c229b4eeead01522f05) {{(pid=88113) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:33:53.954686 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dd5a4c7-617b-41e9-9dab-7a1c24778538 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:33:53.956666 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dd5a4c7-617b-41e9-9dab-7a1c24778538 None admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:33:53.964553 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 195/384] 10.222.0.13 () {56 vars in 1376 bytes} [Wed Aug 31 09:33:53 2022] GET /identity/v3/role_assignments?group.id=fee4c6ea28cd42bcb3a4c3c16446e493&scope.project.id=3f187a2ac46b4c229b4eeead01522f05&role.id=5d4ee63a3add47fc9a6b8b16d39e0a1f => generated 260 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:33:53.983243 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd5a4c7-617b-41e9-9dab-7a1c24778538 None admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:54.079096 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14490a07-7fae-4169-94fb-d1b633c22190 None admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:54.149816 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-963cf793-1c3e-4c37-aa74-394c6fd20f46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:54.150467 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963cf793-1c3e-4c37-aa74-394c6fd20f46 None admin] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:54.150746 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963cf793-1c3e-4c37-aa74-394c6fd20f46 None admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:54.151015 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963cf793-1c3e-4c37-aa74-394c6fd20f46 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:54.161810 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd5a4c7-617b-41e9-9dab-7a1c24778538 None admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:54.170764 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-963cf793-1c3e-4c37-aa74-394c6fd20f46 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 09:33:54.180181 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application [None req-963cf793-1c3e-4c37-aa74-394c6fd20f46 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 31 09:33:54.180181 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:33:54.180181 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:33:54.180181 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:33:54.180181 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:33:54.180181 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:33:54.180181 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:33:54.180181 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:33:54.180181 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:33:54.180181 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:33:54.180181 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:33:54.180181 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:33:54.180181 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 09:33:54.180181 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 09:33:54.180181 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 09:33:54.180181 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 09:33:54.181107 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 09:33:54.181107 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 09:33:54.181107 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 09:33:54.181107 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 09:33:54.181107 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:33:54.181107 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:33:54.181107 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:33:54.181107 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:33:54.181107 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:33:54.181107 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:33:54.181107 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:33:54.181107 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application with Lock( Aug 31 09:33:54.181107 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:33:54.181107 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:33:54.181107 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:33:54.181107 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:33:54.181940 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:33:54.181940 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:33:54.181940 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:33:54.181940 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:33:54.181940 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 09:33:54.181940 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 09:33:54.181940 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 09:33:54.181940 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 09:33:54.181940 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 09:33:54.181940 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 09:33:54.181940 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 31 09:33:54.181940 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application Aug 31 09:33:54.181940 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 190/385] 10.222.0.13 () {56 vars in 1114 bytes} [Wed Aug 31 09:33:53 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 315 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 09:33:54.185746 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=100 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:33:54.188338 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-963cf793-1c3e-4c37-aa74-394c6fd20f46 None admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:54.208444 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=100 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:33:54.226215 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c00d3b-89ca-4b2c-93ea-75d4960344ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:54.226861 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c00d3b-89ca-4b2c-93ea-75d4960344ef None admin] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:54.227133 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c00d3b-89ca-4b2c-93ea-75d4960344ef None admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:54.227398 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c00d3b-89ca-4b2c-93ea-75d4960344ef None admin] PATH_INFO: `/v3/roles/member` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:54.256328 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2c00d3b-89ca-4b2c-93ea-75d4960344ef None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 09:33:54.261966 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application [None req-e2c00d3b-89ca-4b2c-93ea-75d4960344ef None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 31 09:33:54.261966 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:33:54.261966 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:33:54.261966 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:33:54.261966 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:33:54.261966 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:33:54.261966 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:33:54.261966 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:33:54.261966 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:33:54.261966 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:33:54.261966 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:33:54.261966 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:33:54.261966 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 09:33:54.261966 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 09:33:54.261966 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 09:33:54.261966 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 09:33:54.262884 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 09:33:54.262884 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 09:33:54.262884 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 09:33:54.262884 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 09:33:54.262884 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:33:54.262884 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:33:54.262884 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:33:54.262884 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:33:54.262884 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:33:54.262884 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:33:54.262884 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:33:54.262884 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application with Lock( Aug 31 09:33:54.262884 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:33:54.262884 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:33:54.262884 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:33:54.262884 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:33:54.264390 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:33:54.264390 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:33:54.264390 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:33:54.264390 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:33:54.264390 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 09:33:54.264390 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 09:33:54.264390 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 09:33:54.264390 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 09:33:54.264390 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 09:33:54.264390 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 09:33:54.264390 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 31 09:33:54.264390 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application Aug 31 09:33:54.264390 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 196/386] 10.222.0.13 () {56 vars in 1099 bytes} [Wed Aug 31 09:33:53 2022] GET /identity/v3/roles/member => generated 84 bytes in 281 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 09:33:54.270556 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c00d3b-89ca-4b2c-93ea-75d4960344ef None admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:54.316086 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-963cf793-1c3e-4c37-aa74-394c6fd20f46 None admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:54.357341 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd6f662e-a8be-4e70-ba09-53f08d68e6cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:54.358003 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6f662e-a8be-4e70-ba09-53f08d68e6cb None admin] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:54.362326 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6f662e-a8be-4e70-ba09-53f08d68e6cb None admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:54.362597 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6f662e-a8be-4e70-ba09-53f08d68e6cb None admin] PATH_INFO: `/v3/roles` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:54.363623 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd6f662e-a8be-4e70-ba09-53f08d68e6cb None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=88114) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:33:54.363941 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd6f662e-a8be-4e70-ba09-53f08d68e6cb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:33:54.365566 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd6f662e-a8be-4e70-ba09-53f08d68e6cb None admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:33:54.377281 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 191/387] 10.222.0.13 () {56 vars in 1111 bytes} [Wed Aug 31 09:33:54 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:33:54.384065 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd6f662e-a8be-4e70-ba09-53f08d68e6cb None admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:54.489416 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c00d3b-89ca-4b2c-93ea-75d4960344ef None admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:54.542847 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cedd7ca7-1819-45e9-a393-a6de1c89bb14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:54.543537 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cedd7ca7-1819-45e9-a393-a6de1c89bb14 None admin] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:54.543821 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cedd7ca7-1819-45e9-a393-a6de1c89bb14 None admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:54.544098 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cedd7ca7-1819-45e9-a393-a6de1c89bb14 None admin] PATH_INFO: `/v3/roles` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:54.544957 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cedd7ca7-1819-45e9-a393-a6de1c89bb14 None admin] RBAC: Adding query filter params (name=member) {{(pid=88113) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:33:54.545273 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cedd7ca7-1819-45e9-a393-a6de1c89bb14 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:33:54.546729 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cedd7ca7-1819-45e9-a393-a6de1c89bb14 None admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:33:54.556944 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 197/388] 10.222.0.13 () {56 vars in 1101 bytes} [Wed Aug 31 09:33:54 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 287 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:33:54.564550 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cedd7ca7-1819-45e9-a393-a6de1c89bb14 None admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:54.576162 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=100 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:33:54.620487 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd6f662e-a8be-4e70-ba09-53f08d68e6cb None admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:54.646588 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=100 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:33:54.691049 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e68a3ce8-8327-4915-ad75-31fc76fce070 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:54.691692 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68a3ce8-8327-4915-ad75-31fc76fce070 None admin] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:54.691963 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68a3ce8-8327-4915-ad75-31fc76fce070 None admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:54.692232 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68a3ce8-8327-4915-ad75-31fc76fce070 None admin] PATH_INFO: `/v3/groups/468fee363d53431790072ed027f8bb71` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:54.694247 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e68a3ce8-8327-4915-ad75-31fc76fce070 None admin] RBAC: Authorizing `identity:get_group(group_id=468fee363d53431790072ed027f8bb71)` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:33:54.695911 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e68a3ce8-8327-4915-ad75-31fc76fce070 None admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:33:54.700127 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 192/389] 10.222.0.13 () {56 vars in 1180 bytes} [Wed Aug 31 09:33:54 2022] GET /identity/v3/groups/468fee363d53431790072ed027f8bb71 => generated 226 bytes in 314 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:33:54.704430 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e68a3ce8-8327-4915-ad75-31fc76fce070 None admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:54.751849 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cedd7ca7-1819-45e9-a393-a6de1c89bb14 None admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:54.821731 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f2f8ae-0243-4500-bb6c-f8c7c8b78472 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:54.822616 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f2f8ae-0243-4500-bb6c-f8c7c8b78472 None admin] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:54.823058 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f2f8ae-0243-4500-bb6c-f8c7c8b78472 None admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:54.823474 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f2f8ae-0243-4500-bb6c-f8c7c8b78472 None admin] PATH_INFO: `/v3/groups/468fee363d53431790072ed027f8bb71` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:54.825716 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4f2f8ae-0243-4500-bb6c-f8c7c8b78472 None admin] RBAC: Authorizing `identity:get_group(group_id=468fee363d53431790072ed027f8bb71)` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:33:54.827396 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4f2f8ae-0243-4500-bb6c-f8c7c8b78472 None admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:33:54.831281 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 198/390] 10.222.0.13 () {56 vars in 1180 bytes} [Wed Aug 31 09:33:54 2022] GET /identity/v3/groups/468fee363d53431790072ed027f8bb71 => generated 226 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:33:54.835146 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f2f8ae-0243-4500-bb6c-f8c7c8b78472 None admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:54.905412 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e68a3ce8-8327-4915-ad75-31fc76fce070 None admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:54.967670 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6817a89b-470f-45cc-bfb8-6d1664520dcb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:54.968653 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6817a89b-470f-45cc-bfb8-6d1664520dcb None admin] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:54.969074 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6817a89b-470f-45cc-bfb8-6d1664520dcb None admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:54.970224 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6817a89b-470f-45cc-bfb8-6d1664520dcb None admin] PATH_INFO: `/v3/projects/1001686acf834f2a8e65b3fc8369f4f6` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:54.973752 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6817a89b-470f-45cc-bfb8-6d1664520dcb None admin] RBAC: Authorizing `identity:get_project(project_id=1001686acf834f2a8e65b3fc8369f4f6)` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:33:54.981553 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6817a89b-470f-45cc-bfb8-6d1664520dcb None admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:33:54.981553 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 193/391] 10.222.0.13 () {56 vars in 1186 bytes} [Wed Aug 31 09:33:54 2022] GET /identity/v3/projects/1001686acf834f2a8e65b3fc8369f4f6 => generated 298 bytes in 276 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 09:33:54.984084 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f2f8ae-0243-4500-bb6c-f8c7c8b78472 None admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:54.996194 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6817a89b-470f-45cc-bfb8-6d1664520dcb None admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:54.996774 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=100 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:33:55.030470 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=100 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:33:55.061374 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03116d22-d89a-47d3-889a-ecc6654d4720 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:55.062046 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03116d22-d89a-47d3-889a-ecc6654d4720 None admin] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:55.062321 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03116d22-d89a-47d3-889a-ecc6654d4720 None admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:55.062590 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03116d22-d89a-47d3-889a-ecc6654d4720 None admin] PATH_INFO: `/v3/projects/1001686acf834f2a8e65b3fc8369f4f6` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:55.067001 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03116d22-d89a-47d3-889a-ecc6654d4720 None admin] RBAC: Authorizing `identity:get_project(project_id=1001686acf834f2a8e65b3fc8369f4f6)` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:33:55.068458 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03116d22-d89a-47d3-889a-ecc6654d4720 None admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:33:55.079711 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 199/392] 10.222.0.13 () {56 vars in 1186 bytes} [Wed Aug 31 09:33:54 2022] GET /identity/v3/projects/1001686acf834f2a8e65b3fc8369f4f6 => generated 298 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:33:55.085027 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03116d22-d89a-47d3-889a-ecc6654d4720 None admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:55.205826 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6817a89b-470f-45cc-bfb8-6d1664520dcb None admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:55.244867 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e6ed774-5c31-457d-b3e6-01e64d9e5bcd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:55.245645 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6ed774-5c31-457d-b3e6-01e64d9e5bcd None admin] REQUEST_METHOD: `PUT` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:55.245921 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6ed774-5c31-457d-b3e6-01e64d9e5bcd None admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:55.246194 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6ed774-5c31-457d-b3e6-01e64d9e5bcd None admin] PATH_INFO: `/v3/projects/1001686acf834f2a8e65b3fc8369f4f6/groups/468fee363d53431790072ed027f8bb71/roles/889857b056624d47b572d4a80ecc3f8e` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:55.248778 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e6ed774-5c31-457d-b3e6-01e64d9e5bcd None admin] RBAC: Authorizing `identity:create_grant(project_id=1001686acf834f2a8e65b3fc8369f4f6, group_id=468fee363d53431790072ed027f8bb71, role_id=889857b056624d47b572d4a80ecc3f8e)` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:33:55.251187 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e6ed774-5c31-457d-b3e6-01e64d9e5bcd None admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:33:55.274751 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 194/393] 10.222.0.13 () {58 vars in 1442 bytes} [Wed Aug 31 09:33:54 2022] PUT /identity/v3/projects/1001686acf834f2a8e65b3fc8369f4f6/groups/468fee363d53431790072ed027f8bb71/roles/889857b056624d47b572d4a80ecc3f8e => generated 0 bytes in 279 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:33:55.305608 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03116d22-d89a-47d3-889a-ecc6654d4720 None admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:55.350623 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-492229cb-fce8-4787-9dc3-a52d7fa13371 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:55.351413 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492229cb-fce8-4787-9dc3-a52d7fa13371 None admin] REQUEST_METHOD: `PUT` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:55.351691 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492229cb-fce8-4787-9dc3-a52d7fa13371 None admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:55.351962 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492229cb-fce8-4787-9dc3-a52d7fa13371 None admin] PATH_INFO: `/v3/projects/1001686acf834f2a8e65b3fc8369f4f6/groups/468fee363d53431790072ed027f8bb71/roles/6c99f5d9f1f243eaa134d8540f21095d` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:55.354173 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-492229cb-fce8-4787-9dc3-a52d7fa13371 None admin] RBAC: Authorizing `identity:create_grant(project_id=1001686acf834f2a8e65b3fc8369f4f6, group_id=468fee363d53431790072ed027f8bb71, role_id=6c99f5d9f1f243eaa134d8540f21095d)` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:33:55.355827 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-492229cb-fce8-4787-9dc3-a52d7fa13371 None admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:33:55.373912 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 200/394] 10.222.0.13 () {58 vars in 1442 bytes} [Wed Aug 31 09:33:55 2022] PUT /identity/v3/projects/1001686acf834f2a8e65b3fc8369f4f6/groups/468fee363d53431790072ed027f8bb71/roles/6c99f5d9f1f243eaa134d8540f21095d => generated 0 bytes in 289 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 09:33:55.414814 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce9e575-f8d6-4991-bb7c-dc08cc1beeec None None] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:55.415143 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce9e575-f8d6-4991-bb7c-dc08cc1beeec None None] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:55.415377 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce9e575-f8d6-4991-bb7c-dc08cc1beeec None None] PATH_INFO: `/` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:55.417190 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 195/395] 10.222.0.13 () {54 vars in 927 bytes} [Wed Aug 31 09:33:55 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:33:55.426868 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b72164b-4714-4d29-a622-39ddac3086ac None None] REQUEST_METHOD: `POST` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:55.427187 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b72164b-4714-4d29-a622-39ddac3086ac None None] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:55.427414 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b72164b-4714-4d29-a622-39ddac3086ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:55.453648 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.auth.core [None req-9b72164b-4714-4d29-a622-39ddac3086ac None None] MFA Rules not processed for user `fd836cf590a145df9acf19cf3ae2ac56`. Rule list: `[]` (Enabled: `True`). {{(pid=88113) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:33:55.455390 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.fernet_utils [None req-9b72164b-4714-4d29-a622-39ddac3086ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:33:55.497255 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:33:55.557698 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 201/396] 10.222.0.13 () {58 vars in 1024 bytes} [Wed Aug 31 09:33:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 09:33:55.983546 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29287f7f-2307-4421-afd6-aa1c04ae3a67 None None] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:55.984070 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29287f7f-2307-4421-afd6-aa1c04ae3a67 None None] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:55.984070 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29287f7f-2307-4421-afd6-aa1c04ae3a67 None None] PATH_INFO: `/` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:55.986294 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 196/397] 10.222.0.13 () {54 vars in 927 bytes} [Wed Aug 31 09:33:55 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:33:55.991468 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b1a383-e24d-4c1f-ad6b-1eb464232042 None None] REQUEST_METHOD: `POST` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:55.992369 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b1a383-e24d-4c1f-ad6b-1eb464232042 None None] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:55.992871 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b1a383-e24d-4c1f-ad6b-1eb464232042 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:56.000561 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29287f7f-2307-4421-afd6-aa1c04ae3a67 None None] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:56.018522 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.auth.core [None req-84b1a383-e24d-4c1f-ad6b-1eb464232042 None None] MFA Rules not processed for user `fd836cf590a145df9acf19cf3ae2ac56`. Rule list: `[]` (Enabled: `True`). {{(pid=88113) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:33:56.019968 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.fernet_utils [None req-84b1a383-e24d-4c1f-ad6b-1eb464232042 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:33:56.134974 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:33:56.157261 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 202/398] 10.222.0.13 () {58 vars in 1024 bytes} [Wed Aug 31 09:33:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 09:33:56.176242 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29287f7f-2307-4421-afd6-aa1c04ae3a67 None None] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:56.228950 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d9bffe6-790d-43a4-bc6d-ca989a5407fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:56.229544 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9bffe6-790d-43a4-bc6d-ca989a5407fd None admin] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:56.229824 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9bffe6-790d-43a4-bc6d-ca989a5407fd None admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:56.230093 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9bffe6-790d-43a4-bc6d-ca989a5407fd None admin] PATH_INFO: `/v3/roles/member` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:56.247267 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d9bffe6-790d-43a4-bc6d-ca989a5407fd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 09:33:56.253400 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application [None req-5d9bffe6-790d-43a4-bc6d-ca989a5407fd None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 31 09:33:56.253400 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:33:56.253400 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:33:56.253400 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:33:56.253400 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:33:56.253400 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:33:56.253400 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:33:56.253400 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:33:56.253400 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:33:56.253400 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:33:56.253400 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:33:56.253400 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:33:56.253400 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 09:33:56.253400 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 09:33:56.253400 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 09:33:56.253400 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 09:33:56.254351 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 09:33:56.254351 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 09:33:56.254351 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 09:33:56.254351 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 09:33:56.254351 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:33:56.254351 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:33:56.254351 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:33:56.254351 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:33:56.254351 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:33:56.254351 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:33:56.254351 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:33:56.254351 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application with Lock( Aug 31 09:33:56.254351 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:33:56.254351 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:33:56.254351 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:33:56.254351 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:33:56.255839 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:33:56.255839 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:33:56.255839 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:33:56.255839 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:33:56.255839 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 09:33:56.255839 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 09:33:56.255839 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 09:33:56.255839 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 09:33:56.255839 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 09:33:56.255839 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 09:33:56.255839 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 31 09:33:56.255839 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application Aug 31 09:33:56.255839 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 197/399] 10.222.0.13 () {56 vars in 1099 bytes} [Wed Aug 31 09:33:55 2022] GET /identity/v3/roles/member => generated 84 bytes in 255 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 09:33:56.260574 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b1a383-e24d-4c1f-ad6b-1eb464232042 None None] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:56.389987 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=100 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:33:56.403484 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b1a383-e24d-4c1f-ad6b-1eb464232042 None None] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:56.449998 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-944749e6-a8d8-489e-a046-d4c0b8b448a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:56.450626 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944749e6-a8d8-489e-a046-d4c0b8b448a6 None admin] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:56.450943 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944749e6-a8d8-489e-a046-d4c0b8b448a6 None admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:56.451172 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944749e6-a8d8-489e-a046-d4c0b8b448a6 None admin] PATH_INFO: `/v3/roles` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:56.451921 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-944749e6-a8d8-489e-a046-d4c0b8b448a6 None admin] RBAC: Adding query filter params (name=member) {{(pid=88113) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:33:56.452528 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-944749e6-a8d8-489e-a046-d4c0b8b448a6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:33:56.454894 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-944749e6-a8d8-489e-a046-d4c0b8b448a6 None admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:33:56.468651 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 203/400] 10.222.0.13 () {56 vars in 1101 bytes} [Wed Aug 31 09:33:56 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:33:56.471662 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770615b5-3ed0-49ad-ad49-0f26e0d46523 None None] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:56.471942 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770615b5-3ed0-49ad-ad49-0f26e0d46523 None None] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:56.472212 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770615b5-3ed0-49ad-ad49-0f26e0d46523 None None] PATH_INFO: `/` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:56.474423 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-944749e6-a8d8-489e-a046-d4c0b8b448a6 None admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:56.475121 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 198/401] 10.222.0.13 () {54 vars in 927 bytes} [Wed Aug 31 09:33:56 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:33:56.482543 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0791ebd6-49bc-4548-b287-e7f2b0441994 None None] REQUEST_METHOD: `POST` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:56.482817 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0791ebd6-49bc-4548-b287-e7f2b0441994 None None] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:56.483475 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0791ebd6-49bc-4548-b287-e7f2b0441994 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:56.520089 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.auth.core [None req-0791ebd6-49bc-4548-b287-e7f2b0441994 None None] MFA Rules not processed for user `fd836cf590a145df9acf19cf3ae2ac56`. Rule list: `[]` (Enabled: `True`). {{(pid=88114) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:33:56.521292 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.fernet_utils [None req-0791ebd6-49bc-4548-b287-e7f2b0441994 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88114) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:33:56.633145 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 199/402] 10.222.0.13 () {58 vars in 1024 bytes} [Wed Aug 31 09:33:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 09:33:56.634149 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0791ebd6-49bc-4548-b287-e7f2b0441994 None None] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:56.689303 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-944749e6-a8d8-489e-a046-d4c0b8b448a6 None admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:56.772416 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=100 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:33:56.777759 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b41d41d7-5cf7-4ca1-b470-12ec4889d636 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:56.778303 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41d41d7-5cf7-4ca1-b470-12ec4889d636 None admin] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:56.778579 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41d41d7-5cf7-4ca1-b470-12ec4889d636 None admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:56.778848 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41d41d7-5cf7-4ca1-b470-12ec4889d636 None admin] PATH_INFO: `/v3/groups/468fee363d53431790072ed027f8bb71` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:56.786267 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b41d41d7-5cf7-4ca1-b470-12ec4889d636 None admin] RBAC: Authorizing `identity:get_group(group_id=468fee363d53431790072ed027f8bb71)` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:33:56.788184 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b41d41d7-5cf7-4ca1-b470-12ec4889d636 None admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:33:56.789855 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 204/403] 10.222.0.13 () {56 vars in 1180 bytes} [Wed Aug 31 09:33:56 2022] GET /identity/v3/groups/468fee363d53431790072ed027f8bb71 => generated 226 bytes in 316 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 09:33:56.795060 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b41d41d7-5cf7-4ca1-b470-12ec4889d636 None admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:56.845246 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=100 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:33:56.857016 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0791ebd6-49bc-4548-b287-e7f2b0441994 None None] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:56.923874 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf3543a1-1ccb-4a08-bc59-48f567eaeb78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:56.925562 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3543a1-1ccb-4a08-bc59-48f567eaeb78 None admin] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:56.926358 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3543a1-1ccb-4a08-bc59-48f567eaeb78 None admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:56.927176 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3543a1-1ccb-4a08-bc59-48f567eaeb78 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:56.945757 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf3543a1-1ccb-4a08-bc59-48f567eaeb78 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 09:33:56.950891 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application [None req-bf3543a1-1ccb-4a08-bc59-48f567eaeb78 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 31 09:33:56.950891 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:33:56.950891 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:33:56.950891 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:33:56.950891 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:33:56.950891 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:33:56.950891 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:33:56.950891 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:33:56.950891 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:33:56.950891 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:33:56.950891 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:33:56.950891 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:33:56.950891 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 09:33:56.950891 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 09:33:56.950891 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 09:33:56.950891 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 09:33:56.952625 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 09:33:56.952625 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 09:33:56.952625 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 09:33:56.952625 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 09:33:56.952625 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:33:56.952625 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:33:56.952625 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:33:56.952625 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:33:56.952625 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:33:56.952625 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:33:56.952625 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:33:56.952625 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application with Lock( Aug 31 09:33:56.952625 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:33:56.952625 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:33:56.952625 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:33:56.952625 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:33:56.954987 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:33:56.954987 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:33:56.954987 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:33:56.954987 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:33:56.954987 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 09:33:56.954987 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 09:33:56.954987 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 09:33:56.954987 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 09:33:56.954987 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 09:33:56.954987 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 09:33:56.954987 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 31 09:33:56.954987 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application Aug 31 09:33:56.957013 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 200/404] 10.222.0.13 () {56 vars in 1114 bytes} [Wed Aug 31 09:33:56 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 319 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:33:56.958124 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf3543a1-1ccb-4a08-bc59-48f567eaeb78 None admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:57.007017 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b41d41d7-5cf7-4ca1-b470-12ec4889d636 None admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:57.047369 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-139417a2-3a64-4b71-b6d9-a9351a888feb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:57.048125 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139417a2-3a64-4b71-b6d9-a9351a888feb None admin] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:57.048471 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139417a2-3a64-4b71-b6d9-a9351a888feb None admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:57.048858 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139417a2-3a64-4b71-b6d9-a9351a888feb None admin] PATH_INFO: `/v3/projects/5258cb5ffb9c49b1ba3cfb741aef69f8` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:57.050227 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-139417a2-3a64-4b71-b6d9-a9351a888feb None admin] RBAC: Authorizing `identity:get_project(project_id=5258cb5ffb9c49b1ba3cfb741aef69f8)` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:33:57.051790 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-139417a2-3a64-4b71-b6d9-a9351a888feb None admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:33:57.053586 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 205/405] 10.222.0.13 () {56 vars in 1186 bytes} [Wed Aug 31 09:33:56 2022] GET /identity/v3/projects/5258cb5ffb9c49b1ba3cfb741aef69f8 => generated 302 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 09:33:57.054431 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-139417a2-3a64-4b71-b6d9-a9351a888feb None admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:57.124933 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf3543a1-1ccb-4a08-bc59-48f567eaeb78 None admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:57.154359 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=100 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:33:57.169704 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e95fe53-a569-48cd-be3e-fa594e5a2454 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:57.170440 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e95fe53-a569-48cd-be3e-fa594e5a2454 None admin] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:57.170818 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e95fe53-a569-48cd-be3e-fa594e5a2454 None admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:57.171043 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e95fe53-a569-48cd-be3e-fa594e5a2454 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:57.183540 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e95fe53-a569-48cd-be3e-fa594e5a2454 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 09:33:57.188673 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application [None req-5e95fe53-a569-48cd-be3e-fa594e5a2454 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 09:33:57.188673 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:33:57.188673 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:33:57.188673 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:33:57.188673 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:33:57.188673 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:33:57.188673 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:33:57.188673 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:33:57.188673 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:33:57.188673 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:33:57.188673 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:33:57.188673 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:33:57.188673 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 09:33:57.188673 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 09:33:57.188673 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 09:33:57.188673 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 09:33:57.190264 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 09:33:57.190264 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 09:33:57.190264 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 09:33:57.190264 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 09:33:57.190264 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:33:57.190264 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:33:57.190264 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:33:57.190264 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:33:57.190264 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:33:57.190264 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:33:57.190264 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:33:57.190264 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application with Lock( Aug 31 09:33:57.190264 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:33:57.190264 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:33:57.190264 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:33:57.190264 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:33:57.191838 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:33:57.191838 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:33:57.191838 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:33:57.191838 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:33:57.191838 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 09:33:57.191838 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 09:33:57.191838 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 09:33:57.191838 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 09:33:57.191838 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 09:33:57.191838 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 09:33:57.191838 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 09:33:57.191838 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application Aug 31 09:33:57.191838 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 201/406] 10.222.0.13 () {56 vars in 1096 bytes} [Wed Aug 31 09:33:56 2022] GET /identity/v3/roles/admin => generated 83 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:33:57.191838 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e95fe53-a569-48cd-be3e-fa594e5a2454 None admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:57.248919 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=100 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:33:57.260813 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-139417a2-3a64-4b71-b6d9-a9351a888feb None admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:57.301638 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b3cdaf-04f9-4dde-b431-74a9dfec6263 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:57.302299 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b3cdaf-04f9-4dde-b431-74a9dfec6263 None admin] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:57.302638 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b3cdaf-04f9-4dde-b431-74a9dfec6263 None admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:57.303029 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b3cdaf-04f9-4dde-b431-74a9dfec6263 None admin] PATH_INFO: `/v3/roles` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:57.303825 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3b3cdaf-04f9-4dde-b431-74a9dfec6263 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=88113) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:33:57.304183 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3b3cdaf-04f9-4dde-b431-74a9dfec6263 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:33:57.305701 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3b3cdaf-04f9-4dde-b431-74a9dfec6263 None admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:33:57.315735 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 206/407] 10.222.0.13 () {56 vars in 1111 bytes} [Wed Aug 31 09:33:57 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:33:57.321424 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b3cdaf-04f9-4dde-b431-74a9dfec6263 None admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:57.392972 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e95fe53-a569-48cd-be3e-fa594e5a2454 None admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:57.451373 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c1a32a-a96e-412c-8964-33c997ef44fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:57.452310 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c1a32a-a96e-412c-8964-33c997ef44fd None admin] REQUEST_METHOD: `PUT` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:57.452627 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c1a32a-a96e-412c-8964-33c997ef44fd None admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:57.452961 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c1a32a-a96e-412c-8964-33c997ef44fd None admin] PATH_INFO: `/v3/projects/5258cb5ffb9c49b1ba3cfb741aef69f8/groups/468fee363d53431790072ed027f8bb71/roles/6c99f5d9f1f243eaa134d8540f21095d` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:57.455814 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02c1a32a-a96e-412c-8964-33c997ef44fd None admin] RBAC: Authorizing `identity:create_grant(project_id=5258cb5ffb9c49b1ba3cfb741aef69f8, group_id=468fee363d53431790072ed027f8bb71, role_id=6c99f5d9f1f243eaa134d8540f21095d)` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:33:57.457413 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02c1a32a-a96e-412c-8964-33c997ef44fd None admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:33:57.477441 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 202/408] 10.222.0.13 () {58 vars in 1442 bytes} [Wed Aug 31 09:33:57 2022] PUT /identity/v3/projects/5258cb5ffb9c49b1ba3cfb741aef69f8/groups/468fee363d53431790072ed027f8bb71/roles/6c99f5d9f1f243eaa134d8540f21095d => generated 0 bytes in 287 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:33:57.478594 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c1a32a-a96e-412c-8964-33c997ef44fd None admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:57.495497 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b3cdaf-04f9-4dde-b431-74a9dfec6263 None admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:57.536139 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:33:57.587840 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d76c7652-dd68-4abb-9aef-d5ceaba69dde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:57.588700 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76c7652-dd68-4abb-9aef-d5ceaba69dde None admin] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:57.589129 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76c7652-dd68-4abb-9aef-d5ceaba69dde None admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:57.589612 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76c7652-dd68-4abb-9aef-d5ceaba69dde None admin] PATH_INFO: `/v3/roles` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:57.590834 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d76c7652-dd68-4abb-9aef-d5ceaba69dde None admin] RBAC: Adding query filter params (name=admin) {{(pid=88113) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:33:57.591352 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d76c7652-dd68-4abb-9aef-d5ceaba69dde None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:33:57.602266 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d76c7652-dd68-4abb-9aef-d5ceaba69dde None admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:33:57.614151 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c1a32a-a96e-412c-8964-33c997ef44fd None admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:57.632302 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 207/409] 10.222.0.13 () {56 vars in 1099 bytes} [Wed Aug 31 09:33:57 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 311 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:33:57.634371 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d76c7652-dd68-4abb-9aef-d5ceaba69dde None admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:57.668261 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-602cbff9-4068-4e02-b4f0-6371d3e7120b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:57.669521 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602cbff9-4068-4e02-b4f0-6371d3e7120b None admin] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:57.670050 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602cbff9-4068-4e02-b4f0-6371d3e7120b None admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:57.670416 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602cbff9-4068-4e02-b4f0-6371d3e7120b None admin] PATH_INFO: `/v3/groups/468fee363d53431790072ed027f8bb71` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:57.671729 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-602cbff9-4068-4e02-b4f0-6371d3e7120b None admin] RBAC: Authorizing `identity:get_group(group_id=468fee363d53431790072ed027f8bb71)` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:33:57.675829 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-602cbff9-4068-4e02-b4f0-6371d3e7120b None admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:33:57.675829 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 203/410] 10.222.0.13 () {56 vars in 1180 bytes} [Wed Aug 31 09:33:57 2022] GET /identity/v3/groups/468fee363d53431790072ed027f8bb71 => generated 226 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:33:57.679253 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=100 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:33:57.682107 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-602cbff9-4068-4e02-b4f0-6371d3e7120b None admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:57.829340 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d76c7652-dd68-4abb-9aef-d5ceaba69dde None admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:57.835421 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-602cbff9-4068-4e02-b4f0-6371d3e7120b None admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:57.855137 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=100 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:33:57.885010 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82bdff7a-17fa-40e2-9449-875e6d0fede8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:57.885727 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82bdff7a-17fa-40e2-9449-875e6d0fede8 None admin] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:57.886067 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82bdff7a-17fa-40e2-9449-875e6d0fede8 None admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:57.886402 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82bdff7a-17fa-40e2-9449-875e6d0fede8 None admin] PATH_INFO: `/v3/projects/5258cb5ffb9c49b1ba3cfb741aef69f8` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:57.887185 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc32d1cf-a714-4003-b22b-b3aa8940e3a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:57.887655 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82bdff7a-17fa-40e2-9449-875e6d0fede8 None admin] RBAC: Authorizing `identity:get_project(project_id=5258cb5ffb9c49b1ba3cfb741aef69f8)` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:33:57.888211 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc32d1cf-a714-4003-b22b-b3aa8940e3a4 None admin] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:57.888669 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc32d1cf-a714-4003-b22b-b3aa8940e3a4 None admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:57.889129 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc32d1cf-a714-4003-b22b-b3aa8940e3a4 None admin] PATH_INFO: `/v3/groups/fee4c6ea28cd42bcb3a4c3c16446e493` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:57.889232 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82bdff7a-17fa-40e2-9449-875e6d0fede8 None admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:33:57.890596 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc32d1cf-a714-4003-b22b-b3aa8940e3a4 None admin] RBAC: Authorizing `identity:get_group(group_id=fee4c6ea28cd42bcb3a4c3c16446e493)` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:33:57.891212 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 204/411] 10.222.0.13 () {56 vars in 1186 bytes} [Wed Aug 31 09:33:57 2022] GET /identity/v3/projects/5258cb5ffb9c49b1ba3cfb741aef69f8 => generated 302 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 09:33:57.892499 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc32d1cf-a714-4003-b22b-b3aa8940e3a4 None admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:33:57.893187 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f45a562-2337-4d64-b432-5dfb3971f281 None None] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:57.893690 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f45a562-2337-4d64-b432-5dfb3971f281 None None] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:57.894163 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f45a562-2337-4d64-b432-5dfb3971f281 None None] PATH_INFO: `/` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:57.895539 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 208/412] 10.222.0.13 () {56 vars in 1180 bytes} [Wed Aug 31 09:33:57 2022] GET /identity/v3/groups/fee4c6ea28cd42bcb3a4c3c16446e493 => generated 229 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:33:57.897098 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff09fcf0-1135-406e-8633-1d72a1e0502d None None] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:57.897609 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff09fcf0-1135-406e-8633-1d72a1e0502d None None] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:57.898054 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff09fcf0-1135-406e-8633-1d72a1e0502d None None] PATH_INFO: `/` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:57.899908 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 209/413] 10.222.0.13 () {54 vars in 927 bytes} [Wed Aug 31 09:33:57 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:33:57.904697 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff09fcf0-1135-406e-8633-1d72a1e0502d None None] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:57.904805 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 205/414] 10.222.0.13 () {54 vars in 927 bytes} [Wed Aug 31 09:33:57 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:33:57.906299 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f45a562-2337-4d64-b432-5dfb3971f281 None None] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:58.028318 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=100 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:33:58.042356 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f45a562-2337-4d64-b432-5dfb3971f281 None None] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:58.052287 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff09fcf0-1135-406e-8633-1d72a1e0502d None None] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:58.085470 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd7f595-f58d-478d-814b-09a81137f935 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:58.086168 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd7f595-f58d-478d-814b-09a81137f935 None admin] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:58.086459 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd7f595-f58d-478d-814b-09a81137f935 None admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:58.086736 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd7f595-f58d-478d-814b-09a81137f935 None admin] PATH_INFO: `/v3/projects/3f187a2ac46b4c229b4eeead01522f05` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:58.087910 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfd7f595-f58d-478d-814b-09a81137f935 None admin] RBAC: Authorizing `identity:get_project(project_id=3f187a2ac46b4c229b4eeead01522f05)` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:33:58.089345 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfd7f595-f58d-478d-814b-09a81137f935 None admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:33:58.091028 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 206/415] 10.222.0.13 () {56 vars in 1186 bytes} [Wed Aug 31 09:33:57 2022] GET /identity/v3/projects/3f187a2ac46b4c229b4eeead01522f05 => generated 344 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:33:58.092164 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c279d8-b9b3-4cd7-80c1-5b9d022d5233 None None] REQUEST_METHOD: `POST` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:58.092452 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c279d8-b9b3-4cd7-80c1-5b9d022d5233 None None] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:58.092724 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c279d8-b9b3-4cd7-80c1-5b9d022d5233 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:58.106585 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db33b903-74c7-4b06-8037-1e448d7dbe3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:58.107722 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db33b903-74c7-4b06-8037-1e448d7dbe3a None admin] REQUEST_METHOD: `PUT` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:58.108222 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db33b903-74c7-4b06-8037-1e448d7dbe3a None admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:58.108798 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db33b903-74c7-4b06-8037-1e448d7dbe3a None admin] PATH_INFO: `/v3/projects/5258cb5ffb9c49b1ba3cfb741aef69f8/groups/468fee363d53431790072ed027f8bb71/roles/889857b056624d47b572d4a80ecc3f8e` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:58.111988 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db33b903-74c7-4b06-8037-1e448d7dbe3a None admin] RBAC: Authorizing `identity:create_grant(project_id=5258cb5ffb9c49b1ba3cfb741aef69f8, group_id=468fee363d53431790072ed027f8bb71, role_id=889857b056624d47b572d4a80ecc3f8e)` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:33:58.113138 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.auth.core [None req-36c279d8-b9b3-4cd7-80c1-5b9d022d5233 None None] MFA Rules not processed for user `fd836cf590a145df9acf19cf3ae2ac56`. Rule list: `[]` (Enabled: `True`). {{(pid=88114) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:33:58.114068 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db33b903-74c7-4b06-8037-1e448d7dbe3a None admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:33:58.114384 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.fernet_utils [None req-36c279d8-b9b3-4cd7-80c1-5b9d022d5233 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88114) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:33:58.141573 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 210/416] 10.222.0.13 () {58 vars in 1442 bytes} [Wed Aug 31 09:33:57 2022] PUT /identity/v3/projects/5258cb5ffb9c49b1ba3cfb741aef69f8/groups/468fee363d53431790072ed027f8bb71/roles/889857b056624d47b572d4a80ecc3f8e => generated 0 bytes in 231 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:33:58.144429 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2fd0a2-ce6d-4d69-8ff1-9ac7537ab9ce None None] REQUEST_METHOD: `POST` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:58.145142 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2fd0a2-ce6d-4d69-8ff1-9ac7537ab9ce None None] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:58.146457 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2fd0a2-ce6d-4d69-8ff1-9ac7537ab9ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:58.178293 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=100 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:33:58.182169 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.auth.core [None req-0e2fd0a2-ce6d-4d69-8ff1-9ac7537ab9ce None None] MFA Rules not processed for user `fd836cf590a145df9acf19cf3ae2ac56`. Rule list: `[]` (Enabled: `True`). {{(pid=88113) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:33:58.183781 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.fernet_utils [None req-0e2fd0a2-ce6d-4d69-8ff1-9ac7537ab9ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:33:58.249923 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 207/417] 10.222.0.13 () {58 vars in 1024 bytes} [Wed Aug 31 09:33:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 09:33:58.252570 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c279d8-b9b3-4cd7-80c1-5b9d022d5233 None None] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:58.275567 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 211/418] 10.222.0.13 () {58 vars in 1024 bytes} [Wed Aug 31 09:33:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 09:33:58.451971 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c279d8-b9b3-4cd7-80c1-5b9d022d5233 None None] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:58.522677 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-118f5c73-a45f-4f98-998f-ba916d68941c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:58.523826 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118f5c73-a45f-4f98-998f-ba916d68941c None admin] REQUEST_METHOD: `PUT` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:58.524423 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118f5c73-a45f-4f98-998f-ba916d68941c None admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:58.524900 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118f5c73-a45f-4f98-998f-ba916d68941c None admin] PATH_INFO: `/v3/projects/3f187a2ac46b4c229b4eeead01522f05/groups/fee4c6ea28cd42bcb3a4c3c16446e493/roles/5d4ee63a3add47fc9a6b8b16d39e0a1f` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:58.534973 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-118f5c73-a45f-4f98-998f-ba916d68941c None admin] RBAC: Authorizing `identity:create_grant(project_id=3f187a2ac46b4c229b4eeead01522f05, group_id=fee4c6ea28cd42bcb3a4c3c16446e493, role_id=5d4ee63a3add47fc9a6b8b16d39e0a1f)` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:33:58.537780 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-118f5c73-a45f-4f98-998f-ba916d68941c None admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:33:58.567878 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e2fd0a2-ce6d-4d69-8ff1-9ac7537ab9ce None None] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:58.575705 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 208/419] 10.222.0.13 () {58 vars in 1442 bytes} [Wed Aug 31 09:33:58 2022] PUT /identity/v3/projects/3f187a2ac46b4c229b4eeead01522f05/groups/fee4c6ea28cd42bcb3a4c3c16446e493/roles/5d4ee63a3add47fc9a6b8b16d39e0a1f => generated 0 bytes in 325 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:33:58.623300 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-118f5c73-a45f-4f98-998f-ba916d68941c None admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:58.716276 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:33:58.739545 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:33:58.780789 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e2fd0a2-ce6d-4d69-8ff1-9ac7537ab9ce None None] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:58.821086 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-118f5c73-a45f-4f98-998f-ba916d68941c None admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:58.862043 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d16df01-e76a-4162-9266-fd4432e7646c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:58.862701 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d16df01-e76a-4162-9266-fd4432e7646c None admin] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:58.863294 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d16df01-e76a-4162-9266-fd4432e7646c None admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:58.863535 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d16df01-e76a-4162-9266-fd4432e7646c None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:58.877539 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9597fe4d-abb0-40ab-b411-d2ce5d76c4f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:58.879024 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d16df01-e76a-4162-9266-fd4432e7646c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 09:33:58.879733 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9597fe4d-abb0-40ab-b411-d2ce5d76c4f6 None admin] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:58.880493 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9597fe4d-abb0-40ab-b411-d2ce5d76c4f6 None admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:58.881219 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9597fe4d-abb0-40ab-b411-d2ce5d76c4f6 None admin] PATH_INFO: `/v3/roles/member` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:58.886158 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application [None req-4d16df01-e76a-4162-9266-fd4432e7646c None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 31 09:33:58.886158 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:33:58.886158 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:33:58.886158 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:33:58.886158 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:33:58.886158 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:33:58.886158 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:33:58.886158 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:33:58.886158 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:33:58.886158 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:33:58.886158 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:33:58.886158 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:33:58.886158 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 09:33:58.886158 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 09:33:58.886158 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 09:33:58.886158 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 09:33:58.887872 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 09:33:58.887872 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 09:33:58.887872 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 09:33:58.887872 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 09:33:58.887872 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:33:58.887872 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:33:58.887872 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:33:58.887872 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:33:58.887872 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:33:58.887872 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:33:58.887872 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:33:58.887872 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application with Lock( Aug 31 09:33:58.887872 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:33:58.887872 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:33:58.887872 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:33:58.887872 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:33:58.889512 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:33:58.889512 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:33:58.889512 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:33:58.889512 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:33:58.889512 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 09:33:58.889512 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 09:33:58.889512 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 09:33:58.889512 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 09:33:58.889512 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 09:33:58.889512 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 09:33:58.889512 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 31 09:33:58.889512 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application Aug 31 09:33:58.889512 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 212/420] 10.222.0.13 () {56 vars in 1114 bytes} [Wed Aug 31 09:33:58 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 320 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 09:33:58.894031 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d16df01-e76a-4162-9266-fd4432e7646c None admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:58.914251 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9597fe4d-abb0-40ab-b411-d2ce5d76c4f6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 09:33:58.925851 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application [None req-9597fe4d-abb0-40ab-b411-d2ce5d76c4f6 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 31 09:33:58.925851 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:33:58.925851 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:33:58.925851 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:33:58.925851 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:33:58.925851 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:33:58.925851 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:33:58.925851 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:33:58.925851 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:33:58.925851 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:33:58.925851 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:33:58.925851 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:33:58.925851 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 09:33:58.925851 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 09:33:58.925851 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 09:33:58.925851 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 09:33:58.927613 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 09:33:58.927613 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 09:33:58.927613 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 09:33:58.927613 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 09:33:58.927613 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:33:58.927613 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:33:58.927613 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:33:58.927613 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:33:58.927613 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:33:58.927613 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:33:58.927613 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:33:58.927613 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application with Lock( Aug 31 09:33:58.927613 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:33:58.927613 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:33:58.927613 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:33:58.927613 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:33:58.929164 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:33:58.929164 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:33:58.929164 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:33:58.929164 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:33:58.929164 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 09:33:58.929164 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 09:33:58.929164 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 09:33:58.929164 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 09:33:58.929164 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 09:33:58.929164 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 09:33:58.929164 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 31 09:33:58.929164 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application Aug 31 09:33:58.933959 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 209/421] 10.222.0.13 () {56 vars in 1099 bytes} [Wed Aug 31 09:33:58 2022] GET /identity/v3/roles/member => generated 84 bytes in 312 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:33:58.943763 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9597fe4d-abb0-40ab-b411-d2ce5d76c4f6 None admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:59.129194 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=100 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:33:59.137753 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d16df01-e76a-4162-9266-fd4432e7646c None admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:59.143656 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9597fe4d-abb0-40ab-b411-d2ce5d76c4f6 None admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:59.210130 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7748083-c6bd-4acb-9cc9-54193b844569 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:59.212393 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7748083-c6bd-4acb-9cc9-54193b844569 None admin] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:59.213164 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7748083-c6bd-4acb-9cc9-54193b844569 None admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:59.213969 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7748083-c6bd-4acb-9cc9-54193b844569 None admin] PATH_INFO: `/v3/roles` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:59.215472 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7748083-c6bd-4acb-9cc9-54193b844569 None admin] RBAC: Adding query filter params (name=member) {{(pid=88114) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:33:59.216350 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7748083-c6bd-4acb-9cc9-54193b844569 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:33:59.220266 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7748083-c6bd-4acb-9cc9-54193b844569 None admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:33:59.230283 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ebe4816-2e3c-4cb1-b7c5-a39e409e6f27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:59.230824 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ebe4816-2e3c-4cb1-b7c5-a39e409e6f27 None admin] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:59.231098 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ebe4816-2e3c-4cb1-b7c5-a39e409e6f27 None admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:59.231368 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ebe4816-2e3c-4cb1-b7c5-a39e409e6f27 None admin] PATH_INFO: `/v3/roles` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:59.232093 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ebe4816-2e3c-4cb1-b7c5-a39e409e6f27 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=88113) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:33:59.233230 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ebe4816-2e3c-4cb1-b7c5-a39e409e6f27 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:33:59.233934 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ebe4816-2e3c-4cb1-b7c5-a39e409e6f27 None admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:33:59.236073 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=100 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:33:59.240643 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 210/422] 10.222.0.13 () {56 vars in 1101 bytes} [Wed Aug 31 09:33:58 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 298 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:33:59.250969 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7748083-c6bd-4acb-9cc9-54193b844569 None admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:59.254031 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 213/423] 10.222.0.13 () {56 vars in 1111 bytes} [Wed Aug 31 09:33:58 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 360 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 09:33:59.260549 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ebe4816-2e3c-4cb1-b7c5-a39e409e6f27 None admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:33:59.452535 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7748083-c6bd-4acb-9cc9-54193b844569 None admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:59.506682 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ebe4816-2e3c-4cb1-b7c5-a39e409e6f27 None admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:33:59.518907 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a41e13-f19c-44fb-9097-cee88a9af726 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:59.519597 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a41e13-f19c-44fb-9097-cee88a9af726 None admin] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:59.520131 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a41e13-f19c-44fb-9097-cee88a9af726 None admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:59.520405 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a41e13-f19c-44fb-9097-cee88a9af726 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:59.572165 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=100 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:33:59.581070 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee2d5d9c-f765-4e5f-9374-4efe4dcf966b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:33:59.582692 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2d5d9c-f765-4e5f-9374-4efe4dcf966b None admin] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:33:59.582968 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2d5d9c-f765-4e5f-9374-4efe4dcf966b None admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:33:59.583241 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2d5d9c-f765-4e5f-9374-4efe4dcf966b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:33:59.725252 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8a41e13-f19c-44fb-9097-cee88a9af726 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:33:59.728022 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8a41e13-f19c-44fb-9097-cee88a9af726 None admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:33:59.760263 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=100 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:33:59.786848 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee2d5d9c-f765-4e5f-9374-4efe4dcf966b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:33:59.795410 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee2d5d9c-f765-4e5f-9374-4efe4dcf966b None admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:00.087002 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 211/424] 10.222.0.13 () {58 vars in 1282 bytes} [Wed Aug 31 09:33:59 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 837 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 31 09:34:00.097526 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a41e13-f19c-44fb-9097-cee88a9af726 None admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:00.127911 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 214/425] 10.222.0.13 () {58 vars in 1282 bytes} [Wed Aug 31 09:33:59 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 868 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 31 09:34:00.154883 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee2d5d9c-f765-4e5f-9374-4efe4dcf966b None admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:00.314264 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=100 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:00.320107 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee2d5d9c-f765-4e5f-9374-4efe4dcf966b None admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:00.323956 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=100 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:00.371224 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd88344d-32ad-4d4f-aea3-769584937d4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:00.371795 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd88344d-32ad-4d4f-aea3-769584937d4b None admin] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:00.372075 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd88344d-32ad-4d4f-aea3-769584937d4b None admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:00.372414 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd88344d-32ad-4d4f-aea3-769584937d4b None admin] PATH_INFO: `/v3/projects/1001686acf834f2a8e65b3fc8369f4f6` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:00.376605 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd88344d-32ad-4d4f-aea3-769584937d4b None admin] RBAC: Authorizing `identity:get_project(project_id=1001686acf834f2a8e65b3fc8369f4f6)` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:00.379883 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd88344d-32ad-4d4f-aea3-769584937d4b None admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:00.383067 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 215/426] 10.222.0.13 () {56 vars in 1186 bytes} [Wed Aug 31 09:34:00 2022] GET /identity/v3/projects/1001686acf834f2a8e65b3fc8369f4f6 => generated 298 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:34:00.393772 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd88344d-32ad-4d4f-aea3-769584937d4b None admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:00.422215 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a41e13-f19c-44fb-9097-cee88a9af726 None admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:00.483163 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b6d016-5b28-462a-a086-38dd1122de58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:00.484164 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b6d016-5b28-462a-a086-38dd1122de58 None admin] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:00.485205 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b6d016-5b28-462a-a086-38dd1122de58 None admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:00.486646 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b6d016-5b28-462a-a086-38dd1122de58 None admin] PATH_INFO: `/v3/projects/1001686acf834f2a8e65b3fc8369f4f6` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:00.488739 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58b6d016-5b28-462a-a086-38dd1122de58 None admin] RBAC: Authorizing `identity:get_project(project_id=1001686acf834f2a8e65b3fc8369f4f6)` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:00.491258 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58b6d016-5b28-462a-a086-38dd1122de58 None admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:00.494808 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 212/427] 10.222.0.13 () {56 vars in 1186 bytes} [Wed Aug 31 09:34:00 2022] GET /identity/v3/projects/1001686acf834f2a8e65b3fc8369f4f6 => generated 298 bytes in 398 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 09:34:00.497020 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0dbe80-7c5c-4a6d-8c33-ff5da5e36424 None None] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:00.497445 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0dbe80-7c5c-4a6d-8c33-ff5da5e36424 None None] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:00.497942 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0dbe80-7c5c-4a6d-8c33-ff5da5e36424 None None] PATH_INFO: `/` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:00.502114 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 213/428] 10.222.0.13 () {54 vars in 927 bytes} [Wed Aug 31 09:34:00 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:34:00.503534 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c0dbe80-7c5c-4a6d-8c33-ff5da5e36424 None None] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:00.510345 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd88344d-32ad-4d4f-aea3-769584937d4b None admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:00.549808 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d7b387-b0b8-4b2e-9e19-72a5484f7e71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:00.550571 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d7b387-b0b8-4b2e-9e19-72a5484f7e71 None admin] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:00.550858 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d7b387-b0b8-4b2e-9e19-72a5484f7e71 None admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:00.551135 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d7b387-b0b8-4b2e-9e19-72a5484f7e71 None admin] PATH_INFO: `/v3/groups/468fee363d53431790072ed027f8bb71` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:00.552360 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5d7b387-b0b8-4b2e-9e19-72a5484f7e71 None admin] RBAC: Authorizing `identity:get_group(group_id=468fee363d53431790072ed027f8bb71)` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:00.553944 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5d7b387-b0b8-4b2e-9e19-72a5484f7e71 None admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:00.556292 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 216/429] 10.222.0.13 () {56 vars in 1180 bytes} [Wed Aug 31 09:34:00 2022] GET /identity/v3/groups/468fee363d53431790072ed027f8bb71 => generated 226 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 09:34:00.557540 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e029a699-d11d-4263-a9dd-7ff4ae2c99f2 None None] REQUEST_METHOD: `POST` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:00.557825 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e029a699-d11d-4263-a9dd-7ff4ae2c99f2 None None] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:00.558099 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e029a699-d11d-4263-a9dd-7ff4ae2c99f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:00.578871 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.auth.core [None req-e029a699-d11d-4263-a9dd-7ff4ae2c99f2 None None] MFA Rules not processed for user `fd836cf590a145df9acf19cf3ae2ac56`. Rule list: `[]` (Enabled: `True`). {{(pid=88113) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:34:00.580584 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.fernet_utils [None req-e029a699-d11d-4263-a9dd-7ff4ae2c99f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:34:00.617183 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=100 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:00.679326 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 217/430] 10.222.0.13 () {58 vars in 1024 bytes} [Wed Aug 31 09:34:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 09:34:00.681262 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e029a699-d11d-4263-a9dd-7ff4ae2c99f2 None None] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:00.696633 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c0dbe80-7c5c-4a6d-8c33-ff5da5e36424 None None] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:00.729901 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=100 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:00.742381 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc904c5-2d30-4289-975f-b17e21e5edb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:00.743175 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc904c5-2d30-4289-975f-b17e21e5edb4 None admin] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:00.743475 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc904c5-2d30-4289-975f-b17e21e5edb4 None admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:00.743788 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc904c5-2d30-4289-975f-b17e21e5edb4 None admin] PATH_INFO: `/v3/groups/468fee363d53431790072ed027f8bb71` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:00.745149 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bc904c5-2d30-4289-975f-b17e21e5edb4 None admin] RBAC: Authorizing `identity:get_group(group_id=468fee363d53431790072ed027f8bb71)` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:00.748030 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bc904c5-2d30-4289-975f-b17e21e5edb4 None admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:00.750055 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 214/431] 10.222.0.13 () {56 vars in 1180 bytes} [Wed Aug 31 09:34:00 2022] GET /identity/v3/groups/468fee363d53431790072ed027f8bb71 => generated 226 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:34:00.751904 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947296fe-0e68-4c68-98d0-67040ad0dc0e None None] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:00.752190 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947296fe-0e68-4c68-98d0-67040ad0dc0e None None] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:00.752463 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947296fe-0e68-4c68-98d0-67040ad0dc0e None None] PATH_INFO: `/` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:00.754334 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 215/432] 10.222.0.13 () {54 vars in 927 bytes} [Wed Aug 31 09:34:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:34:00.755974 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-947296fe-0e68-4c68-98d0-67040ad0dc0e None None] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:00.839180 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e029a699-d11d-4263-a9dd-7ff4ae2c99f2 None None] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:00.905242 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba4c52a0-5946-47b0-807d-e747c3a4d771 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:00.906317 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4c52a0-5946-47b0-807d-e747c3a4d771 None admin] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:00.907375 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4c52a0-5946-47b0-807d-e747c3a4d771 None admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:00.907825 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4c52a0-5946-47b0-807d-e747c3a4d771 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:00.909192 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba4c52a0-5946-47b0-807d-e747c3a4d771 None admin] RBAC: Adding query filter params (group.id=468fee363d53431790072ed027f8bb71, role.id=889857b056624d47b572d4a80ecc3f8e, scope.project.id=1001686acf834f2a8e65b3fc8369f4f6) {{(pid=88113) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:34:00.909678 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba4c52a0-5946-47b0-807d-e747c3a4d771 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:00.912234 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba4c52a0-5946-47b0-807d-e747c3a4d771 None admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:00.929528 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 218/433] 10.222.0.13 () {56 vars in 1376 bytes} [Wed Aug 31 09:34:00 2022] GET /identity/v3/role_assignments?group.id=468fee363d53431790072ed027f8bb71&scope.project.id=1001686acf834f2a8e65b3fc8369f4f6&role.id=889857b056624d47b572d4a80ecc3f8e => generated 611 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:34:00.929528 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e796ccf-67b1-42ee-88ee-7bd66f97516e None None] REQUEST_METHOD: `POST` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:00.929528 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e796ccf-67b1-42ee-88ee-7bd66f97516e None None] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:00.929528 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e796ccf-67b1-42ee-88ee-7bd66f97516e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:00.961507 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-947296fe-0e68-4c68-98d0-67040ad0dc0e None None] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:00.964314 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.auth.core [None req-2e796ccf-67b1-42ee-88ee-7bd66f97516e None None] MFA Rules not processed for user `fd836cf590a145df9acf19cf3ae2ac56`. Rule list: `[]` (Enabled: `True`). {{(pid=88113) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:34:00.966434 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.fernet_utils [None req-2e796ccf-67b1-42ee-88ee-7bd66f97516e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:34:01.017591 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c7eb955-a706-48c5-b553-82cbad1149d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:01.018333 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=100 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:01.020157 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7eb955-a706-48c5-b553-82cbad1149d8 None admin] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:01.020451 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7eb955-a706-48c5-b553-82cbad1149d8 None admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:01.020719 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7eb955-a706-48c5-b553-82cbad1149d8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:01.021587 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c7eb955-a706-48c5-b553-82cbad1149d8 None admin] RBAC: Adding query filter params (group.id=468fee363d53431790072ed027f8bb71, role.id=6c99f5d9f1f243eaa134d8540f21095d, scope.project.id=1001686acf834f2a8e65b3fc8369f4f6) {{(pid=88114) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:34:01.021867 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c7eb955-a706-48c5-b553-82cbad1149d8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:01.023646 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c7eb955-a706-48c5-b553-82cbad1149d8 None admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:01.037885 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 216/434] 10.222.0.13 () {56 vars in 1376 bytes} [Wed Aug 31 09:34:00 2022] GET /identity/v3/role_assignments?group.id=468fee363d53431790072ed027f8bb71&scope.project.id=1001686acf834f2a8e65b3fc8369f4f6&role.id=6c99f5d9f1f243eaa134d8540f21095d => generated 611 bytes in 282 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:34:01.038907 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c7eb955-a706-48c5-b553-82cbad1149d8 None admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:01.133891 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 219/435] 10.222.0.13 () {58 vars in 1024 bytes} [Wed Aug 31 09:34:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 09:34:01.138927 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff65d4f6-d83d-4266-8cf0-4cc893ed9afd None None] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:01.145981 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff65d4f6-d83d-4266-8cf0-4cc893ed9afd None None] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:01.146650 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff65d4f6-d83d-4266-8cf0-4cc893ed9afd None None] PATH_INFO: `/` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:01.148360 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 220/436] 10.222.0.13 () {54 vars in 927 bytes} [Wed Aug 31 09:34:01 2022] GET /identity => generated 270 bytes in 12 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:34:01.158887 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cfced34-e36e-4362-90ab-884c4a197ff9 None None] REQUEST_METHOD: `POST` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:01.159512 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cfced34-e36e-4362-90ab-884c4a197ff9 None None] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:01.159835 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cfced34-e36e-4362-90ab-884c4a197ff9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:01.189079 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.auth.core [None req-7cfced34-e36e-4362-90ab-884c4a197ff9 None None] MFA Rules not processed for user `fd836cf590a145df9acf19cf3ae2ac56`. Rule list: `[]` (Enabled: `True`). {{(pid=88113) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:34:01.191254 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.fernet_utils [None req-7cfced34-e36e-4362-90ab-884c4a197ff9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:34:01.201760 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=100 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:01.289792 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c7eb955-a706-48c5-b553-82cbad1149d8 None admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:01.308677 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 221/437] 10.222.0.13 () {58 vars in 1024 bytes} [Wed Aug 31 09:34:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 09:34:01.351324 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a791e813-9674-4f7f-9a2c-6f656f287f81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:01.351970 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a791e813-9674-4f7f-9a2c-6f656f287f81 None admin] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:01.352246 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a791e813-9674-4f7f-9a2c-6f656f287f81 None admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:01.352506 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a791e813-9674-4f7f-9a2c-6f656f287f81 None admin] PATH_INFO: `/v3/roles/member` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:01.364288 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a791e813-9674-4f7f-9a2c-6f656f287f81 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 09:34:01.369800 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application [None req-a791e813-9674-4f7f-9a2c-6f656f287f81 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 31 09:34:01.369800 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:34:01.369800 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:34:01.369800 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:34:01.369800 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:34:01.369800 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:34:01.369800 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:34:01.369800 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:34:01.369800 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:34:01.369800 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:34:01.369800 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:34:01.369800 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:34:01.369800 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 09:34:01.369800 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 09:34:01.369800 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 09:34:01.369800 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 09:34:01.370705 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 09:34:01.370705 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 09:34:01.370705 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 09:34:01.370705 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 09:34:01.370705 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:34:01.370705 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:34:01.370705 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:34:01.370705 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:34:01.370705 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:34:01.370705 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:34:01.370705 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:34:01.370705 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application with Lock( Aug 31 09:34:01.370705 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:34:01.370705 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:34:01.370705 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:34:01.370705 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:34:01.372104 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:34:01.372104 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:34:01.372104 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:34:01.372104 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:34:01.372104 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 09:34:01.372104 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 09:34:01.372104 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 09:34:01.372104 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 09:34:01.372104 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 09:34:01.372104 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 09:34:01.372104 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 31 09:34:01.372104 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application Aug 31 09:34:01.372104 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 217/438] 10.222.0.13 () {56 vars in 1099 bytes} [Wed Aug 31 09:34:01 2022] GET /identity/v3/roles/member => generated 84 bytes in 333 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:34:01.377183 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cfced34-e36e-4362-90ab-884c4a197ff9 None None] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:01.484125 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=100 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:01.503341 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cfced34-e36e-4362-90ab-884c4a197ff9 None None] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:01.529454 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a791e813-9674-4f7f-9a2c-6f656f287f81 None admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:01.550224 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51fdb479-eb1b-4068-855c-e6710808c859 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:01.550807 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51fdb479-eb1b-4068-855c-e6710808c859 None admin] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:01.551183 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51fdb479-eb1b-4068-855c-e6710808c859 None admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:01.551837 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51fdb479-eb1b-4068-855c-e6710808c859 None admin] PATH_INFO: `/v3/roles` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:01.552020 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51fdb479-eb1b-4068-855c-e6710808c859 None admin] RBAC: Adding query filter params (name=member) {{(pid=88113) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:34:01.552395 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51fdb479-eb1b-4068-855c-e6710808c859 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:01.553986 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51fdb479-eb1b-4068-855c-e6710808c859 None admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:01.564815 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 222/439] 10.222.0.13 () {56 vars in 1101 bytes} [Wed Aug 31 09:34:01 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 09:34:01.579407 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51fdb479-eb1b-4068-855c-e6710808c859 None admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:01.681036 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a791e813-9674-4f7f-9a2c-6f656f287f81 None admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:01.713745 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=100 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:01.717533 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51fdb479-eb1b-4068-855c-e6710808c859 None admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:01.745330 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-959fdf1a-d1a0-4201-9848-2f9e82a47e24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:01.746056 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959fdf1a-d1a0-4201-9848-2f9e82a47e24 None admin] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:01.746335 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959fdf1a-d1a0-4201-9848-2f9e82a47e24 None admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:01.746609 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959fdf1a-d1a0-4201-9848-2f9e82a47e24 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:01.770280 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-959fdf1a-d1a0-4201-9848-2f9e82a47e24 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 09:34:01.770509 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-232b3d38-3c21-460b-81a4-01eb7efbda05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:01.771288 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232b3d38-3c21-460b-81a4-01eb7efbda05 None admin] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:01.771769 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232b3d38-3c21-460b-81a4-01eb7efbda05 None admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:01.772176 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232b3d38-3c21-460b-81a4-01eb7efbda05 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:01.775900 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application [None req-959fdf1a-d1a0-4201-9848-2f9e82a47e24 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 31 09:34:01.775900 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:34:01.775900 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:34:01.775900 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:34:01.775900 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:34:01.775900 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:34:01.775900 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:34:01.775900 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:34:01.775900 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:34:01.775900 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:34:01.775900 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:34:01.775900 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:34:01.775900 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 09:34:01.775900 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 09:34:01.775900 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 09:34:01.775900 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 09:34:01.776959 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 09:34:01.776959 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 09:34:01.776959 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 09:34:01.776959 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 09:34:01.776959 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:34:01.776959 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:34:01.776959 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:34:01.776959 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:34:01.776959 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:34:01.776959 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:34:01.776959 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:34:01.776959 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application with Lock( Aug 31 09:34:01.776959 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:34:01.776959 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:34:01.776959 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:34:01.776959 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:34:01.777899 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:34:01.777899 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:34:01.777899 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:34:01.777899 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:34:01.777899 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 09:34:01.777899 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 09:34:01.777899 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 09:34:01.777899 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 09:34:01.777899 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 09:34:01.777899 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 09:34:01.777899 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 31 09:34:01.777899 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application Aug 31 09:34:01.778521 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 218/440] 10.222.0.13 () {56 vars in 1114 bytes} [Wed Aug 31 09:34:01 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 249 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 09:34:01.779108 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-959fdf1a-d1a0-4201-9848-2f9e82a47e24 None admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:01.802637 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=100 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:01.886672 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-232b3d38-3c21-460b-81a4-01eb7efbda05 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:01.888187 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-232b3d38-3c21-460b-81a4-01eb7efbda05 None admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:01.914065 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-959fdf1a-d1a0-4201-9848-2f9e82a47e24 None admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:01.955514 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7abe0636-d625-4651-abac-bb38b602b332 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:01.956174 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7abe0636-d625-4651-abac-bb38b602b332 None admin] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:01.956445 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7abe0636-d625-4651-abac-bb38b602b332 None admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:01.956713 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7abe0636-d625-4651-abac-bb38b602b332 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:01.969192 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7abe0636-d625-4651-abac-bb38b602b332 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 09:34:01.974532 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application [None req-7abe0636-d625-4651-abac-bb38b602b332 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 09:34:01.974532 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:34:01.974532 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:34:01.974532 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:34:01.974532 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:34:01.974532 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:34:01.974532 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:34:01.974532 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:34:01.974532 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:34:01.974532 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:34:01.974532 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:34:01.974532 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:34:01.974532 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 09:34:01.974532 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 09:34:01.974532 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 09:34:01.974532 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 09:34:01.975503 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 09:34:01.975503 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 09:34:01.975503 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 09:34:01.975503 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 09:34:01.975503 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:34:01.975503 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:34:01.975503 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:34:01.975503 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:34:01.975503 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:34:01.975503 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:34:01.975503 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:34:01.975503 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application with Lock( Aug 31 09:34:01.975503 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:34:01.975503 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:34:01.975503 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:34:01.975503 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:34:01.976370 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:34:01.976370 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:34:01.976370 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:34:01.976370 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:34:01.976370 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 09:34:01.976370 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 09:34:01.976370 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 09:34:01.976370 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 09:34:01.976370 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 09:34:01.976370 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 09:34:01.976370 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 09:34:01.976370 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application Aug 31 09:34:01.976370 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 219/441] 10.222.0.13 () {56 vars in 1096 bytes} [Wed Aug 31 09:34:01 2022] GET /identity/v3/roles/admin => generated 83 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:34:01.977010 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7abe0636-d625-4651-abac-bb38b602b332 None admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:02.022825 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=100 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:02.032874 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 223/442] 10.222.0.13 () {58 vars in 1282 bytes} [Wed Aug 31 09:34:01 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 448 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 31 09:34:02.032874 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-232b3d38-3c21-460b-81a4-01eb7efbda05 None admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:02.095851 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=100 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:02.096346 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7abe0636-d625-4651-abac-bb38b602b332 None admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:02.142799 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-613cbe42-8f5d-4f90-a3c1-eedd783851d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:02.143544 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613cbe42-8f5d-4f90-a3c1-eedd783851d5 None admin] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:02.143901 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613cbe42-8f5d-4f90-a3c1-eedd783851d5 None admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:02.144258 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613cbe42-8f5d-4f90-a3c1-eedd783851d5 None admin] PATH_INFO: `/v3/roles` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:02.145303 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-613cbe42-8f5d-4f90-a3c1-eedd783851d5 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=88114) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:34:02.145739 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-613cbe42-8f5d-4f90-a3c1-eedd783851d5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:02.147342 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-613cbe42-8f5d-4f90-a3c1-eedd783851d5 None admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:02.156223 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 220/443] 10.222.0.13 () {56 vars in 1111 bytes} [Wed Aug 31 09:34:01 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:34:02.157677 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-613cbe42-8f5d-4f90-a3c1-eedd783851d5 None admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:02.180016 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-232b3d38-3c21-460b-81a4-01eb7efbda05 None admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:02.225413 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d79c25a-5916-4bb3-a121-61856a89ee97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:02.226038 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d79c25a-5916-4bb3-a121-61856a89ee97 None admin] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:02.226311 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d79c25a-5916-4bb3-a121-61856a89ee97 None admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:02.226575 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d79c25a-5916-4bb3-a121-61856a89ee97 None admin] PATH_INFO: `/v3/roles` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:02.227251 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d79c25a-5916-4bb3-a121-61856a89ee97 None admin] RBAC: Adding query filter params (name=admin) {{(pid=88113) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:34:02.227545 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d79c25a-5916-4bb3-a121-61856a89ee97 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:02.228970 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d79c25a-5916-4bb3-a121-61856a89ee97 None admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:02.238461 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 224/444] 10.222.0.13 () {56 vars in 1099 bytes} [Wed Aug 31 09:34:02 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:34:02.239333 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d79c25a-5916-4bb3-a121-61856a89ee97 None admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:02.293297 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=100 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:02.296724 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-613cbe42-8f5d-4f90-a3c1-eedd783851d5 None admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:02.348937 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1acea6a7-cfae-4f73-9292-636ac206a73d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:02.350031 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1acea6a7-cfae-4f73-9292-636ac206a73d None admin] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:02.350516 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1acea6a7-cfae-4f73-9292-636ac206a73d None admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:02.351001 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1acea6a7-cfae-4f73-9292-636ac206a73d None admin] PATH_INFO: `/v3/projects/5258cb5ffb9c49b1ba3cfb741aef69f8` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:02.352994 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1acea6a7-cfae-4f73-9292-636ac206a73d None admin] RBAC: Authorizing `identity:get_project(project_id=5258cb5ffb9c49b1ba3cfb741aef69f8)` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:02.355729 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1acea6a7-cfae-4f73-9292-636ac206a73d None admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:02.359096 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 221/445] 10.222.0.13 () {56 vars in 1186 bytes} [Wed Aug 31 09:34:02 2022] GET /identity/v3/projects/5258cb5ffb9c49b1ba3cfb741aef69f8 => generated 302 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:34:02.360884 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1acea6a7-cfae-4f73-9292-636ac206a73d None admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:02.382953 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d79c25a-5916-4bb3-a121-61856a89ee97 None admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:02.394514 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=100 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:02.426145 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-471d6108-484d-41b5-a872-9d53e7c17a76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:02.426994 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471d6108-484d-41b5-a872-9d53e7c17a76 None admin] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:02.427358 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471d6108-484d-41b5-a872-9d53e7c17a76 None admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:02.427818 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471d6108-484d-41b5-a872-9d53e7c17a76 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:02.502609 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1acea6a7-cfae-4f73-9292-636ac206a73d None admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:02.510598 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-471d6108-484d-41b5-a872-9d53e7c17a76 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:02.512087 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-471d6108-484d-41b5-a872-9d53e7c17a76 None admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:02.562463 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae43c71-bb9a-4d00-a6bf-d914df390905 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:02.563222 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae43c71-bb9a-4d00-a6bf-d914df390905 None admin] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:02.563548 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae43c71-bb9a-4d00-a6bf-d914df390905 None admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:02.563825 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae43c71-bb9a-4d00-a6bf-d914df390905 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:02.609104 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=100 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:02.636741 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 225/446] 10.222.0.13 () {58 vars in 1282 bytes} [Wed Aug 31 09:34:02 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 398 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 31 09:34:02.638192 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-471d6108-484d-41b5-a872-9d53e7c17a76 None admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:02.655406 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=100 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:02.662129 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ae43c71-bb9a-4d00-a6bf-d914df390905 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:02.664004 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ae43c71-bb9a-4d00-a6bf-d914df390905 None admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:02.779386 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-471d6108-484d-41b5-a872-9d53e7c17a76 None admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:02.804557 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 222/447] 10.222.0.13 () {58 vars in 1282 bytes} [Wed Aug 31 09:34:02 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 445 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 31 09:34:02.806036 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae43c71-bb9a-4d00-a6bf-d914df390905 None admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:02.833801 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db40e36-bcc2-4f35-b532-6be6ae811230 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:02.834340 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db40e36-bcc2-4f35-b532-6be6ae811230 None admin] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:02.834616 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db40e36-bcc2-4f35-b532-6be6ae811230 None admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:02.834885 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db40e36-bcc2-4f35-b532-6be6ae811230 None admin] PATH_INFO: `/v3/groups/468fee363d53431790072ed027f8bb71` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:02.836135 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8db40e36-bcc2-4f35-b532-6be6ae811230 None admin] RBAC: Authorizing `identity:get_group(group_id=468fee363d53431790072ed027f8bb71)` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:02.837762 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8db40e36-bcc2-4f35-b532-6be6ae811230 None admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:02.845762 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 226/448] 10.222.0.13 () {56 vars in 1180 bytes} [Wed Aug 31 09:34:02 2022] GET /identity/v3/groups/468fee363d53431790072ed027f8bb71 => generated 226 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:34:02.845762 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db40e36-bcc2-4f35-b532-6be6ae811230 None admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:02.920934 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=100 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:02.943972 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=100 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:02.963643 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae43c71-bb9a-4d00-a6bf-d914df390905 None admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:02.971194 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db40e36-bcc2-4f35-b532-6be6ae811230 None admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:03.009834 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e650401-a78f-4706-bf9e-a94577560056 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:03.010600 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e650401-a78f-4706-bf9e-a94577560056 None admin] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:03.011035 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e650401-a78f-4706-bf9e-a94577560056 None admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:03.011377 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e650401-a78f-4706-bf9e-a94577560056 None admin] PATH_INFO: `/v3/projects/5258cb5ffb9c49b1ba3cfb741aef69f8` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:03.012697 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e650401-a78f-4706-bf9e-a94577560056 None admin] RBAC: Authorizing `identity:get_project(project_id=5258cb5ffb9c49b1ba3cfb741aef69f8)` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:03.014538 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e650401-a78f-4706-bf9e-a94577560056 None admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:03.017431 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 223/449] 10.222.0.13 () {56 vars in 1186 bytes} [Wed Aug 31 09:34:02 2022] GET /identity/v3/projects/5258cb5ffb9c49b1ba3cfb741aef69f8 => generated 302 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:34:03.018749 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e650401-a78f-4706-bf9e-a94577560056 None admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:03.024163 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c1a7af2-9681-441b-bd4e-c7ec799294bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:03.024967 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1a7af2-9681-441b-bd4e-c7ec799294bb None admin] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:03.025377 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1a7af2-9681-441b-bd4e-c7ec799294bb None admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:03.025751 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1a7af2-9681-441b-bd4e-c7ec799294bb None admin] PATH_INFO: `/v3/projects/3f187a2ac46b4c229b4eeead01522f05` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:03.027441 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c1a7af2-9681-441b-bd4e-c7ec799294bb None admin] RBAC: Authorizing `identity:get_project(project_id=3f187a2ac46b4c229b4eeead01522f05)` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:03.029094 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c1a7af2-9681-441b-bd4e-c7ec799294bb None admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:03.030890 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 227/450] 10.222.0.13 () {56 vars in 1186 bytes} [Wed Aug 31 09:34:02 2022] GET /identity/v3/projects/3f187a2ac46b4c229b4eeead01522f05 => generated 344 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:34:03.032741 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c1a7af2-9681-441b-bd4e-c7ec799294bb None admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:03.159125 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e650401-a78f-4706-bf9e-a94577560056 None admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:03.189126 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c1a7af2-9681-441b-bd4e-c7ec799294bb None admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:03.208175 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91da1559-6b34-44e1-948f-af915088a07c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:03.208951 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91da1559-6b34-44e1-948f-af915088a07c None admin] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:03.209338 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91da1559-6b34-44e1-948f-af915088a07c None admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:03.209727 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91da1559-6b34-44e1-948f-af915088a07c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:03.210628 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91da1559-6b34-44e1-948f-af915088a07c None admin] RBAC: Adding query filter params (group.id=468fee363d53431790072ed027f8bb71, role.id=6c99f5d9f1f243eaa134d8540f21095d, scope.project.id=5258cb5ffb9c49b1ba3cfb741aef69f8) {{(pid=88114) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:34:03.211053 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91da1559-6b34-44e1-948f-af915088a07c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:03.212984 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91da1559-6b34-44e1-948f-af915088a07c None admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:03.216688 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=100 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:03.229853 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 224/451] 10.222.0.13 () {56 vars in 1376 bytes} [Wed Aug 31 09:34:03 2022] GET /identity/v3/role_assignments?group.id=468fee363d53431790072ed027f8bb71&scope.project.id=5258cb5ffb9c49b1ba3cfb741aef69f8&role.id=6c99f5d9f1f243eaa134d8540f21095d => generated 611 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:34:03.229853 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91da1559-6b34-44e1-948f-af915088a07c None admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:03.243029 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e2e4e76-7247-4458-a760-38bf4831488d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:03.243911 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2e4e76-7247-4458-a760-38bf4831488d None admin] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:03.244290 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2e4e76-7247-4458-a760-38bf4831488d None admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:03.244604 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2e4e76-7247-4458-a760-38bf4831488d None admin] PATH_INFO: `/v3/groups/468fee363d53431790072ed027f8bb71` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:03.246015 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e2e4e76-7247-4458-a760-38bf4831488d None admin] RBAC: Authorizing `identity:get_group(group_id=468fee363d53431790072ed027f8bb71)` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:03.247621 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e2e4e76-7247-4458-a760-38bf4831488d None admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:03.250582 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 228/452] 10.222.0.13 () {56 vars in 1180 bytes} [Wed Aug 31 09:34:03 2022] GET /identity/v3/groups/468fee363d53431790072ed027f8bb71 => generated 226 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:34:03.258493 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e2e4e76-7247-4458-a760-38bf4831488d None admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:03.286654 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=100 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:03.385377 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91da1559-6b34-44e1-948f-af915088a07c None admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:03.434238 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e2e4e76-7247-4458-a760-38bf4831488d None admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:03.436952 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c69094-1b63-4b6e-9c05-990b3a301f7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:03.438045 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c69094-1b63-4b6e-9c05-990b3a301f7b None admin] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:03.438488 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c69094-1b63-4b6e-9c05-990b3a301f7b None admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:03.439708 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c69094-1b63-4b6e-9c05-990b3a301f7b None admin] PATH_INFO: `/v3/groups/fee4c6ea28cd42bcb3a4c3c16446e493` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:03.440408 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8c69094-1b63-4b6e-9c05-990b3a301f7b None admin] RBAC: Authorizing `identity:get_group(group_id=fee4c6ea28cd42bcb3a4c3c16446e493)` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:03.442952 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8c69094-1b63-4b6e-9c05-990b3a301f7b None admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:03.451594 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 225/453] 10.222.0.13 () {56 vars in 1180 bytes} [Wed Aug 31 09:34:03 2022] GET /identity/v3/groups/fee4c6ea28cd42bcb3a4c3c16446e493 => generated 229 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:34:03.453772 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c69094-1b63-4b6e-9c05-990b3a301f7b None admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:03.500129 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c5bae4-e718-414d-bcc3-2b19129be15c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:03.501875 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c5bae4-e718-414d-bcc3-2b19129be15c None admin] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:03.502432 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c5bae4-e718-414d-bcc3-2b19129be15c None admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:03.502912 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c5bae4-e718-414d-bcc3-2b19129be15c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:03.504224 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59c5bae4-e718-414d-bcc3-2b19129be15c None admin] RBAC: Adding query filter params (group.id=468fee363d53431790072ed027f8bb71, role.id=889857b056624d47b572d4a80ecc3f8e, scope.project.id=5258cb5ffb9c49b1ba3cfb741aef69f8) {{(pid=88113) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:34:03.504755 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59c5bae4-e718-414d-bcc3-2b19129be15c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:03.507489 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59c5bae4-e718-414d-bcc3-2b19129be15c None admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:03.521414 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 229/454] 10.222.0.13 () {56 vars in 1376 bytes} [Wed Aug 31 09:34:03 2022] GET /identity/v3/role_assignments?group.id=468fee363d53431790072ed027f8bb71&scope.project.id=5258cb5ffb9c49b1ba3cfb741aef69f8&role.id=889857b056624d47b572d4a80ecc3f8e => generated 611 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 09:34:03.572216 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=100 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:03.631391 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c69094-1b63-4b6e-9c05-990b3a301f7b None admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:03.702009 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c9624b6-22ea-4887-9112-8f8bbcad7f2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:03.702948 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9624b6-22ea-4887-9112-8f8bbcad7f2a None admin] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:03.703466 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9624b6-22ea-4887-9112-8f8bbcad7f2a None admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:03.703937 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9624b6-22ea-4887-9112-8f8bbcad7f2a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:03.705309 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c9624b6-22ea-4887-9112-8f8bbcad7f2a None admin] RBAC: Adding query filter params (group.id=fee4c6ea28cd42bcb3a4c3c16446e493, role.id=5d4ee63a3add47fc9a6b8b16d39e0a1f, scope.project.id=3f187a2ac46b4c229b4eeead01522f05) {{(pid=88114) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:34:03.705901 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c9624b6-22ea-4887-9112-8f8bbcad7f2a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:03.708666 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c9624b6-22ea-4887-9112-8f8bbcad7f2a None admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:03.721710 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 226/455] 10.222.0.13 () {56 vars in 1376 bytes} [Wed Aug 31 09:34:03 2022] GET /identity/v3/role_assignments?group.id=fee4c6ea28cd42bcb3a4c3c16446e493&scope.project.id=3f187a2ac46b4c229b4eeead01522f05&role.id=5d4ee63a3add47fc9a6b8b16d39e0a1f => generated 611 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 09:34:06.232361 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629616cb-1ac1-4af8-abe4-edd955a7eb63 None None] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:06.232849 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629616cb-1ac1-4af8-abe4-edd955a7eb63 None None] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:06.233205 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629616cb-1ac1-4af8-abe4-edd955a7eb63 None None] PATH_INFO: `/` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:06.236528 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 230/456] 10.222.0.13 () {54 vars in 927 bytes} [Wed Aug 31 09:34:06 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:34:06.243430 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453e04e3-c736-4d2f-b1b4-a8a6ab2d66b0 None None] REQUEST_METHOD: `POST` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:06.243930 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453e04e3-c736-4d2f-b1b4-a8a6ab2d66b0 None None] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:06.244377 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453e04e3-c736-4d2f-b1b4-a8a6ab2d66b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:06.286153 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.auth.core [None req-453e04e3-c736-4d2f-b1b4-a8a6ab2d66b0 None None] MFA Rules not processed for user `fd836cf590a145df9acf19cf3ae2ac56`. Rule list: `[]` (Enabled: `True`). {{(pid=88114) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:34:06.288175 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.fernet_utils [None req-453e04e3-c736-4d2f-b1b4-a8a6ab2d66b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88114) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:34:06.452601 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 227/457] 10.222.0.13 () {58 vars in 1024 bytes} [Wed Aug 31 09:34:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 09:34:06.963060 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-629616cb-1ac1-4af8-abe4-edd955a7eb63 None None] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:07.079963 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=100 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:07.114992 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-629616cb-1ac1-4af8-abe4-edd955a7eb63 None None] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:07.161338 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-989066b7-03b9-49e3-bd02-232ff8a7a56c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:07.162639 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989066b7-03b9-49e3-bd02-232ff8a7a56c None admin] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:07.163383 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989066b7-03b9-49e3-bd02-232ff8a7a56c None admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:07.164286 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989066b7-03b9-49e3-bd02-232ff8a7a56c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:07.190931 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-989066b7-03b9-49e3-bd02-232ff8a7a56c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:07.192750 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-989066b7-03b9-49e3-bd02-232ff8a7a56c None admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:07.215437 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application [None req-989066b7-03b9-49e3-bd02-232ff8a7a56c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 09:34:07.215437 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:34:07.215437 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:34:07.215437 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:34:07.215437 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:34:07.215437 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:34:07.215437 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:34:07.215437 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:34:07.215437 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:34:07.215437 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:34:07.215437 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:34:07.215437 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:34:07.215437 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 09:34:07.215437 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 09:34:07.215437 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 09:34:07.215437 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 09:34:07.216396 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:34:07.216396 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:34:07.216396 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:34:07.216396 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:34:07.216396 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:34:07.216396 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:34:07.216396 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:34:07.216396 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application with Lock( Aug 31 09:34:07.216396 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:34:07.216396 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:34:07.216396 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:34:07.216396 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:34:07.216396 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:34:07.216396 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:34:07.216396 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:34:07.216396 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:34:07.216396 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 09:34:07.217248 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 09:34:07.217248 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 09:34:07.217248 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application Aug 31 09:34:07.219320 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 231/458] 10.222.0.13 () {56 vars in 1108 bytes} [Wed Aug 31 09:34:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 257 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:34:07.231191 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-453e04e3-c736-4d2f-b1b4-a8a6ab2d66b0 None None] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:07.385950 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=100 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:07.450428 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-453e04e3-c736-4d2f-b1b4-a8a6ab2d66b0 None None] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:07.523462 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75fc6392-1385-471d-a974-dfa3cb3e99f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:07.524426 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75fc6392-1385-471d-a974-dfa3cb3e99f5 None admin] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:07.524899 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75fc6392-1385-471d-a974-dfa3cb3e99f5 None admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:07.525360 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75fc6392-1385-471d-a974-dfa3cb3e99f5 None admin] PATH_INFO: `/v3/domains` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:07.526556 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75fc6392-1385-471d-a974-dfa3cb3e99f5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88114) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:34:07.527074 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75fc6392-1385-471d-a974-dfa3cb3e99f5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:07.529646 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75fc6392-1385-471d-a974-dfa3cb3e99f5 None admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:07.564746 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 228/459] 10.222.0.13 () {56 vars in 1109 bytes} [Wed Aug 31 09:34:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 335 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:34:07.573589 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-989066b7-03b9-49e3-bd02-232ff8a7a56c None admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:07.681754 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d40fcb-730b-4ce7-b8ed-026e1b9accad None None] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:07.682240 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d40fcb-730b-4ce7-b8ed-026e1b9accad None None] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:07.682703 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d40fcb-730b-4ce7-b8ed-026e1b9accad None None] PATH_INFO: `/` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:07.685276 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 229/460] 10.222.0.13 () {54 vars in 927 bytes} [Wed Aug 31 09:34:07 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:34:07.693222 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235bdc23-302d-485f-af22-8099913ff1a0 None None] REQUEST_METHOD: `POST` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:07.694096 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235bdc23-302d-485f-af22-8099913ff1a0 None None] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:07.694734 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235bdc23-302d-485f-af22-8099913ff1a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:07.748109 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.auth.core [None req-235bdc23-302d-485f-af22-8099913ff1a0 None None] MFA Rules not processed for user `fd836cf590a145df9acf19cf3ae2ac56`. Rule list: `[]` (Enabled: `True`). {{(pid=88114) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:34:07.751513 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.fernet_utils [None req-235bdc23-302d-485f-af22-8099913ff1a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88114) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:34:07.868343 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-989066b7-03b9-49e3-bd02-232ff8a7a56c None admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:07.904526 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=100 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:07.911978 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff7eeebf-ad16-4c4d-8474-1f941dd6824a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:07.913026 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7eeebf-ad16-4c4d-8474-1f941dd6824a None admin] REQUEST_METHOD: `POST` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:07.913552 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7eeebf-ad16-4c4d-8474-1f941dd6824a None admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:07.914031 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7eeebf-ad16-4c4d-8474-1f941dd6824a None admin] PATH_INFO: `/v3/users` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:07.914923 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff7eeebf-ad16-4c4d-8474-1f941dd6824a None admin] RBAC: Authorizing `identity:create_user()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:07.916654 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff7eeebf-ad16-4c4d-8474-1f941dd6824a None admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:07.925109 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 230/461] 10.222.0.13 () {58 vars in 1024 bytes} [Wed Aug 31 09:34:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 233 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 31 09:34:07.927860 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0362114c-70fa-4c90-9fe8-0f553d8d4778 None None] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:07.928574 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0362114c-70fa-4c90-9fe8-0f553d8d4778 None None] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:07.929228 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0362114c-70fa-4c90-9fe8-0f553d8d4778 None None] PATH_INFO: `/` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:07.937613 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 231/462] 10.222.0.13 () {54 vars in 927 bytes} [Wed Aug 31 09:34:07 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:34:07.942193 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda46638-b067-45b8-8e48-eed55b6c67c0 None None] REQUEST_METHOD: `POST` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:07.943022 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda46638-b067-45b8-8e48-eed55b6c67c0 None None] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:07.943709 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda46638-b067-45b8-8e48-eed55b6c67c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:07.960074 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 232/463] 10.222.0.13 () {60 vars in 1132 bytes} [Wed Aug 31 09:34:07 2022] POST /identity/v3/users => generated 246 bytes in 388 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 09:34:07.977691 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.auth.core [None req-cda46638-b067-45b8-8e48-eed55b6c67c0 None None] MFA Rules not processed for user `fd836cf590a145df9acf19cf3ae2ac56`. Rule list: `[]` (Enabled: `True`). {{(pid=88114) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:34:07.979934 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.fernet_utils [None req-cda46638-b067-45b8-8e48-eed55b6c67c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88114) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:34:08.071556 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=100 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:08.110190 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 232/464] 10.222.0.13 () {58 vars in 1024 bytes} [Wed Aug 31 09:34:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 09:34:08.160239 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3b14f8-eb5a-4834-a4ea-6c47a195e895 None None] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:08.160606 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3b14f8-eb5a-4834-a4ea-6c47a195e895 None None] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:08.160990 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3b14f8-eb5a-4834-a4ea-6c47a195e895 None None] PATH_INFO: `/` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:08.163013 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 233/465] 10.222.0.13 () {54 vars in 927 bytes} [Wed Aug 31 09:34:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:34:08.168709 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96aee47-e2d4-4682-add2-fe3a4b0421c6 None None] REQUEST_METHOD: `POST` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:08.169269 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96aee47-e2d4-4682-add2-fe3a4b0421c6 None None] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:08.169704 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96aee47-e2d4-4682-add2-fe3a4b0421c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:08.211274 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.auth.core [None req-f96aee47-e2d4-4682-add2-fe3a4b0421c6 None None] MFA Rules not processed for user `fd836cf590a145df9acf19cf3ae2ac56`. Rule list: `[]` (Enabled: `True`). {{(pid=88114) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:34:08.214105 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.fernet_utils [None req-f96aee47-e2d4-4682-add2-fe3a4b0421c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88114) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:34:08.241449 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c3b14f8-eb5a-4834-a4ea-6c47a195e895 None None] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:08.401943 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 233/466] 10.222.0.13 () {58 vars in 1024 bytes} [Wed Aug 31 09:34:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 234 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 09:34:08.429611 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c3b14f8-eb5a-4834-a4ea-6c47a195e895 None None] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:08.477788 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddbd6bd9-82df-4bfc-8d81-20bb42cee4b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:08.478454 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddbd6bd9-82df-4bfc-8d81-20bb42cee4b7 None admin] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:08.478730 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddbd6bd9-82df-4bfc-8d81-20bb42cee4b7 None admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:08.479018 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddbd6bd9-82df-4bfc-8d81-20bb42cee4b7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:08.493308 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddbd6bd9-82df-4bfc-8d81-20bb42cee4b7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:08.494959 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddbd6bd9-82df-4bfc-8d81-20bb42cee4b7 None admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:08.512311 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application [None req-ddbd6bd9-82df-4bfc-8d81-20bb42cee4b7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 09:34:08.512311 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:34:08.512311 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:34:08.512311 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:34:08.512311 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:34:08.512311 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:34:08.512311 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:34:08.512311 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:34:08.512311 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:34:08.512311 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:34:08.512311 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:34:08.512311 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:34:08.512311 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 09:34:08.512311 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 09:34:08.512311 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 09:34:08.512311 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 09:34:08.513960 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:34:08.513960 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:34:08.513960 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:34:08.513960 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:34:08.513960 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:34:08.513960 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:34:08.513960 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:34:08.513960 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application with Lock( Aug 31 09:34:08.513960 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:34:08.513960 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:34:08.513960 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:34:08.513960 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:34:08.513960 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:34:08.513960 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:34:08.513960 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:34:08.513960 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:34:08.513960 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 09:34:08.515349 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 09:34:08.515349 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 09:34:08.515349 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application Aug 31 09:34:08.515349 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 234/467] 10.222.0.13 () {56 vars in 1108 bytes} [Wed Aug 31 09:34:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 273 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 09:34:08.520667 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f96aee47-e2d4-4682-add2-fe3a4b0421c6 None None] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:08.625698 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddbd6bd9-82df-4bfc-8d81-20bb42cee4b7 None admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:08.700619 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:08.726719 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f96aee47-e2d4-4682-add2-fe3a4b0421c6 None None] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:08.755932 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=100 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:08.803693 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e64f4491-f638-487a-9d4c-9f04e96484e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:08.804671 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64f4491-f638-487a-9d4c-9f04e96484e4 None admin] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:08.805142 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64f4491-f638-487a-9d4c-9f04e96484e4 None admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:08.805672 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64f4491-f638-487a-9d4c-9f04e96484e4 None admin] PATH_INFO: `/v3/domains` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:08.806818 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e64f4491-f638-487a-9d4c-9f04e96484e4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88114) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:34:08.807329 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e64f4491-f638-487a-9d4c-9f04e96484e4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:08.810434 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e64f4491-f638-487a-9d4c-9f04e96484e4 None admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:08.835412 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 234/468] 10.222.0.13 () {56 vars in 1109 bytes} [Wed Aug 31 09:34:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 315 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 09:34:08.848692 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddbd6bd9-82df-4bfc-8d81-20bb42cee4b7 None admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:08.854347 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e64f4491-f638-487a-9d4c-9f04e96484e4 None admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:08.893756 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2007b32e-065b-4715-9e87-d0fbf7632100 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:08.895112 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2007b32e-065b-4715-9e87-d0fbf7632100 None admin] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:08.895596 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2007b32e-065b-4715-9e87-d0fbf7632100 None admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:08.896063 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2007b32e-065b-4715-9e87-d0fbf7632100 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:08.908306 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2007b32e-065b-4715-9e87-d0fbf7632100 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:08.910312 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2007b32e-065b-4715-9e87-d0fbf7632100 None admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:08.927930 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application [None req-2007b32e-065b-4715-9e87-d0fbf7632100 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 09:34:08.927930 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:34:08.927930 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:34:08.927930 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:34:08.927930 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:34:08.927930 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:34:08.927930 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:34:08.927930 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:34:08.927930 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:34:08.927930 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:34:08.927930 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:34:08.927930 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:34:08.927930 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 09:34:08.927930 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 09:34:08.927930 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 09:34:08.927930 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 09:34:08.928897 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:34:08.928897 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:34:08.928897 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:34:08.928897 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:34:08.928897 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:34:08.928897 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:34:08.928897 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:34:08.928897 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application with Lock( Aug 31 09:34:08.928897 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:34:08.928897 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:34:08.928897 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:34:08.928897 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:34:08.928897 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:34:08.928897 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:34:08.928897 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:34:08.928897 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:34:08.928897 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 09:34:08.930195 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 09:34:08.930195 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 09:34:08.930195 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application Aug 31 09:34:08.932105 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 235/469] 10.222.0.13 () {56 vars in 1108 bytes} [Wed Aug 31 09:34:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 307 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:34:08.933227 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2007b32e-065b-4715-9e87-d0fbf7632100 None admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:09.051447 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=100 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:09.064457 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2007b32e-065b-4715-9e87-d0fbf7632100 None admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:09.099682 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e64f4491-f638-487a-9d4c-9f04e96484e4 None admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:09.104085 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2201a77e-3d12-4311-8349-c2e2ee0e172c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:09.105110 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2201a77e-3d12-4311-8349-c2e2ee0e172c None admin] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:09.105676 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2201a77e-3d12-4311-8349-c2e2ee0e172c None admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:09.106149 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2201a77e-3d12-4311-8349-c2e2ee0e172c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:09.118920 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2201a77e-3d12-4311-8349-c2e2ee0e172c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:09.120753 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2201a77e-3d12-4311-8349-c2e2ee0e172c None admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:09.137673 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application [None req-2201a77e-3d12-4311-8349-c2e2ee0e172c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 09:34:09.137673 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:34:09.137673 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:34:09.137673 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:34:09.137673 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:34:09.137673 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:34:09.137673 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:34:09.137673 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:34:09.137673 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:34:09.137673 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:34:09.137673 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:34:09.137673 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:34:09.137673 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 09:34:09.137673 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 09:34:09.137673 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 09:34:09.137673 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 09:34:09.138666 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:34:09.138666 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:34:09.138666 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:34:09.138666 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:34:09.138666 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:34:09.138666 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:34:09.138666 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:34:09.138666 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application with Lock( Aug 31 09:34:09.138666 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:34:09.138666 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:34:09.138666 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:34:09.138666 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:34:09.138666 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:34:09.138666 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:34:09.138666 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:34:09.138666 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:34:09.138666 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 09:34:09.139506 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 09:34:09.139506 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 09:34:09.139506 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application Aug 31 09:34:09.142075 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 236/470] 10.222.0.13 () {56 vars in 1108 bytes} [Wed Aug 31 09:34:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:34:09.143179 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2201a77e-3d12-4311-8349-c2e2ee0e172c None admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:09.174092 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-453761d5-75c2-41c1-9524-c51cee29bd3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:09.175199 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453761d5-75c2-41c1-9524-c51cee29bd3b None admin] REQUEST_METHOD: `POST` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:09.175683 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453761d5-75c2-41c1-9524-c51cee29bd3b None admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:09.176176 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453761d5-75c2-41c1-9524-c51cee29bd3b None admin] PATH_INFO: `/v3/users` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:09.177455 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-453761d5-75c2-41c1-9524-c51cee29bd3b None admin] RBAC: Authorizing `identity:create_user()` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:09.188236 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-453761d5-75c2-41c1-9524-c51cee29bd3b None admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:09.233504 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 235/471] 10.222.0.13 () {60 vars in 1132 bytes} [Wed Aug 31 09:34:08 2022] POST /identity/v3/users => generated 248 bytes in 376 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 09:34:09.233504 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-453761d5-75c2-41c1-9524-c51cee29bd3b None admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:09.287376 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2201a77e-3d12-4311-8349-c2e2ee0e172c None admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:09.315875 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:09.333830 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f602375-366c-4665-bb32-83cbcdafcc7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:09.334980 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f602375-366c-4665-bb32-83cbcdafcc7d None admin] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:09.336597 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f602375-366c-4665-bb32-83cbcdafcc7d None admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:09.337127 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f602375-366c-4665-bb32-83cbcdafcc7d None admin] PATH_INFO: `/v3/domains` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:09.338128 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f602375-366c-4665-bb32-83cbcdafcc7d None admin] RBAC: Adding query filter params (name=Default) {{(pid=88113) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:34:09.338636 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f602375-366c-4665-bb32-83cbcdafcc7d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:09.340333 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f602375-366c-4665-bb32-83cbcdafcc7d None admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:09.356449 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 237/472] 10.222.0.13 () {56 vars in 1109 bytes} [Wed Aug 31 09:34:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:34:09.363054 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f602375-366c-4665-bb32-83cbcdafcc7d None admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:09.366970 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=100 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:09.482653 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-453761d5-75c2-41c1-9524-c51cee29bd3b None admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:09.509145 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f602375-366c-4665-bb32-83cbcdafcc7d None admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:09.555890 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d7824a-4ca7-4700-a761-efe177d8d129 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:09.557131 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d7824a-4ca7-4700-a761-efe177d8d129 None admin] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:09.557723 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d7824a-4ca7-4700-a761-efe177d8d129 None admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:09.558097 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d7824a-4ca7-4700-a761-efe177d8d129 None admin] PATH_INFO: `/v3/domains` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:09.559334 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3d7824a-4ca7-4700-a761-efe177d8d129 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88114) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:34:09.559857 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3d7824a-4ca7-4700-a761-efe177d8d129 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:09.563377 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3d7824a-4ca7-4700-a761-efe177d8d129 None admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:09.565117 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a666f5f9-3747-43ff-bbbe-97eeb3baa00d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:09.565917 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a666f5f9-3747-43ff-bbbe-97eeb3baa00d None admin] REQUEST_METHOD: `POST` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:09.566180 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a666f5f9-3747-43ff-bbbe-97eeb3baa00d None admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:09.566479 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a666f5f9-3747-43ff-bbbe-97eeb3baa00d None admin] PATH_INFO: `/v3/users` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:09.567137 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a666f5f9-3747-43ff-bbbe-97eeb3baa00d None admin] RBAC: Authorizing `identity:create_user()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:09.568765 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a666f5f9-3747-43ff-bbbe-97eeb3baa00d None admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:09.587991 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 236/473] 10.222.0.13 () {56 vars in 1109 bytes} [Wed Aug 31 09:34:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 359 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:34:09.591048 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da367ff-bcda-4ed3-a00a-3d6ee3fa68b5 None None] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:09.591577 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da367ff-bcda-4ed3-a00a-3d6ee3fa68b5 None None] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:09.592046 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da367ff-bcda-4ed3-a00a-3d6ee3fa68b5 None None] PATH_INFO: `/` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:09.596085 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 237/474] 10.222.0.13 () {54 vars in 927 bytes} [Wed Aug 31 09:34:09 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:34:09.602777 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de39a6f-af7a-4f30-ac65-2621a6d9db95 None None] REQUEST_METHOD: `POST` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:09.603153 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de39a6f-af7a-4f30-ac65-2621a6d9db95 None None] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:09.603602 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de39a6f-af7a-4f30-ac65-2621a6d9db95 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:09.608730 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 238/475] 10.222.0.13 () {60 vars in 1132 bytes} [Wed Aug 31 09:34:09 2022] POST /identity/v3/users => generated 248 bytes in 246 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 09:34:09.610170 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a666f5f9-3747-43ff-bbbe-97eeb3baa00d None admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:09.635248 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.auth.core [None req-9de39a6f-af7a-4f30-ac65-2621a6d9db95 None None] MFA Rules not processed for user `fd836cf590a145df9acf19cf3ae2ac56`. Rule list: `[]` (Enabled: `True`). {{(pid=88114) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:34:09.637426 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.fernet_utils [None req-9de39a6f-af7a-4f30-ac65-2621a6d9db95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88114) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:34:09.685639 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:09.740467 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a666f5f9-3747-43ff-bbbe-97eeb3baa00d None admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:09.759288 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 238/476] 10.222.0.13 () {58 vars in 1024 bytes} [Wed Aug 31 09:34:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 09:34:09.781329 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f8a4d02-0693-4a81-bdae-88afecb943aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:09.782067 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8a4d02-0693-4a81-bdae-88afecb943aa None admin] REQUEST_METHOD: `POST` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:09.782576 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8a4d02-0693-4a81-bdae-88afecb943aa None admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:09.782857 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8a4d02-0693-4a81-bdae-88afecb943aa None admin] PATH_INFO: `/v3/users` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:09.783531 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f8a4d02-0693-4a81-bdae-88afecb943aa None admin] RBAC: Authorizing `identity:create_user()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:09.785110 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f8a4d02-0693-4a81-bdae-88afecb943aa None admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:09.823794 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 239/477] 10.222.0.13 () {60 vars in 1132 bytes} [Wed Aug 31 09:34:09 2022] POST /identity/v3/users => generated 249 bytes in 214 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 09:34:10.303752 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de39a6f-af7a-4f30-ac65-2621a6d9db95 None None] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:10.354060 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=100 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:10.560306 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de39a6f-af7a-4f30-ac65-2621a6d9db95 None None] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:10.639934 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ebf6be5-6793-4382-ae33-c7925b1a542f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:10.641012 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ebf6be5-6793-4382-ae33-c7925b1a542f None admin] REQUEST_METHOD: `POST` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:10.641661 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ebf6be5-6793-4382-ae33-c7925b1a542f None admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:10.641894 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ebf6be5-6793-4382-ae33-c7925b1a542f None admin] PATH_INFO: `/v3/roles` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:10.643096 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ebf6be5-6793-4382-ae33-c7925b1a542f None admin] RBAC: Authorizing `identity:create_role()` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:10.645977 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ebf6be5-6793-4382-ae33-c7925b1a542f None admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:10.661343 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.sql.core [None req-6ebf6be5-6793-4382-ae33-c7925b1a542f None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Aug 31 09:34:10.661343 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Aug 31 09:34:10.661343 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [parameters: {'id': '33931d92401149949a6c11bfdd781acd', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Aug 31 09:34:10.661343 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=88114) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Aug 31 09:34:10.695656 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application [None req-6ebf6be5-6793-4382-ae33-c7925b1a542f None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Aug 31 09:34:10.695656 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:34:10.695656 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 31 09:34:10.695656 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 31 09:34:10.695656 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 31 09:34:10.695656 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 31 09:34:10.695656 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 31 09:34:10.695656 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application result = self._query(query) Aug 31 09:34:10.695656 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 31 09:34:10.695656 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application conn.query(q) Aug 31 09:34:10.695656 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 31 09:34:10.695656 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 31 09:34:10.695656 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 31 09:34:10.695656 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application result.read() Aug 31 09:34:10.695656 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 31 09:34:10.696591 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 31 09:34:10.696591 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 31 09:34:10.696591 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 31 09:34:10.696591 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 31 09:34:10.696591 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 31 09:34:10.696591 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 31 09:34:10.696591 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 31 09:34:10.696591 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Aug 31 09:34:10.696591 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application Aug 31 09:34:10.696591 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Aug 31 09:34:10.696591 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application Aug 31 09:34:10.696591 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:34:10.696591 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Aug 31 09:34:10.696591 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return method(*args, **kwargs) Aug 31 09:34:10.696591 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Aug 31 09:34:10.696591 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return ref.to_dict() Aug 31 09:34:10.696591 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 31 09:34:10.696591 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application next(self.gen) Aug 31 09:34:10.697515 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Aug 31 09:34:10.697515 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application yield resource Aug 31 09:34:10.697515 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 31 09:34:10.697515 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application next(self.gen) Aug 31 09:34:10.697515 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Aug 31 09:34:10.697515 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application self.session.rollback() Aug 31 09:34:10.697515 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Aug 31 09:34:10.697515 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application self.force_reraise() Aug 31 09:34:10.697515 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Aug 31 09:34:10.697515 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application raise self.value Aug 31 09:34:10.697515 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Aug 31 09:34:10.697515 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Aug 31 09:34:10.697515 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Aug 31 09:34:10.697515 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application session.commit() Aug 31 09:34:10.697515 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Aug 31 09:34:10.697515 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Aug 31 09:34:10.697515 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Aug 31 09:34:10.698359 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application self._prepare_impl() Aug 31 09:34:10.698359 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Aug 31 09:34:10.698359 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application self.session.flush() Aug 31 09:34:10.698359 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Aug 31 09:34:10.698359 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application self._flush(objects) Aug 31 09:34:10.698359 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Aug 31 09:34:10.698359 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Aug 31 09:34:10.698359 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Aug 31 09:34:10.698359 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application compat.raise_( Aug 31 09:34:10.698359 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 31 09:34:10.698359 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application raise exception Aug 31 09:34:10.698359 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Aug 31 09:34:10.698359 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application flush_context.execute() Aug 31 09:34:10.698359 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Aug 31 09:34:10.698359 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application rec.execute(self) Aug 31 09:34:10.698359 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Aug 31 09:34:10.698359 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Aug 31 09:34:10.699226 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Aug 31 09:34:10.699226 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application _emit_insert_statements( Aug 31 09:34:10.699226 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Aug 31 09:34:10.699226 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application c = connection._execute_20( Aug 31 09:34:10.699226 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Aug 31 09:34:10.699226 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Aug 31 09:34:10.699226 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Aug 31 09:34:10.699226 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Aug 31 09:34:10.699226 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Aug 31 09:34:10.699226 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application ret = self._execute_context( Aug 31 09:34:10.699226 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Aug 31 09:34:10.699226 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Aug 31 09:34:10.699226 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Aug 31 09:34:10.699226 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Aug 31 09:34:10.699226 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 31 09:34:10.699226 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application raise exception Aug 31 09:34:10.700039 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 31 09:34:10.700039 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 31 09:34:10.700039 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 31 09:34:10.700039 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 31 09:34:10.700039 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 31 09:34:10.700039 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application result = self._query(query) Aug 31 09:34:10.700039 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 31 09:34:10.700039 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application conn.query(q) Aug 31 09:34:10.700039 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 31 09:34:10.700039 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 31 09:34:10.700039 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 31 09:34:10.700039 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application result.read() Aug 31 09:34:10.700039 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 31 09:34:10.700039 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 31 09:34:10.700039 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 31 09:34:10.700039 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 31 09:34:10.700817 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 31 09:34:10.700817 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 31 09:34:10.700817 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 31 09:34:10.700817 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 31 09:34:10.700817 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Aug 31 09:34:10.700817 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Aug 31 09:34:10.700817 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application [parameters: {'id': '33931d92401149949a6c11bfdd781acd', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Aug 31 09:34:10.700817 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Aug 31 09:34:10.700817 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application Aug 31 09:34:10.700817 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 31 09:34:10.700817 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application Aug 31 09:34:10.700817 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:34:10.700817 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:34:10.700817 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:34:10.700817 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:34:10.700817 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:34:10.703202 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:34:10.703202 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:34:10.703202 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:34:10.703202 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:34:10.703202 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:34:10.703202 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:34:10.703202 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Aug 31 09:34:10.703202 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Aug 31 09:34:10.703202 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:34:10.703202 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:34:10.703202 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Aug 31 09:34:10.703202 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Aug 31 09:34:10.703202 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Aug 31 09:34:10.703202 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Aug 31 09:34:10.703202 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Aug 31 09:34:10.703202 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application Aug 31 09:34:10.705935 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 239/478] 10.222.0.13 () {60 vars in 1131 bytes} [Wed Aug 31 09:34:10 2022] POST /identity/v3/roles => generated 144 bytes in 396 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:34:10.711450 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f8a4d02-0693-4a81-bdae-88afecb943aa None admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:10.926719 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f8a4d02-0693-4a81-bdae-88afecb943aa None admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:10.966385 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:11.017053 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece5d853-1e9d-4a7b-be57-cfa3f0cb0471 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:11.017813 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece5d853-1e9d-4a7b-be57-cfa3f0cb0471 None admin] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:11.018296 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece5d853-1e9d-4a7b-be57-cfa3f0cb0471 None admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:11.018748 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece5d853-1e9d-4a7b-be57-cfa3f0cb0471 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:11.050958 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ece5d853-1e9d-4a7b-be57-cfa3f0cb0471 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 09:34:11.060922 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application [None req-ece5d853-1e9d-4a7b-be57-cfa3f0cb0471 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 31 09:34:11.060922 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:34:11.060922 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:34:11.060922 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:34:11.060922 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:34:11.060922 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:34:11.060922 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:34:11.060922 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:34:11.060922 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:34:11.060922 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:34:11.060922 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:34:11.060922 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:34:11.060922 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 09:34:11.060922 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 09:34:11.060922 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 09:34:11.060922 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 09:34:11.064342 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 09:34:11.064342 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 09:34:11.064342 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 09:34:11.064342 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 09:34:11.064342 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:34:11.064342 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:34:11.064342 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:34:11.064342 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:34:11.064342 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:34:11.064342 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:34:11.064342 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:34:11.064342 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application with Lock( Aug 31 09:34:11.064342 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:34:11.064342 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:34:11.064342 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:34:11.064342 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:34:11.066316 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:34:11.066316 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:34:11.066316 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:34:11.066316 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:34:11.066316 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 09:34:11.066316 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 09:34:11.066316 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 09:34:11.066316 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 09:34:11.066316 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 09:34:11.066316 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 09:34:11.066316 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 31 09:34:11.066316 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application Aug 31 09:34:11.066316 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 240/479] 10.222.0.13 () {56 vars in 1114 bytes} [Wed Aug 31 09:34:10 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 353 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:34:11.073108 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ebf6be5-6793-4382-ae33-c7925b1a542f None admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:11.286678 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:11.309321 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ebf6be5-6793-4382-ae33-c7925b1a542f None admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:11.378021 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca0f344-a99c-4439-944b-81ec6cbd445f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:11.379524 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca0f344-a99c-4439-944b-81ec6cbd445f None admin] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:11.380088 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca0f344-a99c-4439-944b-81ec6cbd445f None admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:11.380656 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca0f344-a99c-4439-944b-81ec6cbd445f None admin] PATH_INFO: `/v3/roles` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:11.382117 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bca0f344-a99c-4439-944b-81ec6cbd445f None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=88114) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:34:11.382655 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bca0f344-a99c-4439-944b-81ec6cbd445f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:11.385327 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bca0f344-a99c-4439-944b-81ec6cbd445f None admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:11.404315 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 240/480] 10.222.0.13 () {56 vars in 1111 bytes} [Wed Aug 31 09:34:11 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 332 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:34:11.927523 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e598e016-5438-459b-84d3-7ac874348646 None None] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:11.928044 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e598e016-5438-459b-84d3-7ac874348646 None None] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:11.928044 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e598e016-5438-459b-84d3-7ac874348646 None None] PATH_INFO: `/` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:11.929813 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 241/481] 10.222.0.13 () {54 vars in 927 bytes} [Wed Aug 31 09:34:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 31 09:34:11.935493 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1bf6e40-61e7-49ba-b9f3-f1f102ed04a3 None None] REQUEST_METHOD: `POST` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:11.935988 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1bf6e40-61e7-49ba-b9f3-f1f102ed04a3 None None] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:11.936061 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1bf6e40-61e7-49ba-b9f3-f1f102ed04a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:11.960782 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.auth.core [None req-d1bf6e40-61e7-49ba-b9f3-f1f102ed04a3 None None] MFA Rules not processed for user `fd836cf590a145df9acf19cf3ae2ac56`. Rule list: `[]` (Enabled: `True`). {{(pid=88114) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:34:11.962021 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.fernet_utils [None req-d1bf6e40-61e7-49ba-b9f3-f1f102ed04a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88114) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:34:12.086006 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 241/482] 10.222.0.13 () {58 vars in 1024 bytes} [Wed Aug 31 09:34:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 09:34:12.417534 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e598e016-5438-459b-84d3-7ac874348646 None None] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:12.543242 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e598e016-5438-459b-84d3-7ac874348646 None None] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:12.591456 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-906c2369-93a2-4660-b4cf-0c971980e1c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:12.592121 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906c2369-93a2-4660-b4cf-0c971980e1c9 None admin] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:12.592398 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906c2369-93a2-4660-b4cf-0c971980e1c9 None admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:12.592672 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906c2369-93a2-4660-b4cf-0c971980e1c9 None admin] PATH_INFO: `/v3/roles/service` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:12.607084 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-906c2369-93a2-4660-b4cf-0c971980e1c9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 09:34:12.613314 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application [None req-906c2369-93a2-4660-b4cf-0c971980e1c9 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 31 09:34:12.613314 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:34:12.613314 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:34:12.613314 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:34:12.613314 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:34:12.613314 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:34:12.613314 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:34:12.613314 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:34:12.613314 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:34:12.613314 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:34:12.613314 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:34:12.613314 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:34:12.613314 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 09:34:12.613314 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 09:34:12.613314 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 09:34:12.613314 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 09:34:12.615396 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 09:34:12.615396 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 09:34:12.615396 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 09:34:12.615396 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 09:34:12.615396 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:34:12.615396 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:34:12.615396 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:34:12.615396 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:34:12.615396 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:34:12.615396 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:34:12.615396 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:34:12.615396 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application with Lock( Aug 31 09:34:12.615396 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:34:12.615396 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:34:12.615396 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:34:12.615396 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:34:12.617076 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:34:12.617076 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:34:12.617076 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:34:12.617076 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:34:12.617076 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 09:34:12.617076 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 09:34:12.617076 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 09:34:12.617076 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 09:34:12.617076 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 09:34:12.617076 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 09:34:12.617076 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 31 09:34:12.617076 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application Aug 31 09:34:12.617076 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 242/483] 10.222.0.13 () {56 vars in 1102 bytes} [Wed Aug 31 09:34:12 2022] GET /identity/v3/roles/service => generated 85 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:34:12.627577 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1bf6e40-61e7-49ba-b9f3-f1f102ed04a3 None None] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:12.760814 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1552420a-e4a1-4884-ae7b-adfc838c62ca None None] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:12.761300 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1552420a-e4a1-4884-ae7b-adfc838c62ca None None] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:12.761503 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1552420a-e4a1-4884-ae7b-adfc838c62ca None None] PATH_INFO: `/` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:12.763128 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 243/484] 10.222.0.13 () {54 vars in 927 bytes} [Wed Aug 31 09:34:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:34:12.769290 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780f87ac-23d9-4b0b-bcc6-a44095d9b7c0 None None] REQUEST_METHOD: `POST` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:12.769967 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780f87ac-23d9-4b0b-bcc6-a44095d9b7c0 None None] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:12.770205 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780f87ac-23d9-4b0b-bcc6-a44095d9b7c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:12.791710 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.auth.core [None req-780f87ac-23d9-4b0b-bcc6-a44095d9b7c0 None None] MFA Rules not processed for user `fd836cf590a145df9acf19cf3ae2ac56`. Rule list: `[]` (Enabled: `True`). {{(pid=88113) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:34:12.792954 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.fernet_utils [None req-780f87ac-23d9-4b0b-bcc6-a44095d9b7c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:34:12.826657 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=100 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:12.839790 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=100 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:12.875469 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1bf6e40-61e7-49ba-b9f3-f1f102ed04a3 None None] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:12.944529 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f329e8e7-18ff-4210-ae0c-c656b06e7de3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:12.945807 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f329e8e7-18ff-4210-ae0c-c656b06e7de3 None admin] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:12.946412 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f329e8e7-18ff-4210-ae0c-c656b06e7de3 None admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:12.946822 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f329e8e7-18ff-4210-ae0c-c656b06e7de3 None admin] PATH_INFO: `/v3/roles` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:12.948252 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f329e8e7-18ff-4210-ae0c-c656b06e7de3 None admin] RBAC: Adding query filter params (name=service) {{(pid=88114) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:34:12.949035 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f329e8e7-18ff-4210-ae0c-c656b06e7de3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:12.951834 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f329e8e7-18ff-4210-ae0c-c656b06e7de3 None admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:12.969807 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 242/485] 10.222.0.13 () {56 vars in 1103 bytes} [Wed Aug 31 09:34:12 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 343 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:34:12.972540 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17a8aa9-8502-40c4-ac72-53d60c5443a9 None None] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:12.973355 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17a8aa9-8502-40c4-ac72-53d60c5443a9 None None] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:12.974154 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17a8aa9-8502-40c4-ac72-53d60c5443a9 None None] PATH_INFO: `/` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:12.976786 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 244/486] 10.222.0.13 () {58 vars in 1024 bytes} [Wed Aug 31 09:34:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 208 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 09:34:12.977994 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 243/487] 10.222.0.13 () {54 vars in 927 bytes} [Wed Aug 31 09:34:12 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:34:12.978363 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-780f87ac-23d9-4b0b-bcc6-a44095d9b7c0 None None] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:12.985029 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573dbcc9-8544-48ea-90a1-71c6ee64a21c None None] REQUEST_METHOD: `POST` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:12.985339 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573dbcc9-8544-48ea-90a1-71c6ee64a21c None None] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:12.986757 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573dbcc9-8544-48ea-90a1-71c6ee64a21c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:13.047549 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.auth.core [None req-573dbcc9-8544-48ea-90a1-71c6ee64a21c None None] MFA Rules not processed for user `fd836cf590a145df9acf19cf3ae2ac56`. Rule list: `[]` (Enabled: `True`). {{(pid=88114) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:34:13.049486 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.fernet_utils [None req-573dbcc9-8544-48ea-90a1-71c6ee64a21c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88114) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:34:13.215376 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 244/488] 10.222.0.13 () {58 vars in 1024 bytes} [Wed Aug 31 09:34:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 233 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 09:34:13.234100 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-780f87ac-23d9-4b0b-bcc6-a44095d9b7c0 None None] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:13.295715 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c2a5e85-1a0e-4e95-b382-0553d1261984 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:13.296376 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2a5e85-1a0e-4e95-b382-0553d1261984 None admin] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:13.297183 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2a5e85-1a0e-4e95-b382-0553d1261984 None admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:13.297500 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2a5e85-1a0e-4e95-b382-0553d1261984 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:13.308466 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-573dbcc9-8544-48ea-90a1-71c6ee64a21c None None] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:13.317965 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c2a5e85-1a0e-4e95-b382-0553d1261984 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:13.320147 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c2a5e85-1a0e-4e95-b382-0553d1261984 None admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:13.352462 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application [None req-7c2a5e85-1a0e-4e95-b382-0553d1261984 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 09:34:13.352462 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:34:13.352462 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:34:13.352462 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:34:13.352462 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:34:13.352462 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:34:13.352462 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:34:13.352462 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:34:13.352462 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:34:13.352462 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:34:13.352462 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:34:13.352462 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:34:13.352462 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 09:34:13.352462 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 09:34:13.352462 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 09:34:13.352462 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 09:34:13.353472 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:34:13.353472 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:34:13.353472 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:34:13.353472 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:34:13.353472 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:34:13.353472 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:34:13.353472 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:34:13.353472 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application with Lock( Aug 31 09:34:13.353472 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:34:13.353472 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:34:13.353472 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:34:13.353472 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:34:13.353472 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:34:13.353472 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:34:13.353472 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:34:13.353472 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:34:13.353472 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 09:34:13.354326 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 09:34:13.354326 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 09:34:13.354326 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application Aug 31 09:34:13.359914 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 245/489] 10.222.0.13 () {56 vars in 1108 bytes} [Wed Aug 31 09:34:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 382 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:34:13.365812 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c2a5e85-1a0e-4e95-b382-0553d1261984 None admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:13.398934 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=100 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:13.486196 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c2a5e85-1a0e-4e95-b382-0553d1261984 None admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:13.510545 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=100 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:13.531881 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce533766-52bb-4d86-92e3-657d8788cfe1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:13.533191 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce533766-52bb-4d86-92e3-657d8788cfe1 None admin] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:13.533732 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce533766-52bb-4d86-92e3-657d8788cfe1 None admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:13.534203 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce533766-52bb-4d86-92e3-657d8788cfe1 None admin] PATH_INFO: `/v3/domains` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:13.535092 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce533766-52bb-4d86-92e3-657d8788cfe1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88113) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:34:13.535605 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce533766-52bb-4d86-92e3-657d8788cfe1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:13.537271 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce533766-52bb-4d86-92e3-657d8788cfe1 None admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:13.563333 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 246/490] 10.222.0.13 () {56 vars in 1109 bytes} [Wed Aug 31 09:34:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:34:13.568948 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce533766-52bb-4d86-92e3-657d8788cfe1 None admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:13.598464 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-573dbcc9-8544-48ea-90a1-71c6ee64a21c None None] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:13.662877 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2680fc0-1391-4c83-8c44-86dd8a272a45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:13.663891 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2680fc0-1391-4c83-8c44-86dd8a272a45 None admin] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:13.664382 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2680fc0-1391-4c83-8c44-86dd8a272a45 None admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:13.665025 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2680fc0-1391-4c83-8c44-86dd8a272a45 None admin] PATH_INFO: `/v3/roles/service` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:13.690588 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2680fc0-1391-4c83-8c44-86dd8a272a45 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 09:34:13.695545 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application [None req-a2680fc0-1391-4c83-8c44-86dd8a272a45 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 31 09:34:13.695545 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:34:13.695545 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:34:13.695545 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:34:13.695545 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:34:13.695545 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:34:13.695545 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:34:13.695545 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:34:13.695545 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:34:13.695545 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:34:13.695545 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:34:13.695545 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:34:13.695545 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 09:34:13.695545 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 09:34:13.695545 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 09:34:13.695545 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 09:34:13.697310 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 09:34:13.697310 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 09:34:13.697310 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 09:34:13.697310 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 09:34:13.697310 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:34:13.697310 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:34:13.697310 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:34:13.697310 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:34:13.697310 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:34:13.697310 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:34:13.697310 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:34:13.697310 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application with Lock( Aug 31 09:34:13.697310 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:34:13.697310 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:34:13.697310 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:34:13.697310 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:34:13.699422 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:34:13.699422 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:34:13.699422 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:34:13.699422 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:34:13.699422 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 09:34:13.699422 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 09:34:13.699422 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 09:34:13.699422 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 09:34:13.699422 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 09:34:13.699422 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 09:34:13.699422 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 31 09:34:13.699422 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application Aug 31 09:34:13.701600 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 245/491] 10.222.0.13 () {56 vars in 1102 bytes} [Wed Aug 31 09:34:13 2022] GET /identity/v3/roles/service => generated 85 bytes in 394 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 09:34:13.702506 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2680fc0-1391-4c83-8c44-86dd8a272a45 None admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:13.730360 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=100 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:13.778315 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce533766-52bb-4d86-92e3-657d8788cfe1 None admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:13.821662 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2680fc0-1391-4c83-8c44-86dd8a272a45 None admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:13.849265 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca2ec2b-f912-4bd3-8222-eeb9d8f64d7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:13.850846 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca2ec2b-f912-4bd3-8222-eeb9d8f64d7e None admin] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:13.851606 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca2ec2b-f912-4bd3-8222-eeb9d8f64d7e None admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:13.852417 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca2ec2b-f912-4bd3-8222-eeb9d8f64d7e None admin] PATH_INFO: `/v3/users/nova` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:13.858447 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=100 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:13.870003 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e30d1c9c-3085-4228-b434-351cb3824ef8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:13.870620 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30d1c9c-3085-4228-b434-351cb3824ef8 None admin] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:13.870898 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30d1c9c-3085-4228-b434-351cb3824ef8 None admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:13.871168 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30d1c9c-3085-4228-b434-351cb3824ef8 None admin] PATH_INFO: `/v3/roles/service` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:13.871461 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ca2ec2b-f912-4bd3-8222-eeb9d8f64d7e None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:13.874370 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ca2ec2b-f912-4bd3-8222-eeb9d8f64d7e None admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:13.884884 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e30d1c9c-3085-4228-b434-351cb3824ef8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 09:34:13.889946 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application [None req-e30d1c9c-3085-4228-b434-351cb3824ef8 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 31 09:34:13.889946 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:34:13.889946 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:34:13.889946 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:34:13.889946 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:34:13.889946 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:34:13.889946 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:34:13.889946 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:34:13.889946 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:34:13.889946 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:34:13.889946 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:34:13.889946 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:34:13.889946 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 09:34:13.889946 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 09:34:13.889946 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 09:34:13.889946 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 09:34:13.890955 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 09:34:13.890955 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 09:34:13.890955 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 09:34:13.890955 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 09:34:13.890955 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:34:13.890955 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:34:13.890955 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:34:13.890955 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:34:13.890955 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:34:13.890955 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:34:13.890955 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:34:13.890955 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application with Lock( Aug 31 09:34:13.890955 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:34:13.890955 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:34:13.890955 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:34:13.890955 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:34:13.891761 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:34:13.891761 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:34:13.891761 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:34:13.891761 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:34:13.891761 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 09:34:13.891761 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 09:34:13.891761 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 09:34:13.891761 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 09:34:13.891761 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 09:34:13.891761 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 09:34:13.891761 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 31 09:34:13.891761 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application Aug 31 09:34:13.891761 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 246/492] 10.222.0.13 () {56 vars in 1102 bytes} [Wed Aug 31 09:34:13 2022] GET /identity/v3/roles/service => generated 85 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:34:13.892924 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df1a1fa-a9c0-4e71-b7ad-aef729211381 None None] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:13.893085 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df1a1fa-a9c0-4e71-b7ad-aef729211381 None None] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:13.893326 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df1a1fa-a9c0-4e71-b7ad-aef729211381 None None] PATH_INFO: `/` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:13.898770 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 247/493] 10.222.0.13 () {54 vars in 927 bytes} [Wed Aug 31 09:34:13 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:34:13.898770 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df1a1fa-a9c0-4e71-b7ad-aef729211381 None None] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:13.910844 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application [None req-4ca2ec2b-f912-4bd3-8222-eeb9d8f64d7e None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 31 09:34:13.910844 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:34:13.910844 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:34:13.910844 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:34:13.910844 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:34:13.910844 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:34:13.910844 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:34:13.910844 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:34:13.910844 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:34:13.910844 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:34:13.910844 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:34:13.910844 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:34:13.910844 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 31 09:34:13.910844 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 31 09:34:13.910844 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 31 09:34:13.910844 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 31 09:34:13.911813 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:34:13.911813 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:34:13.911813 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 31 09:34:13.911813 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 09:34:13.911813 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 31 09:34:13.911813 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 09:34:13.911813 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:34:13.911813 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:34:13.911813 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:34:13.911813 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:34:13.911813 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:34:13.911813 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application with Lock( Aug 31 09:34:13.911813 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:34:13.911813 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:34:13.911813 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:34:13.911813 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:34:13.911813 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:34:13.911813 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:34:13.912725 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:34:13.912725 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:34:13.912725 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 31 09:34:13.912725 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 31 09:34:13.912725 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 31 09:34:13.912725 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 31 09:34:13.912725 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 31 09:34:13.912725 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 31 09:34:13.912725 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 31 09:34:13.912725 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application Aug 31 09:34:13.913241 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 247/494] 10.222.0.13 () {56 vars in 1093 bytes} [Wed Aug 31 09:34:13 2022] GET /identity/v3/users/nova => generated 82 bytes in 344 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:34:13.914527 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca2ec2b-f912-4bd3-8222-eeb9d8f64d7e None admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:14.077755 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df1a1fa-a9c0-4e71-b7ad-aef729211381 None None] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:14.120091 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca2ec2b-f912-4bd3-8222-eeb9d8f64d7e None admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:14.132404 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e79ffaf-7c6d-4a54-bc76-87f9e71c15e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:14.133013 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e79ffaf-7c6d-4a54-bc76-87f9e71c15e3 None admin] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:14.133301 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e79ffaf-7c6d-4a54-bc76-87f9e71c15e3 None admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:14.133625 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e79ffaf-7c6d-4a54-bc76-87f9e71c15e3 None admin] PATH_INFO: `/v3/roles` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:14.134263 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e79ffaf-7c6d-4a54-bc76-87f9e71c15e3 None admin] RBAC: Adding query filter params (name=service) {{(pid=88114) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:34:14.134566 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e79ffaf-7c6d-4a54-bc76-87f9e71c15e3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:14.135990 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e79ffaf-7c6d-4a54-bc76-87f9e71c15e3 None admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:14.147416 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 248/495] 10.222.0.13 () {56 vars in 1103 bytes} [Wed Aug 31 09:34:13 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:34:14.151308 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42fdf10-2454-42c6-927d-a0e9f1c1add4 None None] REQUEST_METHOD: `POST` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:14.152077 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42fdf10-2454-42c6-927d-a0e9f1c1add4 None None] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:14.152870 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42fdf10-2454-42c6-927d-a0e9f1c1add4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:14.178067 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.auth.core [None req-f42fdf10-2454-42c6-927d-a0e9f1c1add4 None None] MFA Rules not processed for user `fd836cf590a145df9acf19cf3ae2ac56`. Rule list: `[]` (Enabled: `True`). {{(pid=88114) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:34:14.179111 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76df165f-39db-4302-b79e-7f3492af18a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:14.179738 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76df165f-39db-4302-b79e-7f3492af18a9 None admin] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:14.180016 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76df165f-39db-4302-b79e-7f3492af18a9 None admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:14.180128 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.fernet_utils [None req-f42fdf10-2454-42c6-927d-a0e9f1c1add4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88114) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:34:14.180282 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76df165f-39db-4302-b79e-7f3492af18a9 None admin] PATH_INFO: `/v3/roles` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:14.180959 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76df165f-39db-4302-b79e-7f3492af18a9 None admin] RBAC: Adding query filter params (name=service) {{(pid=88113) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:34:14.181262 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76df165f-39db-4302-b79e-7f3492af18a9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:14.182693 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76df165f-39db-4302-b79e-7f3492af18a9 None admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:14.189715 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=100 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:14.199476 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 248/496] 10.222.0.13 () {56 vars in 1103 bytes} [Wed Aug 31 09:34:13 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 286 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:34:14.200356 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76df165f-39db-4302-b79e-7f3492af18a9 None admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:14.264607 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=100 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:14.304242 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 249/497] 10.222.0.13 () {58 vars in 1024 bytes} [Wed Aug 31 09:34:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 09:34:14.305757 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f42fdf10-2454-42c6-927d-a0e9f1c1add4 None None] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:14.375755 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76df165f-39db-4302-b79e-7f3492af18a9 None admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:14.439597 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c22d1d6-f0cf-4f47-b763-9b7bed58cae8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:14.440624 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c22d1d6-f0cf-4f47-b763-9b7bed58cae8 None admin] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:14.441168 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c22d1d6-f0cf-4f47-b763-9b7bed58cae8 None admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:14.441689 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c22d1d6-f0cf-4f47-b763-9b7bed58cae8 None admin] PATH_INFO: `/v3/users` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:14.442698 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c22d1d6-f0cf-4f47-b763-9b7bed58cae8 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=88113) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:34:14.443210 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c22d1d6-f0cf-4f47-b763-9b7bed58cae8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:14.444941 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c22d1d6-f0cf-4f47-b763-9b7bed58cae8 None admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:14.498441 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 249/498] 10.222.0.13 () {56 vars in 1133 bytes} [Wed Aug 31 09:34:14 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 298 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:34:14.499722 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c22d1d6-f0cf-4f47-b763-9b7bed58cae8 None admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:14.501013 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f42fdf10-2454-42c6-927d-a0e9f1c1add4 None None] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:14.574955 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c97af34-1895-44f2-899a-5ba20d3db17a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:14.576071 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c97af34-1895-44f2-899a-5ba20d3db17a None admin] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:14.577231 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c97af34-1895-44f2-899a-5ba20d3db17a None admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:14.577708 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c97af34-1895-44f2-899a-5ba20d3db17a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:14.600018 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c97af34-1895-44f2-899a-5ba20d3db17a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:14.602588 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c97af34-1895-44f2-899a-5ba20d3db17a None admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:14.610775 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=100 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:14.634119 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application [None req-9c97af34-1895-44f2-899a-5ba20d3db17a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 09:34:14.634119 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:34:14.634119 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:34:14.634119 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:34:14.634119 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:34:14.634119 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:34:14.634119 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:34:14.634119 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:34:14.634119 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:34:14.634119 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:34:14.634119 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:34:14.634119 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:34:14.634119 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 09:34:14.634119 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 09:34:14.634119 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 09:34:14.634119 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 09:34:14.636100 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:34:14.636100 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:34:14.636100 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:34:14.636100 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:34:14.636100 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:34:14.636100 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:34:14.636100 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:34:14.636100 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application with Lock( Aug 31 09:34:14.636100 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:34:14.636100 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:34:14.636100 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:34:14.636100 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:34:14.636100 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:34:14.636100 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:34:14.636100 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:34:14.636100 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:34:14.636100 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 09:34:14.638037 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 09:34:14.638037 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 09:34:14.638037 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application Aug 31 09:34:14.642183 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 250/499] 10.222.0.13 () {56 vars in 1108 bytes} [Wed Aug 31 09:34:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 337 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:34:14.643832 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c97af34-1895-44f2-899a-5ba20d3db17a None admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:14.703819 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c22d1d6-f0cf-4f47-b763-9b7bed58cae8 None admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:14.706993 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=100 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:14.755084 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f34035c6-6ec6-4140-98d5-16068d052252 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:14.755748 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34035c6-6ec6-4140-98d5-16068d052252 None admin] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:14.756027 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34035c6-6ec6-4140-98d5-16068d052252 None admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:14.756302 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34035c6-6ec6-4140-98d5-16068d052252 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:14.768933 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f34035c6-6ec6-4140-98d5-16068d052252 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:14.770745 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f34035c6-6ec6-4140-98d5-16068d052252 None admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:14.788910 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application [None req-f34035c6-6ec6-4140-98d5-16068d052252 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 09:34:14.788910 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:34:14.788910 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:34:14.788910 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:34:14.788910 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:34:14.788910 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:34:14.788910 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:34:14.788910 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:34:14.788910 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:34:14.788910 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:34:14.788910 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:34:14.788910 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:34:14.788910 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 09:34:14.788910 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 09:34:14.788910 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 09:34:14.788910 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 09:34:14.794285 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:34:14.794285 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:34:14.794285 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:34:14.794285 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:34:14.794285 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:34:14.794285 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:34:14.794285 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:34:14.794285 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application with Lock( Aug 31 09:34:14.794285 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:34:14.794285 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:34:14.794285 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:34:14.794285 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:34:14.794285 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:34:14.794285 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:34:14.794285 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:34:14.794285 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:34:14.794285 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 09:34:14.795926 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 09:34:14.795926 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 09:34:14.795926 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application Aug 31 09:34:14.795926 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 250/500] 10.222.0.13 () {56 vars in 1108 bytes} [Wed Aug 31 09:34:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 291 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:34:14.795926 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f34035c6-6ec6-4140-98d5-16068d052252 None admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:14.821073 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c97af34-1895-44f2-899a-5ba20d3db17a None admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:14.880369 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc53fe3-647d-4fbd-b082-17a58b6f226d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:14.880986 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc53fe3-647d-4fbd-b082-17a58b6f226d None admin] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:14.881863 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc53fe3-647d-4fbd-b082-17a58b6f226d None admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:14.882138 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc53fe3-647d-4fbd-b082-17a58b6f226d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:14.944439 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f34035c6-6ec6-4140-98d5-16068d052252 None admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:14.954871 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=100 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:14.984128 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cc53fe3-647d-4fbd-b082-17a58b6f226d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:14.986765 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cc53fe3-647d-4fbd-b082-17a58b6f226d None admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:14.989846 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b06f210-73d6-40dd-88aa-6052fcdca24b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:14.990475 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b06f210-73d6-40dd-88aa-6052fcdca24b None admin] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:14.990748 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b06f210-73d6-40dd-88aa-6052fcdca24b None admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:14.991018 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b06f210-73d6-40dd-88aa-6052fcdca24b None admin] PATH_INFO: `/v3/domains` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:14.991678 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b06f210-73d6-40dd-88aa-6052fcdca24b None admin] RBAC: Adding query filter params (name=Default) {{(pid=88113) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:34:14.991980 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b06f210-73d6-40dd-88aa-6052fcdca24b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:14.993423 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b06f210-73d6-40dd-88aa-6052fcdca24b None admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:15.008304 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 251/501] 10.222.0.13 () {56 vars in 1109 bytes} [Wed Aug 31 09:34:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:34:15.010267 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b06f210-73d6-40dd-88aa-6052fcdca24b None admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:15.036582 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=100 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:15.155492 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 251/502] 10.222.0.13 () {58 vars in 1282 bytes} [Wed Aug 31 09:34:14 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 513 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 31 09:34:15.158241 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc53fe3-647d-4fbd-b082-17a58b6f226d None admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:15.174996 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b06f210-73d6-40dd-88aa-6052fcdca24b None admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:15.221331 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f3c606-1a88-4352-922f-3157e3f649b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:15.222010 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f3c606-1a88-4352-922f-3157e3f649b0 None admin] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:15.225906 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f3c606-1a88-4352-922f-3157e3f649b0 None admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:15.226326 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f3c606-1a88-4352-922f-3157e3f649b0 None admin] PATH_INFO: `/v3/roles/service` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:15.242912 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51f3c606-1a88-4352-922f-3157e3f649b0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 09:34:15.252915 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application [None req-51f3c606-1a88-4352-922f-3157e3f649b0 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 31 09:34:15.252915 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:34:15.252915 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:34:15.252915 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:34:15.252915 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:34:15.252915 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:34:15.252915 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:34:15.252915 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:34:15.252915 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:34:15.252915 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:34:15.252915 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:34:15.252915 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:34:15.252915 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 09:34:15.252915 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 09:34:15.252915 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 09:34:15.252915 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 09:34:15.254876 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 09:34:15.254876 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 09:34:15.254876 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 09:34:15.254876 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 09:34:15.254876 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:34:15.254876 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:34:15.254876 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:34:15.254876 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:34:15.254876 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:34:15.254876 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:34:15.254876 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:34:15.254876 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application with Lock( Aug 31 09:34:15.254876 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:34:15.254876 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:34:15.254876 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:34:15.254876 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:34:15.256803 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:34:15.256803 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:34:15.256803 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:34:15.256803 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:34:15.256803 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 09:34:15.256803 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 09:34:15.256803 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 09:34:15.256803 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 09:34:15.256803 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 09:34:15.256803 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 09:34:15.256803 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 31 09:34:15.256803 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application Aug 31 09:34:15.256803 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 252/503] 10.222.0.13 () {56 vars in 1102 bytes} [Wed Aug 31 09:34:15 2022] GET /identity/v3/roles/service => generated 85 bytes in 246 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:34:15.256803 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f3c606-1a88-4352-922f-3157e3f649b0 None admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:15.309580 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=100 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:15.317065 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc53fe3-647d-4fbd-b082-17a58b6f226d None admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:15.356638 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=100 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:15.362782 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad889985-b7a2-4565-81b2-a32ea4d81dc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:15.363478 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad889985-b7a2-4565-81b2-a32ea4d81dc9 None admin] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:15.364053 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad889985-b7a2-4565-81b2-a32ea4d81dc9 None admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:15.364322 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad889985-b7a2-4565-81b2-a32ea4d81dc9 None admin] PATH_INFO: `/v3/domains` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:15.364981 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad889985-b7a2-4565-81b2-a32ea4d81dc9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88114) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:34:15.365287 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad889985-b7a2-4565-81b2-a32ea4d81dc9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:15.366725 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad889985-b7a2-4565-81b2-a32ea4d81dc9 None admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:15.379911 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 252/504] 10.222.0.13 () {56 vars in 1109 bytes} [Wed Aug 31 09:34:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:34:15.380770 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad889985-b7a2-4565-81b2-a32ea4d81dc9 None admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:15.402569 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f3c606-1a88-4352-922f-3157e3f649b0 None admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:15.442107 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d36c312-4e22-43fb-b271-4d01d88637fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:15.442926 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d36c312-4e22-43fb-b271-4d01d88637fb None admin] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:15.443330 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d36c312-4e22-43fb-b271-4d01d88637fb None admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:15.443714 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d36c312-4e22-43fb-b271-4d01d88637fb None admin] PATH_INFO: `/v3/users/glance` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:15.452064 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d36c312-4e22-43fb-b271-4d01d88637fb None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:15.454480 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d36c312-4e22-43fb-b271-4d01d88637fb None admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:15.467463 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application [None req-0d36c312-4e22-43fb-b271-4d01d88637fb None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Aug 31 09:34:15.467463 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:34:15.467463 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:34:15.467463 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:34:15.467463 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:34:15.467463 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:34:15.467463 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:34:15.467463 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:34:15.467463 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:34:15.467463 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:34:15.467463 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:34:15.467463 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:34:15.467463 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 31 09:34:15.467463 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 31 09:34:15.467463 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 31 09:34:15.467463 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 31 09:34:15.468433 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:34:15.468433 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:34:15.468433 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 31 09:34:15.468433 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 09:34:15.468433 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 31 09:34:15.468433 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 09:34:15.468433 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:34:15.468433 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:34:15.468433 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:34:15.468433 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:34:15.468433 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:34:15.468433 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application with Lock( Aug 31 09:34:15.468433 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:34:15.468433 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:34:15.468433 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:34:15.468433 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:34:15.468433 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:34:15.468433 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:34:15.470336 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:34:15.470336 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:34:15.470336 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 31 09:34:15.470336 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 31 09:34:15.470336 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 31 09:34:15.470336 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 31 09:34:15.470336 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 31 09:34:15.470336 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 31 09:34:15.470336 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Aug 31 09:34:15.470336 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application Aug 31 09:34:15.473676 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 253/505] 10.222.0.13 () {56 vars in 1099 bytes} [Wed Aug 31 09:34:15 2022] GET /identity/v3/users/glance => generated 84 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:34:15.475214 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6702f7-a787-4c83-873f-3615af915764 None None] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:15.475500 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6702f7-a787-4c83-873f-3615af915764 None None] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:15.475773 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6702f7-a787-4c83-873f-3615af915764 None None] PATH_INFO: `/` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:15.477098 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 254/506] 10.222.0.13 () {54 vars in 927 bytes} [Wed Aug 31 09:34:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:34:15.477842 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa6702f7-a787-4c83-873f-3615af915764 None None] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:15.500934 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad889985-b7a2-4565-81b2-a32ea4d81dc9 None admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:15.541128 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3519b3d5-f27e-40b1-a97c-ed133eb5b2ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:15.542213 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3519b3d5-f27e-40b1-a97c-ed133eb5b2ba None admin] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:15.542683 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3519b3d5-f27e-40b1-a97c-ed133eb5b2ba None admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:15.543160 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3519b3d5-f27e-40b1-a97c-ed133eb5b2ba None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:15.563750 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3519b3d5-f27e-40b1-a97c-ed133eb5b2ba None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:15.566557 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3519b3d5-f27e-40b1-a97c-ed133eb5b2ba None admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:15.577824 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=100 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:15.587893 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application [None req-3519b3d5-f27e-40b1-a97c-ed133eb5b2ba None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 09:34:15.587893 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:34:15.587893 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:34:15.587893 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:34:15.587893 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:34:15.587893 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:34:15.587893 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:34:15.587893 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:34:15.587893 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:34:15.587893 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:34:15.587893 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:34:15.587893 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:34:15.587893 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 09:34:15.587893 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 09:34:15.587893 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 09:34:15.587893 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 09:34:15.589727 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:34:15.589727 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:34:15.589727 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:34:15.589727 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:34:15.589727 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:34:15.589727 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:34:15.589727 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:34:15.589727 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application with Lock( Aug 31 09:34:15.589727 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:34:15.589727 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:34:15.589727 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:34:15.589727 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:34:15.589727 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:34:15.589727 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:34:15.589727 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:34:15.589727 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:34:15.589727 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 09:34:15.591433 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 09:34:15.591433 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 09:34:15.591433 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application Aug 31 09:34:15.591433 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 253/507] 10.222.0.13 () {56 vars in 1108 bytes} [Wed Aug 31 09:34:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:34:15.591433 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3519b3d5-f27e-40b1-a97c-ed133eb5b2ba None admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:15.610188 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa6702f7-a787-4c83-873f-3615af915764 None None] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:15.648357 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=100 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:15.658728 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e42a9df7-2670-4009-9c06-4580712a8282 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:15.659528 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42a9df7-2670-4009-9c06-4580712a8282 None admin] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:15.659897 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42a9df7-2670-4009-9c06-4580712a8282 None admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:15.660334 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42a9df7-2670-4009-9c06-4580712a8282 None admin] PATH_INFO: `/v3/roles` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:15.661186 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e42a9df7-2670-4009-9c06-4580712a8282 None admin] RBAC: Adding query filter params (name=service) {{(pid=88113) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:34:15.661623 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e42a9df7-2670-4009-9c06-4580712a8282 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:15.663290 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e42a9df7-2670-4009-9c06-4580712a8282 None admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:15.675197 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 255/508] 10.222.0.13 () {56 vars in 1103 bytes} [Wed Aug 31 09:34:15 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:34:15.676631 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e42a9df7-2670-4009-9c06-4580712a8282 None admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:15.736024 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3519b3d5-f27e-40b1-a97c-ed133eb5b2ba None admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:15.783805 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c50ba5d1-1ebb-4d8b-bd9d-1723d7c44d4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:15.784451 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50ba5d1-1ebb-4d8b-bd9d-1723d7c44d4f None admin] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:15.784722 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50ba5d1-1ebb-4d8b-bd9d-1723d7c44d4f None admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:15.785012 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50ba5d1-1ebb-4d8b-bd9d-1723d7c44d4f None admin] PATH_INFO: `/v3/users/cinder` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:15.793714 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c50ba5d1-1ebb-4d8b-bd9d-1723d7c44d4f None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:15.795086 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c50ba5d1-1ebb-4d8b-bd9d-1723d7c44d4f None admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:15.805717 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e42a9df7-2670-4009-9c06-4580712a8282 None admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:15.809306 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application [None req-c50ba5d1-1ebb-4d8b-bd9d-1723d7c44d4f None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 31 09:34:15.809306 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:34:15.809306 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:34:15.809306 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:34:15.809306 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:34:15.809306 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:34:15.809306 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:34:15.809306 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:34:15.809306 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:34:15.809306 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:34:15.809306 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:34:15.809306 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:34:15.809306 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 31 09:34:15.809306 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 31 09:34:15.809306 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 31 09:34:15.809306 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 31 09:34:15.811148 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:34:15.811148 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:34:15.811148 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 31 09:34:15.811148 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 09:34:15.811148 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 31 09:34:15.811148 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 09:34:15.811148 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:34:15.811148 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:34:15.811148 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:34:15.811148 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:34:15.811148 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:34:15.811148 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application with Lock( Aug 31 09:34:15.811148 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:34:15.811148 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:34:15.811148 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:34:15.811148 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:34:15.811148 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:34:15.811148 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:34:15.813467 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:34:15.813467 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:34:15.813467 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 31 09:34:15.813467 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 31 09:34:15.813467 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 31 09:34:15.813467 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 31 09:34:15.813467 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 31 09:34:15.813467 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 31 09:34:15.813467 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 31 09:34:15.813467 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application Aug 31 09:34:15.813467 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 254/509] 10.222.0.13 () {56 vars in 1099 bytes} [Wed Aug 31 09:34:15 2022] GET /identity/v3/users/cinder => generated 84 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:34:15.813467 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded93d48-b9da-465d-913d-c63d11253e08 None None] REQUEST_METHOD: `POST` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:15.813467 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded93d48-b9da-465d-913d-c63d11253e08 None None] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:15.816142 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded93d48-b9da-465d-913d-c63d11253e08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:15.835163 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.auth.core [None req-ded93d48-b9da-465d-913d-c63d11253e08 None None] MFA Rules not processed for user `fd836cf590a145df9acf19cf3ae2ac56`. Rule list: `[]` (Enabled: `True`). {{(pid=88114) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:34:15.836404 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.fernet_utils [None req-ded93d48-b9da-465d-913d-c63d11253e08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88114) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:34:15.861470 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e822f3ea-9ebd-4d19-9a1e-b46fea28c6d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:15.862441 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e822f3ea-9ebd-4d19-9a1e-b46fea28c6d0 None admin] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:15.862912 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e822f3ea-9ebd-4d19-9a1e-b46fea28c6d0 None admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:15.863448 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e822f3ea-9ebd-4d19-9a1e-b46fea28c6d0 None admin] PATH_INFO: `/v3/users` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:15.864415 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e822f3ea-9ebd-4d19-9a1e-b46fea28c6d0 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=88113) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:34:15.864850 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e822f3ea-9ebd-4d19-9a1e-b46fea28c6d0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:15.866577 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e822f3ea-9ebd-4d19-9a1e-b46fea28c6d0 None admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:15.888858 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 256/510] 10.222.0.13 () {56 vars in 1137 bytes} [Wed Aug 31 09:34:15 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:34:15.889824 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e822f3ea-9ebd-4d19-9a1e-b46fea28c6d0 None admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:15.910628 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=100 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:15.943532 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 255/511] 10.222.0.13 () {58 vars in 1024 bytes} [Wed Aug 31 09:34:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 09:34:15.944900 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ded93d48-b9da-465d-913d-c63d11253e08 None None] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:16.018369 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=100 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:16.050601 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e822f3ea-9ebd-4d19-9a1e-b46fea28c6d0 None admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:16.113559 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-633d9e49-8523-4d96-945b-c9671ba14192 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:16.114484 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633d9e49-8523-4d96-945b-c9671ba14192 None admin] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:16.114923 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633d9e49-8523-4d96-945b-c9671ba14192 None admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:16.115366 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633d9e49-8523-4d96-945b-c9671ba14192 None admin] PATH_INFO: `/v3/domains` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:16.116555 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-633d9e49-8523-4d96-945b-c9671ba14192 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88113) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:34:16.117091 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-633d9e49-8523-4d96-945b-c9671ba14192 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:16.119783 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-633d9e49-8523-4d96-945b-c9671ba14192 None admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:16.145835 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 257/512] 10.222.0.13 () {56 vars in 1109 bytes} [Wed Aug 31 09:34:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:34:16.145835 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-633d9e49-8523-4d96-945b-c9671ba14192 None admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:16.155709 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ded93d48-b9da-465d-913d-c63d11253e08 None None] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:16.221634 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-750d0bff-7546-4894-84f2-dda9d602b55a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:16.222287 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750d0bff-7546-4894-84f2-dda9d602b55a None admin] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:16.222567 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750d0bff-7546-4894-84f2-dda9d602b55a None admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:16.222838 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750d0bff-7546-4894-84f2-dda9d602b55a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:16.235667 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-750d0bff-7546-4894-84f2-dda9d602b55a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:16.237153 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-750d0bff-7546-4894-84f2-dda9d602b55a None admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:16.254341 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=100 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:16.261132 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application [None req-750d0bff-7546-4894-84f2-dda9d602b55a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 09:34:16.261132 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:34:16.261132 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:34:16.261132 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:34:16.261132 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:34:16.261132 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:34:16.261132 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:34:16.261132 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:34:16.261132 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:34:16.261132 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:34:16.261132 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:34:16.261132 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:34:16.261132 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 09:34:16.261132 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 09:34:16.261132 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 09:34:16.261132 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 09:34:16.262855 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:34:16.262855 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:34:16.262855 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:34:16.262855 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:34:16.262855 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:34:16.262855 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:34:16.262855 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:34:16.262855 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application with Lock( Aug 31 09:34:16.262855 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:34:16.262855 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:34:16.262855 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:34:16.262855 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:34:16.262855 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:34:16.262855 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:34:16.262855 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:34:16.262855 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:34:16.262855 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 09:34:16.272370 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 09:34:16.272370 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 09:34:16.272370 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application Aug 31 09:34:16.272370 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 256/513] 10.222.0.13 () {56 vars in 1108 bytes} [Wed Aug 31 09:34:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 319 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:34:16.272370 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-750d0bff-7546-4894-84f2-dda9d602b55a None admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:16.304666 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-633d9e49-8523-4d96-945b-c9671ba14192 None admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:16.355366 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce702b6-d6aa-46f0-ba08-34191267ca37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:16.356112 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce702b6-d6aa-46f0-ba08-34191267ca37 None admin] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:16.356470 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce702b6-d6aa-46f0-ba08-34191267ca37 None admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:16.356823 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce702b6-d6aa-46f0-ba08-34191267ca37 None admin] PATH_INFO: `/v3/users` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:16.357824 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ce702b6-d6aa-46f0-ba08-34191267ca37 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=88113) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:34:16.358188 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ce702b6-d6aa-46f0-ba08-34191267ca37 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:16.360399 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ce702b6-d6aa-46f0-ba08-34191267ca37 None admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:16.382070 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 258/514] 10.222.0.13 () {56 vars in 1137 bytes} [Wed Aug 31 09:34:16 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:34:16.383509 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce702b6-d6aa-46f0-ba08-34191267ca37 None admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:16.387997 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=100 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:16.408071 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-750d0bff-7546-4894-84f2-dda9d602b55a None admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:16.464099 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-489debbb-af57-4010-b6b8-6bc753c07b9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:16.464990 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489debbb-af57-4010-b6b8-6bc753c07b9a None admin] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:16.465434 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489debbb-af57-4010-b6b8-6bc753c07b9a None admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:16.465847 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489debbb-af57-4010-b6b8-6bc753c07b9a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:16.509304 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce702b6-d6aa-46f0-ba08-34191267ca37 None admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:16.544707 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=100 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:16.556101 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e788c7-ba0a-4404-8802-2d7dbcb7badd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:16.556737 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e788c7-ba0a-4404-8802-2d7dbcb7badd None admin] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:16.557012 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e788c7-ba0a-4404-8802-2d7dbcb7badd None admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:16.557281 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e788c7-ba0a-4404-8802-2d7dbcb7badd None admin] PATH_INFO: `/v3/projects/service` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:16.565200 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10e788c7-ba0a-4404-8802-2d7dbcb7badd None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:16.566720 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10e788c7-ba0a-4404-8802-2d7dbcb7badd None admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:16.567878 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-489debbb-af57-4010-b6b8-6bc753c07b9a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:16.569691 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-489debbb-af57-4010-b6b8-6bc753c07b9a None admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:16.579685 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application [None req-10e788c7-ba0a-4404-8802-2d7dbcb7badd None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 09:34:16.579685 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:34:16.579685 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:34:16.579685 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:34:16.579685 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:34:16.579685 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:34:16.579685 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:34:16.579685 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:34:16.579685 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:34:16.579685 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:34:16.579685 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:34:16.579685 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:34:16.579685 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 31 09:34:16.579685 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 31 09:34:16.579685 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 31 09:34:16.588878 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 31 09:34:16.588878 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:34:16.588878 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:34:16.588878 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:34:16.588878 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:34:16.588878 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:34:16.588878 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:34:16.588878 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:34:16.588878 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application with Lock( Aug 31 09:34:16.588878 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:34:16.588878 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:34:16.588878 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:34:16.588878 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:34:16.588878 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:34:16.588878 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:34:16.588878 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:34:16.588878 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:34:16.590744 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 31 09:34:16.590744 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 31 09:34:16.590744 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 31 09:34:16.590744 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 31 09:34:16.590744 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 31 09:34:16.590744 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 31 09:34:16.590744 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 09:34:16.590744 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application Aug 31 09:34:16.590744 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 259/515] 10.222.0.13 () {56 vars in 1111 bytes} [Wed Aug 31 09:34:16 2022] GET /identity/v3/projects/service => generated 88 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:34:16.590744 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e788c7-ba0a-4404-8802-2d7dbcb7badd None admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:16.696453 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e788c7-ba0a-4404-8802-2d7dbcb7badd None admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:16.706014 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=100 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:16.734861 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 257/516] 10.222.0.13 () {58 vars in 1282 bytes} [Wed Aug 31 09:34:16 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 471 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 31 09:34:16.736324 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-489debbb-af57-4010-b6b8-6bc753c07b9a None admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:16.740779 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a99ad2a-38fe-4a75-b79a-6d3cf80503e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:16.741553 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a99ad2a-38fe-4a75-b79a-6d3cf80503e5 None admin] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:16.741876 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a99ad2a-38fe-4a75-b79a-6d3cf80503e5 None admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:16.742230 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a99ad2a-38fe-4a75-b79a-6d3cf80503e5 None admin] PATH_INFO: `/v3/domains` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:16.743349 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a99ad2a-38fe-4a75-b79a-6d3cf80503e5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88113) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:34:16.743828 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a99ad2a-38fe-4a75-b79a-6d3cf80503e5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:16.746237 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a99ad2a-38fe-4a75-b79a-6d3cf80503e5 None admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:16.765466 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 260/517] 10.222.0.13 () {56 vars in 1109 bytes} [Wed Aug 31 09:34:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:34:16.766654 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a99ad2a-38fe-4a75-b79a-6d3cf80503e5 None admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:16.837102 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=100 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:16.877102 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-489debbb-af57-4010-b6b8-6bc753c07b9a None admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:16.910324 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a99ad2a-38fe-4a75-b79a-6d3cf80503e5 None admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:16.925048 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ababf89b-60a2-48bc-94fd-c3c6ffc6c715 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:16.926124 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ababf89b-60a2-48bc-94fd-c3c6ffc6c715 None admin] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:16.926600 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ababf89b-60a2-48bc-94fd-c3c6ffc6c715 None admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:16.927068 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ababf89b-60a2-48bc-94fd-c3c6ffc6c715 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:16.940394 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ababf89b-60a2-48bc-94fd-c3c6ffc6c715 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:16.941918 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ababf89b-60a2-48bc-94fd-c3c6ffc6c715 None admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:16.959150 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application [None req-ababf89b-60a2-48bc-94fd-c3c6ffc6c715 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 09:34:16.959150 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:34:16.959150 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:34:16.959150 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:34:16.959150 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:34:16.959150 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:34:16.959150 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:34:16.959150 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:34:16.959150 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:34:16.959150 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:34:16.959150 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:34:16.959150 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:34:16.959150 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 09:34:16.959150 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 09:34:16.959150 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 09:34:16.959150 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 09:34:16.960966 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:34:16.960966 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:34:16.960966 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:34:16.960966 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:34:16.960966 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:34:16.960966 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:34:16.960966 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:34:16.960966 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application with Lock( Aug 31 09:34:16.960966 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:34:16.960966 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:34:16.960966 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:34:16.960966 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:34:16.960966 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:34:16.960966 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:34:16.960966 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:34:16.960966 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:34:16.960966 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 09:34:16.963281 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 09:34:16.963281 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 09:34:16.963281 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application Aug 31 09:34:16.963281 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 258/518] 10.222.0.13 () {56 vars in 1108 bytes} [Wed Aug 31 09:34:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:34:16.963281 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ababf89b-60a2-48bc-94fd-c3c6ffc6c715 None admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:16.978450 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d87d754-e524-4a39-baeb-ad3a64fb9b29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:16.979073 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d87d754-e524-4a39-baeb-ad3a64fb9b29 None admin] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:16.979372 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d87d754-e524-4a39-baeb-ad3a64fb9b29 None admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:16.979644 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d87d754-e524-4a39-baeb-ad3a64fb9b29 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:17.018154 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=100 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:17.059212 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d87d754-e524-4a39-baeb-ad3a64fb9b29 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:17.060757 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d87d754-e524-4a39-baeb-ad3a64fb9b29 None admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:17.099830 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ababf89b-60a2-48bc-94fd-c3c6ffc6c715 None admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:17.136190 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=100 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:17.141038 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c56131-fab8-4301-9013-6640557625d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:17.142040 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c56131-fab8-4301-9013-6640557625d4 None admin] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:17.142493 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c56131-fab8-4301-9013-6640557625d4 None admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:17.143169 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c56131-fab8-4301-9013-6640557625d4 None admin] PATH_INFO: `/v3/projects` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:17.144701 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7c56131-fab8-4301-9013-6640557625d4 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88114) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:34:17.145140 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7c56131-fab8-4301-9013-6640557625d4 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:17.147640 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7c56131-fab8-4301-9013-6640557625d4 None admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:17.177515 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 259/519] 10.222.0.13 () {56 vars in 1148 bytes} [Wed Aug 31 09:34:16 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:34:17.178618 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c56131-fab8-4301-9013-6640557625d4 None admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:17.222128 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 261/520] 10.222.0.13 () {58 vars in 1282 bytes} [Wed Aug 31 09:34:16 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 456 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 31 09:34:17.224213 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d87d754-e524-4a39-baeb-ad3a64fb9b29 None admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:17.290492 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=100 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:17.297164 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c56131-fab8-4301-9013-6640557625d4 None admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:17.335666 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a1dc24-3fdb-4a33-baf2-dfa6b46406be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:17.336300 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a1dc24-3fdb-4a33-baf2-dfa6b46406be None admin] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:17.336578 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a1dc24-3fdb-4a33-baf2-dfa6b46406be None admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:17.336851 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a1dc24-3fdb-4a33-baf2-dfa6b46406be None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:17.348724 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68a1dc24-3fdb-4a33-baf2-dfa6b46406be None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:17.350275 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68a1dc24-3fdb-4a33-baf2-dfa6b46406be None admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:17.366392 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application [None req-68a1dc24-3fdb-4a33-baf2-dfa6b46406be None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 09:34:17.366392 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:34:17.366392 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:34:17.366392 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:34:17.366392 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:34:17.366392 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:34:17.366392 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:34:17.366392 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:34:17.366392 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:34:17.366392 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:34:17.366392 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:34:17.366392 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:34:17.366392 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 09:34:17.366392 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 09:34:17.366392 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 09:34:17.366392 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 09:34:17.368076 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:34:17.368076 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:34:17.368076 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:34:17.368076 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:34:17.368076 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:34:17.368076 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:34:17.368076 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:34:17.368076 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application with Lock( Aug 31 09:34:17.368076 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:34:17.368076 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:34:17.368076 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:34:17.368076 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:34:17.368076 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:34:17.368076 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:34:17.368076 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:34:17.368076 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:34:17.368076 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 09:34:17.369862 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 09:34:17.369862 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 09:34:17.369862 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application Aug 31 09:34:17.369862 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 260/521] 10.222.0.13 () {56 vars in 1108 bytes} [Wed Aug 31 09:34:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:34:17.369862 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a1dc24-3fdb-4a33-baf2-dfa6b46406be None admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:17.378510 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d87d754-e524-4a39-baeb-ad3a64fb9b29 None admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:17.420109 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5372e77-cbd3-489f-bb8e-5dee399e4c0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:17.421174 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5372e77-cbd3-489f-bb8e-5dee399e4c0a None admin] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:17.422351 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5372e77-cbd3-489f-bb8e-5dee399e4c0a None admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:17.422814 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5372e77-cbd3-489f-bb8e-5dee399e4c0a None admin] PATH_INFO: `/v3/users/neutron` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:17.430992 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5372e77-cbd3-489f-bb8e-5dee399e4c0a None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:17.433452 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5372e77-cbd3-489f-bb8e-5dee399e4c0a None admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:17.445139 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application [None req-e5372e77-cbd3-489f-bb8e-5dee399e4c0a None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Aug 31 09:34:17.445139 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:34:17.445139 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:34:17.445139 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:34:17.445139 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:34:17.445139 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:34:17.445139 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:34:17.445139 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:34:17.445139 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:34:17.445139 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:34:17.445139 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:34:17.445139 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:34:17.445139 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 31 09:34:17.445139 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 31 09:34:17.445139 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 31 09:34:17.445139 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 31 09:34:17.446328 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:34:17.446328 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:34:17.446328 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 31 09:34:17.446328 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 09:34:17.446328 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 31 09:34:17.446328 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 09:34:17.446328 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:34:17.446328 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:34:17.446328 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:34:17.446328 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:34:17.446328 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:34:17.446328 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application with Lock( Aug 31 09:34:17.446328 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:34:17.446328 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:34:17.446328 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:34:17.446328 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:34:17.446328 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:34:17.446328 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:34:17.447252 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:34:17.447252 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:34:17.447252 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 31 09:34:17.447252 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 31 09:34:17.447252 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 31 09:34:17.447252 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 31 09:34:17.447252 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 31 09:34:17.447252 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 31 09:34:17.447252 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Aug 31 09:34:17.447252 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application Aug 31 09:34:17.447252 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 262/522] 10.222.0.13 () {56 vars in 1102 bytes} [Wed Aug 31 09:34:17 2022] GET /identity/v3/users/neutron => generated 85 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:34:17.447252 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5372e77-cbd3-489f-bb8e-5dee399e4c0a None admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:17.457005 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=100 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:17.490660 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a1dc24-3fdb-4a33-baf2-dfa6b46406be None admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:17.526476 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7869a5c5-b8aa-4267-b05c-d2c19621bbee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:17.527312 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7869a5c5-b8aa-4267-b05c-d2c19621bbee None admin] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:17.527427 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7869a5c5-b8aa-4267-b05c-d2c19621bbee None admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:17.527616 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7869a5c5-b8aa-4267-b05c-d2c19621bbee None admin] PATH_INFO: `/v3/domains` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:17.528260 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7869a5c5-b8aa-4267-b05c-d2c19621bbee None admin] RBAC: Adding query filter params (name=Default) {{(pid=88114) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:34:17.528566 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7869a5c5-b8aa-4267-b05c-d2c19621bbee None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:17.529986 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7869a5c5-b8aa-4267-b05c-d2c19621bbee None admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:17.543550 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 261/523] 10.222.0.13 () {56 vars in 1109 bytes} [Wed Aug 31 09:34:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:34:17.544395 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7869a5c5-b8aa-4267-b05c-d2c19621bbee None admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:17.557088 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=100 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:17.576653 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5372e77-cbd3-489f-bb8e-5dee399e4c0a None admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:17.615601 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e56f8f-bd40-430c-9eb1-77755760bf3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:17.616122 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e56f8f-bd40-430c-9eb1-77755760bf3b None admin] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:17.616396 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e56f8f-bd40-430c-9eb1-77755760bf3b None admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:17.616664 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e56f8f-bd40-430c-9eb1-77755760bf3b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:17.617430 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1e56f8f-bd40-430c-9eb1-77755760bf3b None admin] RBAC: Adding query filter params (role.id=e24ba4fd91584883af350cc84bc96c1d, scope.project.id=49548eb44d62415582af820ab66decee, user.id=dc076493239d4ccdbdc37c0369c49529) {{(pid=88113) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:34:17.617722 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1e56f8f-bd40-430c-9eb1-77755760bf3b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:17.619166 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1e56f8f-bd40-430c-9eb1-77755760bf3b None admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:17.625870 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 263/524] 10.222.0.13 () {56 vars in 1374 bytes} [Wed Aug 31 09:34:17 2022] GET /identity/v3/role_assignments?user.id=dc076493239d4ccdbdc37c0369c49529&scope.project.id=49548eb44d62415582af820ab66decee&role.id=e24ba4fd91584883af350cc84bc96c1d => generated 259 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:34:17.626725 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e56f8f-bd40-430c-9eb1-77755760bf3b None admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:17.673815 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7869a5c5-b8aa-4267-b05c-d2c19621bbee None admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:17.715917 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=100 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:17.722367 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ff410d-9430-4541-b0aa-73ddf06d3542 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:17.723062 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ff410d-9430-4541-b0aa-73ddf06d3542 None admin] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:17.723342 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ff410d-9430-4541-b0aa-73ddf06d3542 None admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:17.723612 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ff410d-9430-4541-b0aa-73ddf06d3542 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:17.737155 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34ff410d-9430-4541-b0aa-73ddf06d3542 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:17.738813 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34ff410d-9430-4541-b0aa-73ddf06d3542 None admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:17.756119 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application [None req-34ff410d-9430-4541-b0aa-73ddf06d3542 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 09:34:17.756119 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:34:17.756119 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:34:17.756119 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:34:17.756119 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:34:17.756119 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:34:17.756119 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:34:17.756119 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:34:17.756119 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:34:17.756119 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:34:17.756119 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:34:17.756119 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:34:17.756119 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 09:34:17.756119 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 09:34:17.756119 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 09:34:17.756119 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 09:34:17.757209 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:34:17.757209 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:34:17.757209 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:34:17.757209 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:34:17.757209 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:34:17.757209 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:34:17.757209 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:34:17.757209 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application with Lock( Aug 31 09:34:17.757209 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:34:17.757209 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:34:17.757209 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:34:17.757209 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:34:17.757209 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:34:17.757209 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:34:17.757209 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:34:17.757209 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:34:17.757209 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 09:34:17.758097 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 09:34:17.758097 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 09:34:17.758097 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application Aug 31 09:34:17.758097 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 262/525] 10.222.0.13 () {56 vars in 1108 bytes} [Wed Aug 31 09:34:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:34:17.758331 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ff410d-9430-4541-b0aa-73ddf06d3542 None admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:17.768750 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e56f8f-bd40-430c-9eb1-77755760bf3b None admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:17.823376 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d07f69-025e-43c5-a9b1-541a649412d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:17.824394 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d07f69-025e-43c5-a9b1-541a649412d3 None admin] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:17.824931 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d07f69-025e-43c5-a9b1-541a649412d3 None admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:17.825628 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d07f69-025e-43c5-a9b1-541a649412d3 None admin] PATH_INFO: `/v3/domains` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:17.826830 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15d07f69-025e-43c5-a9b1-541a649412d3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88113) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:34:17.827380 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15d07f69-025e-43c5-a9b1-541a649412d3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:17.830627 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15d07f69-025e-43c5-a9b1-541a649412d3 None admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:17.836350 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=100 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:17.858044 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 264/526] 10.222.0.13 () {56 vars in 1109 bytes} [Wed Aug 31 09:34:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:34:17.858044 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d07f69-025e-43c5-a9b1-541a649412d3 None admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:17.893153 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ff410d-9430-4541-b0aa-73ddf06d3542 None admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:17.936170 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7984425-2470-4674-a8b9-bfc62987eda9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:17.937757 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7984425-2470-4674-a8b9-bfc62987eda9 None admin] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:17.938287 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7984425-2470-4674-a8b9-bfc62987eda9 None admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:17.939033 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7984425-2470-4674-a8b9-bfc62987eda9 None admin] PATH_INFO: `/v3/users` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:17.940414 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7984425-2470-4674-a8b9-bfc62987eda9 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=88114) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:34:17.940741 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7984425-2470-4674-a8b9-bfc62987eda9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:17.943245 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7984425-2470-4674-a8b9-bfc62987eda9 None admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:17.979779 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d07f69-025e-43c5-a9b1-541a649412d3 None admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:17.999064 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 263/527] 10.222.0.13 () {56 vars in 1139 bytes} [Wed Aug 31 09:34:17 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:34:18.004001 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7984425-2470-4674-a8b9-bfc62987eda9 None admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:18.017520 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=100 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:18.027773 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5654c080-be64-40b0-8822-552605ba8fa5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:18.028572 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5654c080-be64-40b0-8822-552605ba8fa5 None admin] REQUEST_METHOD: `POST` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:18.029300 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5654c080-be64-40b0-8822-552605ba8fa5 None admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:18.029601 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5654c080-be64-40b0-8822-552605ba8fa5 None admin] PATH_INFO: `/v3/users` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:18.030280 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5654c080-be64-40b0-8822-552605ba8fa5 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:18.032320 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5654c080-be64-40b0-8822-552605ba8fa5 None admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:18.057713 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 265/528] 10.222.0.13 () {60 vars in 1132 bytes} [Wed Aug 31 09:34:17 2022] POST /identity/v3/users => generated 247 bytes in 203 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 09:34:18.058572 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5654c080-be64-40b0-8822-552605ba8fa5 None admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:18.186894 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7984425-2470-4674-a8b9-bfc62987eda9 None admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:18.193044 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5654c080-be64-40b0-8822-552605ba8fa5 None admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:18.216885 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=100 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:18.238191 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb3bf6d8-007d-4c84-a55d-cc25f1397db3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:18.238824 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3bf6d8-007d-4c84-a55d-cc25f1397db3 None admin] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:18.239166 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3bf6d8-007d-4c84-a55d-cc25f1397db3 None admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:18.239442 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3bf6d8-007d-4c84-a55d-cc25f1397db3 None admin] PATH_INFO: `/v3/domains` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:18.240153 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb3bf6d8-007d-4c84-a55d-cc25f1397db3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88114) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:34:18.240424 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb3bf6d8-007d-4c84-a55d-cc25f1397db3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:18.241889 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb3bf6d8-007d-4c84-a55d-cc25f1397db3 None admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:18.252934 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1035d591-9c79-4024-9c51-f8d23bd196a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:18.254339 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1035d591-9c79-4024-9c51-f8d23bd196a0 None admin] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:18.254814 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1035d591-9c79-4024-9c51-f8d23bd196a0 None admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:18.255303 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1035d591-9c79-4024-9c51-f8d23bd196a0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:18.260403 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 264/529] 10.222.0.13 () {56 vars in 1109 bytes} [Wed Aug 31 09:34:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:34:18.260403 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb3bf6d8-007d-4c84-a55d-cc25f1397db3 None admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:18.277487 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1035d591-9c79-4024-9c51-f8d23bd196a0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:18.279807 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1035d591-9c79-4024-9c51-f8d23bd196a0 None admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:18.304584 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application [None req-1035d591-9c79-4024-9c51-f8d23bd196a0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 09:34:18.304584 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:34:18.304584 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:34:18.304584 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:34:18.304584 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:34:18.304584 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:34:18.304584 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:34:18.304584 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:34:18.304584 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:34:18.304584 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:34:18.304584 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:34:18.304584 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:34:18.304584 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 31 09:34:18.304584 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 31 09:34:18.304584 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 31 09:34:18.306203 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 31 09:34:18.306203 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:34:18.306203 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:34:18.306203 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:34:18.306203 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:34:18.306203 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:34:18.306203 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:34:18.306203 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:34:18.306203 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application with Lock( Aug 31 09:34:18.306203 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:34:18.306203 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:34:18.306203 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:34:18.306203 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:34:18.306203 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:34:18.306203 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:34:18.306203 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:34:18.306203 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:34:18.308604 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 31 09:34:18.308604 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 31 09:34:18.308604 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 31 09:34:18.308604 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 31 09:34:18.308604 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 31 09:34:18.308604 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 31 09:34:18.308604 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 09:34:18.308604 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application Aug 31 09:34:18.308604 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 266/530] 10.222.0.13 () {56 vars in 1111 bytes} [Wed Aug 31 09:34:18 2022] GET /identity/v3/projects/service => generated 88 bytes in 246 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:34:18.308604 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1035d591-9c79-4024-9c51-f8d23bd196a0 None admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:18.383232 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:18.449807 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb3bf6d8-007d-4c84-a55d-cc25f1397db3 None admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:18.493823 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1035d591-9c79-4024-9c51-f8d23bd196a0 None admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:18.508489 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd13654-29d6-47fe-91ad-33e299a2378a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:18.510584 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd13654-29d6-47fe-91ad-33e299a2378a None admin] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:18.511850 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd13654-29d6-47fe-91ad-33e299a2378a None admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:18.512300 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd13654-29d6-47fe-91ad-33e299a2378a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:18.545012 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4509b7d0-1c91-46ea-9061-606332565bba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:18.547048 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4509b7d0-1c91-46ea-9061-606332565bba None admin] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:18.547048 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4509b7d0-1c91-46ea-9061-606332565bba None admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:18.547048 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4509b7d0-1c91-46ea-9061-606332565bba None admin] PATH_INFO: `/v3/projects/service` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:18.555907 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4509b7d0-1c91-46ea-9061-606332565bba None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:18.557453 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4509b7d0-1c91-46ea-9061-606332565bba None admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:18.572137 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application [None req-4509b7d0-1c91-46ea-9061-606332565bba None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 09:34:18.572137 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:34:18.572137 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:34:18.572137 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:34:18.572137 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:34:18.572137 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:34:18.572137 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:34:18.572137 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:34:18.572137 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:34:18.572137 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:34:18.572137 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:34:18.572137 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:34:18.572137 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 31 09:34:18.572137 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 31 09:34:18.572137 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 31 09:34:18.573152 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 31 09:34:18.573152 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:34:18.573152 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:34:18.573152 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:34:18.573152 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:34:18.573152 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:34:18.573152 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:34:18.573152 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:34:18.573152 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application with Lock( Aug 31 09:34:18.573152 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:34:18.573152 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:34:18.573152 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:34:18.573152 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:34:18.573152 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:34:18.573152 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:34:18.573152 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:34:18.573152 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:34:18.574119 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 31 09:34:18.574119 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 31 09:34:18.574119 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 31 09:34:18.574119 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 31 09:34:18.574119 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 31 09:34:18.574119 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 31 09:34:18.574119 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 09:34:18.574119 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application Aug 31 09:34:18.579721 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 267/531] 10.222.0.13 () {56 vars in 1111 bytes} [Wed Aug 31 09:34:18 2022] GET /identity/v3/projects/service => generated 88 bytes in 273 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:34:18.580935 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4509b7d0-1c91-46ea-9061-606332565bba None admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:18.585619 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=100 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:18.628382 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbd13654-29d6-47fe-91ad-33e299a2378a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:18.631472 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbd13654-29d6-47fe-91ad-33e299a2378a None admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:18.729194 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4509b7d0-1c91-46ea-9061-606332565bba None admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:18.739420 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=100 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:18.776786 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f83c68ae-8996-4a04-be66-b4e73bbd5359 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:18.777438 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83c68ae-8996-4a04-be66-b4e73bbd5359 None admin] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:18.777726 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83c68ae-8996-4a04-be66-b4e73bbd5359 None admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:18.778003 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83c68ae-8996-4a04-be66-b4e73bbd5359 None admin] PATH_INFO: `/v3/projects` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:18.778724 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f83c68ae-8996-4a04-be66-b4e73bbd5359 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88113) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:34:18.779033 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f83c68ae-8996-4a04-be66-b4e73bbd5359 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:18.780502 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f83c68ae-8996-4a04-be66-b4e73bbd5359 None admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:18.802552 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 268/532] 10.222.0.13 () {56 vars in 1148 bytes} [Wed Aug 31 09:34:18 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:34:18.803512 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f83c68ae-8996-4a04-be66-b4e73bbd5359 None admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:18.843912 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 265/533] 10.222.0.13 () {58 vars in 1282 bytes} [Wed Aug 31 09:34:18 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 586 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 31 09:34:18.853970 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd13654-29d6-47fe-91ad-33e299a2378a None admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:18.945943 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f83c68ae-8996-4a04-be66-b4e73bbd5359 None admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:18.959229 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=100 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:18.992380 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd13654-29d6-47fe-91ad-33e299a2378a None admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:19.014998 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c2d2667-fc34-4968-98a5-4a202c2a688c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:19.016227 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2d2667-fc34-4968-98a5-4a202c2a688c None admin] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:19.016683 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2d2667-fc34-4968-98a5-4a202c2a688c None admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:19.017122 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2d2667-fc34-4968-98a5-4a202c2a688c None admin] PATH_INFO: `/v3/projects` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:19.018050 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c2d2667-fc34-4968-98a5-4a202c2a688c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88113) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:34:19.018527 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c2d2667-fc34-4968-98a5-4a202c2a688c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:19.020876 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c2d2667-fc34-4968-98a5-4a202c2a688c None admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:19.040941 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 269/534] 10.222.0.13 () {56 vars in 1148 bytes} [Wed Aug 31 09:34:18 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:34:19.042213 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c2d2667-fc34-4968-98a5-4a202c2a688c None admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:19.046162 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-453a5bd1-2899-4006-b967-d302e858e30f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:19.048189 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453a5bd1-2899-4006-b967-d302e858e30f None admin] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:19.048670 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453a5bd1-2899-4006-b967-d302e858e30f None admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:19.049047 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453a5bd1-2899-4006-b967-d302e858e30f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:19.049991 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-453a5bd1-2899-4006-b967-d302e858e30f None admin] RBAC: Adding query filter params (role.id=e24ba4fd91584883af350cc84bc96c1d, scope.project.id=49548eb44d62415582af820ab66decee, user.id=be216d156b54452cbbf978c7734343aa) {{(pid=88114) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:34:19.050410 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-453a5bd1-2899-4006-b967-d302e858e30f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:19.053120 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-453a5bd1-2899-4006-b967-d302e858e30f None admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:19.061291 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 266/535] 10.222.0.13 () {56 vars in 1374 bytes} [Wed Aug 31 09:34:18 2022] GET /identity/v3/role_assignments?user.id=be216d156b54452cbbf978c7734343aa&scope.project.id=49548eb44d62415582af820ab66decee&role.id=e24ba4fd91584883af350cc84bc96c1d => generated 259 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:34:19.061291 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-453a5bd1-2899-4006-b967-d302e858e30f None admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:19.084963 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=100 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:19.195248 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-453a5bd1-2899-4006-b967-d302e858e30f None admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:19.244590 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c2d2667-fc34-4968-98a5-4a202c2a688c None admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:19.244801 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad898d33-c7ad-444f-9ea1-3f26da1bccb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:19.245294 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad898d33-c7ad-444f-9ea1-3f26da1bccb9 None admin] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:19.245674 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad898d33-c7ad-444f-9ea1-3f26da1bccb9 None admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:19.246017 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad898d33-c7ad-444f-9ea1-3f26da1bccb9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:19.247217 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad898d33-c7ad-444f-9ea1-3f26da1bccb9 None admin] RBAC: Adding query filter params (role.id=e24ba4fd91584883af350cc84bc96c1d, scope.project.id=49548eb44d62415582af820ab66decee, user.id=0d52b69c8e94441dacfcff944326f169) {{(pid=88114) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:34:19.247603 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad898d33-c7ad-444f-9ea1-3f26da1bccb9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:19.249256 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad898d33-c7ad-444f-9ea1-3f26da1bccb9 None admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:19.266405 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 267/536] 10.222.0.13 () {56 vars in 1374 bytes} [Wed Aug 31 09:34:19 2022] GET /identity/v3/role_assignments?user.id=0d52b69c8e94441dacfcff944326f169&scope.project.id=49548eb44d62415582af820ab66decee&role.id=e24ba4fd91584883af350cc84bc96c1d => generated 259 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:34:19.302777 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeab055c-5fb4-4ce8-a5fe-75458d5a1c89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:19.303566 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeab055c-5fb4-4ce8-a5fe-75458d5a1c89 None admin] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:19.303976 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeab055c-5fb4-4ce8-a5fe-75458d5a1c89 None admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:19.304281 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeab055c-5fb4-4ce8-a5fe-75458d5a1c89 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:19.321119 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeab055c-5fb4-4ce8-a5fe-75458d5a1c89 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:19.322798 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeab055c-5fb4-4ce8-a5fe-75458d5a1c89 None admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:19.345031 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application [None req-aeab055c-5fb4-4ce8-a5fe-75458d5a1c89 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 09:34:19.345031 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:34:19.345031 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:34:19.345031 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:34:19.345031 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:34:19.345031 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:34:19.345031 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:34:19.345031 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:34:19.345031 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:34:19.345031 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:34:19.345031 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:34:19.345031 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:34:19.345031 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 09:34:19.345031 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 09:34:19.345031 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 09:34:19.345031 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 09:34:19.346228 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:34:19.346228 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:34:19.346228 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:34:19.346228 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:34:19.346228 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:34:19.346228 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:34:19.346228 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:34:19.346228 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application with Lock( Aug 31 09:34:19.346228 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:34:19.346228 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:34:19.346228 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:34:19.346228 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:34:19.346228 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:34:19.346228 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:34:19.346228 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:34:19.346228 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:34:19.346228 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 09:34:19.347092 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 09:34:19.347092 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 09:34:19.347092 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application Aug 31 09:34:19.347092 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 270/537] 10.222.0.13 () {56 vars in 1108 bytes} [Wed Aug 31 09:34:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 305 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:34:19.352603 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad898d33-c7ad-444f-9ea1-3f26da1bccb9 None admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:19.362432 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=100 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:19.537308 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad898d33-c7ad-444f-9ea1-3f26da1bccb9 None admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:19.589719 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dfcde06-6ec8-4cff-a284-18a4f97d1e6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:19.590350 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dfcde06-6ec8-4cff-a284-18a4f97d1e6f None admin] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:19.590667 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dfcde06-6ec8-4cff-a284-18a4f97d1e6f None admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:19.591010 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dfcde06-6ec8-4cff-a284-18a4f97d1e6f None admin] PATH_INFO: `/v3/domains` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:19.591762 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dfcde06-6ec8-4cff-a284-18a4f97d1e6f None admin] RBAC: Adding query filter params (name=Default) {{(pid=88114) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:34:19.592169 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dfcde06-6ec8-4cff-a284-18a4f97d1e6f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:19.595288 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dfcde06-6ec8-4cff-a284-18a4f97d1e6f None admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:19.622436 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 268/538] 10.222.0.13 () {56 vars in 1109 bytes} [Wed Aug 31 09:34:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:34:19.627703 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeab055c-5fb4-4ce8-a5fe-75458d5a1c89 None admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:19.757933 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=100 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:19.798377 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeab055c-5fb4-4ce8-a5fe-75458d5a1c89 None admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:19.860266 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b63fe8-e050-4e15-bf77-1d792bf8dae6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:19.860940 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b63fe8-e050-4e15-bf77-1d792bf8dae6 None admin] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:19.861220 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b63fe8-e050-4e15-bf77-1d792bf8dae6 None admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:19.861522 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b63fe8-e050-4e15-bf77-1d792bf8dae6 None admin] PATH_INFO: `/v3/projects/service` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:19.873337 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1b63fe8-e050-4e15-bf77-1d792bf8dae6 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:19.875401 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1b63fe8-e050-4e15-bf77-1d792bf8dae6 None admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:19.890046 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application [None req-f1b63fe8-e050-4e15-bf77-1d792bf8dae6 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 09:34:19.890046 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:34:19.890046 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:34:19.890046 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:34:19.890046 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:34:19.890046 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:34:19.890046 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:34:19.890046 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:34:19.890046 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:34:19.890046 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:34:19.890046 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:34:19.890046 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:34:19.890046 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 31 09:34:19.890046 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 31 09:34:19.890046 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 31 09:34:19.891832 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 31 09:34:19.891832 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:34:19.891832 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:34:19.891832 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:34:19.891832 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:34:19.891832 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:34:19.891832 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:34:19.891832 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:34:19.891832 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application with Lock( Aug 31 09:34:19.891832 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:34:19.891832 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:34:19.891832 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:34:19.891832 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:34:19.891832 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:34:19.891832 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:34:19.891832 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:34:19.891832 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:34:19.892684 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 31 09:34:19.892684 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 31 09:34:19.892684 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 31 09:34:19.892684 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 31 09:34:19.892684 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 31 09:34:19.892684 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 31 09:34:19.892684 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 09:34:19.892684 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application Aug 31 09:34:19.897149 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 271/539] 10.222.0.13 () {56 vars in 1111 bytes} [Wed Aug 31 09:34:19 2022] GET /identity/v3/projects/service => generated 88 bytes in 265 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:34:19.903407 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dfcde06-6ec8-4cff-a284-18a4f97d1e6f None admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:20.063230 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=100 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:20.103573 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dfcde06-6ec8-4cff-a284-18a4f97d1e6f None admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:20.147148 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a0f17a-ce6d-4ace-ae3b-ee2edc9f8169 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:20.147789 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a0f17a-ce6d-4ace-ae3b-ee2edc9f8169 None admin] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:20.148068 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a0f17a-ce6d-4ace-ae3b-ee2edc9f8169 None admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:20.148333 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a0f17a-ce6d-4ace-ae3b-ee2edc9f8169 None admin] PATH_INFO: `/v3/projects` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:20.149063 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78a0f17a-ce6d-4ace-ae3b-ee2edc9f8169 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88114) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:34:20.149366 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78a0f17a-ce6d-4ace-ae3b-ee2edc9f8169 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:20.150884 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78a0f17a-ce6d-4ace-ae3b-ee2edc9f8169 None admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:20.165142 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 269/540] 10.222.0.13 () {56 vars in 1148 bytes} [Wed Aug 31 09:34:19 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:34:20.171069 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b63fe8-e050-4e15-bf77-1d792bf8dae6 None admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:20.348229 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b63fe8-e050-4e15-bf77-1d792bf8dae6 None admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:20.358280 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2938e0a-e486-4caa-a9b1-65df06e2db44 None None] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:20.359120 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2938e0a-e486-4caa-a9b1-65df06e2db44 None None] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:20.359120 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2938e0a-e486-4caa-a9b1-65df06e2db44 None None] PATH_INFO: `/` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:20.360342 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 270/541] 10.222.0.13 () {54 vars in 927 bytes} [Wed Aug 31 09:34:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:34:20.375897 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6140f57c-b1c0-4ac0-8b17-f534f697c769 None None] REQUEST_METHOD: `POST` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:20.376180 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6140f57c-b1c0-4ac0-8b17-f534f697c769 None None] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:20.376473 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6140f57c-b1c0-4ac0-8b17-f534f697c769 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:20.397540 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3483a936-4088-4ca6-84d3-233928945f67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:20.398173 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3483a936-4088-4ca6-84d3-233928945f67 None admin] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:20.398447 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3483a936-4088-4ca6-84d3-233928945f67 None admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:20.398782 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3483a936-4088-4ca6-84d3-233928945f67 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:20.399539 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3483a936-4088-4ca6-84d3-233928945f67 None admin] RBAC: Adding query filter params (role.id=e24ba4fd91584883af350cc84bc96c1d, scope.project.id=49548eb44d62415582af820ab66decee, user.id=a9bdaacd597442c2bcde2c555f2cc140) {{(pid=88113) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:34:20.399904 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3483a936-4088-4ca6-84d3-233928945f67 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:20.401401 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3483a936-4088-4ca6-84d3-233928945f67 None admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:20.408835 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 272/542] 10.222.0.13 () {56 vars in 1374 bytes} [Wed Aug 31 09:34:20 2022] GET /identity/v3/role_assignments?user.id=a9bdaacd597442c2bcde2c555f2cc140&scope.project.id=49548eb44d62415582af820ab66decee&role.id=e24ba4fd91584883af350cc84bc96c1d => generated 259 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:34:20.442149 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.auth.core [None req-6140f57c-b1c0-4ac0-8b17-f534f697c769 None None] MFA Rules not processed for user `fd836cf590a145df9acf19cf3ae2ac56`. Rule list: `[]` (Enabled: `True`). {{(pid=88114) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:34:20.443361 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.fernet_utils [None req-6140f57c-b1c0-4ac0-8b17-f534f697c769 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88114) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:34:20.564601 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 271/543] 10.222.0.13 () {58 vars in 1024 bytes} [Wed Aug 31 09:34:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 31 09:34:20.924788 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3483a936-4088-4ca6-84d3-233928945f67 None admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:20.928161 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=100 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:21.054824 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3483a936-4088-4ca6-84d3-233928945f67 None admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:21.097686 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c46e3be-8c1f-4ff8-9bbd-b3e1185db727 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:21.098503 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c46e3be-8c1f-4ff8-9bbd-b3e1185db727 None admin] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:21.098783 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c46e3be-8c1f-4ff8-9bbd-b3e1185db727 None admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:21.099053 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c46e3be-8c1f-4ff8-9bbd-b3e1185db727 None admin] PATH_INFO: `/v3/roles/service` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:21.112364 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c46e3be-8c1f-4ff8-9bbd-b3e1185db727 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 09:34:21.119284 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application [None req-1c46e3be-8c1f-4ff8-9bbd-b3e1185db727 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 31 09:34:21.119284 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:34:21.119284 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:34:21.119284 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:34:21.119284 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:34:21.119284 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:34:21.119284 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:34:21.119284 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:34:21.119284 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:34:21.119284 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:34:21.119284 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:34:21.119284 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:34:21.119284 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 09:34:21.119284 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 09:34:21.119284 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 09:34:21.119284 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 09:34:21.120919 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 09:34:21.120919 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 09:34:21.120919 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 09:34:21.120919 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 09:34:21.120919 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:34:21.120919 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:34:21.120919 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:34:21.120919 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:34:21.120919 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:34:21.120919 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:34:21.120919 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:34:21.120919 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application with Lock( Aug 31 09:34:21.120919 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:34:21.120919 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:34:21.120919 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:34:21.120919 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:34:21.122992 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:34:21.122992 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:34:21.122992 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:34:21.122992 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:34:21.122992 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 09:34:21.122992 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 09:34:21.122992 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 09:34:21.122992 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 09:34:21.122992 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 09:34:21.122992 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 09:34:21.122992 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 31 09:34:21.122992 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application Aug 31 09:34:21.122992 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 273/544] 10.222.0.13 () {56 vars in 1102 bytes} [Wed Aug 31 09:34:20 2022] GET /identity/v3/roles/service => generated 85 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:34:21.127434 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6140f57c-b1c0-4ac0-8b17-f534f697c769 None None] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:21.193982 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=100 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:21.220192 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6d4e00-bdb5-408f-96c5-77d03d72a367 None None] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:21.220621 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6d4e00-bdb5-408f-96c5-77d03d72a367 None None] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:21.220864 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6d4e00-bdb5-408f-96c5-77d03d72a367 None None] PATH_INFO: `/` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:21.222191 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 274/545] 10.222.0.13 () {54 vars in 927 bytes} [Wed Aug 31 09:34:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:34:21.235480 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca144df-4e7a-488e-b988-0285e0f3b44a None None] REQUEST_METHOD: `POST` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:21.235919 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca144df-4e7a-488e-b988-0285e0f3b44a None None] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:21.236166 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca144df-4e7a-488e-b988-0285e0f3b44a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:21.250788 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6140f57c-b1c0-4ac0-8b17-f534f697c769 None None] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:21.291338 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c54bdc-64b2-42a0-9239-0d75144b0267 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:21.292157 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c54bdc-64b2-42a0-9239-0d75144b0267 None admin] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:21.292571 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c54bdc-64b2-42a0-9239-0d75144b0267 None admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:21.292852 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c54bdc-64b2-42a0-9239-0d75144b0267 None admin] PATH_INFO: `/v3/roles` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:21.293580 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45c54bdc-64b2-42a0-9239-0d75144b0267 None admin] RBAC: Adding query filter params (name=service) {{(pid=88114) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:34:21.293883 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45c54bdc-64b2-42a0-9239-0d75144b0267 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:21.295292 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45c54bdc-64b2-42a0-9239-0d75144b0267 None admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:21.306214 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 272/546] 10.222.0.13 () {56 vars in 1103 bytes} [Wed Aug 31 09:34:21 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 09:34:21.307276 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.auth.core [None req-5ca144df-4e7a-488e-b988-0285e0f3b44a None None] MFA Rules not processed for user `fd836cf590a145df9acf19cf3ae2ac56`. Rule list: `[]` (Enabled: `True`). {{(pid=88113) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:34:21.309278 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.fernet_utils [None req-5ca144df-4e7a-488e-b988-0285e0f3b44a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:34:21.311796 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c54bdc-64b2-42a0-9239-0d75144b0267 None admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:21.427985 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c54bdc-64b2-42a0-9239-0d75144b0267 None admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:21.469953 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=100 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:21.471843 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e01262b-8ee0-47c1-be54-3829e72eeb19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:21.474386 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e01262b-8ee0-47c1-be54-3829e72eeb19 None admin] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:21.474919 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e01262b-8ee0-47c1-be54-3829e72eeb19 None admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:21.475186 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e01262b-8ee0-47c1-be54-3829e72eeb19 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:21.502922 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e01262b-8ee0-47c1-be54-3829e72eeb19 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:21.504418 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e01262b-8ee0-47c1-be54-3829e72eeb19 None admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:21.524993 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=100 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:21.531836 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 275/547] 10.222.0.13 () {58 vars in 1024 bytes} [Wed Aug 31 09:34:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 298 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 31 09:34:21.540613 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application [None req-1e01262b-8ee0-47c1-be54-3829e72eeb19 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 09:34:21.540613 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:34:21.540613 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:34:21.540613 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:34:21.540613 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:34:21.540613 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:34:21.540613 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:34:21.540613 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:34:21.540613 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:34:21.540613 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:34:21.540613 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:34:21.540613 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:34:21.540613 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 09:34:21.540613 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 09:34:21.540613 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 09:34:21.540613 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 09:34:21.542422 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:34:21.542422 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:34:21.542422 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:34:21.542422 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:34:21.542422 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:34:21.542422 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:34:21.542422 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:34:21.542422 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application with Lock( Aug 31 09:34:21.542422 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:34:21.542422 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:34:21.542422 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:34:21.542422 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:34:21.542422 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:34:21.542422 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:34:21.542422 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:34:21.542422 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:34:21.542422 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 09:34:21.543810 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 09:34:21.543810 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 09:34:21.543810 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application Aug 31 09:34:21.543810 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 273/548] 10.222.0.13 () {56 vars in 1108 bytes} [Wed Aug 31 09:34:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 09:34:21.548389 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca144df-4e7a-488e-b988-0285e0f3b44a None None] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:21.715076 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca144df-4e7a-488e-b988-0285e0f3b44a None None] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:21.790886 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5486dd19-12ec-40a2-9ba1-8759107e8b8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:21.792319 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5486dd19-12ec-40a2-9ba1-8759107e8b8a None admin] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:21.793078 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5486dd19-12ec-40a2-9ba1-8759107e8b8a None admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:21.793850 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5486dd19-12ec-40a2-9ba1-8759107e8b8a None admin] PATH_INFO: `/v3/domains` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:21.795286 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5486dd19-12ec-40a2-9ba1-8759107e8b8a None admin] RBAC: Adding query filter params (name=Default) {{(pid=88113) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:34:21.796080 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5486dd19-12ec-40a2-9ba1-8759107e8b8a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:21.801431 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5486dd19-12ec-40a2-9ba1-8759107e8b8a None admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:21.831233 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 276/549] 10.222.0.13 () {56 vars in 1109 bytes} [Wed Aug 31 09:34:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 283 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:34:21.836515 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e01262b-8ee0-47c1-be54-3829e72eeb19 None admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:21.997876 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e01262b-8ee0-47c1-be54-3829e72eeb19 None admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:22.086772 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2678cabb-184b-49cb-ae55-e4239c198955 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:22.087775 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2678cabb-184b-49cb-ae55-e4239c198955 None admin] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:22.088255 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2678cabb-184b-49cb-ae55-e4239c198955 None admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:22.088735 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2678cabb-184b-49cb-ae55-e4239c198955 None admin] PATH_INFO: `/v3/users/nova` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:22.107139 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2678cabb-184b-49cb-ae55-e4239c198955 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:22.110508 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2678cabb-184b-49cb-ae55-e4239c198955 None admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:22.125677 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application [None req-2678cabb-184b-49cb-ae55-e4239c198955 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 31 09:34:22.125677 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:34:22.125677 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:34:22.125677 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:34:22.125677 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:34:22.125677 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:34:22.125677 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:34:22.125677 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:34:22.125677 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:34:22.125677 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:34:22.125677 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:34:22.125677 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:34:22.125677 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 31 09:34:22.125677 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 31 09:34:22.125677 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 31 09:34:22.125677 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 31 09:34:22.127648 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:34:22.127648 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:34:22.127648 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 31 09:34:22.127648 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 09:34:22.127648 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 31 09:34:22.127648 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 09:34:22.127648 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:34:22.127648 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:34:22.127648 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:34:22.127648 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:34:22.127648 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:34:22.127648 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application with Lock( Aug 31 09:34:22.127648 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:34:22.127648 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:34:22.127648 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:34:22.127648 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:34:22.127648 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:34:22.127648 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:34:22.129355 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:34:22.129355 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:34:22.129355 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 31 09:34:22.129355 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 31 09:34:22.129355 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 31 09:34:22.129355 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 31 09:34:22.129355 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 31 09:34:22.129355 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 31 09:34:22.129355 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 31 09:34:22.129355 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application Aug 31 09:34:22.129355 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 274/550] 10.222.0.13 () {56 vars in 1093 bytes} [Wed Aug 31 09:34:21 2022] GET /identity/v3/users/nova => generated 82 bytes in 291 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 09:34:22.133248 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5486dd19-12ec-40a2-9ba1-8759107e8b8a None admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:22.151559 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2678cabb-184b-49cb-ae55-e4239c198955 None admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:22.196766 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=100 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:22.268591 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2678cabb-184b-49cb-ae55-e4239c198955 None admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:22.292920 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=100 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:22.310744 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2218eee1-d5d3-43aa-9d7d-ce2a590813dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:22.312062 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2218eee1-d5d3-43aa-9d7d-ce2a590813dc None admin] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:22.312573 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2218eee1-d5d3-43aa-9d7d-ce2a590813dc None admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:22.313040 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2218eee1-d5d3-43aa-9d7d-ce2a590813dc None admin] PATH_INFO: `/v3/roles/service` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:22.325173 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2218eee1-d5d3-43aa-9d7d-ce2a590813dc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 09:34:22.332642 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application [None req-2218eee1-d5d3-43aa-9d7d-ce2a590813dc None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 31 09:34:22.332642 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:34:22.332642 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:34:22.332642 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:34:22.332642 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:34:22.332642 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:34:22.332642 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:34:22.332642 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:34:22.332642 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:34:22.332642 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:34:22.332642 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:34:22.332642 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:34:22.332642 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 09:34:22.332642 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 09:34:22.332642 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 09:34:22.332642 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 09:34:22.333801 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 09:34:22.333801 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 09:34:22.333801 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 09:34:22.333801 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 09:34:22.333801 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:34:22.333801 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:34:22.333801 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:34:22.333801 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:34:22.333801 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:34:22.333801 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:34:22.333801 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:34:22.333801 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application with Lock( Aug 31 09:34:22.333801 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:34:22.333801 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:34:22.333801 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:34:22.333801 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:34:22.334625 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:34:22.334625 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:34:22.334625 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:34:22.334625 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:34:22.334625 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 09:34:22.334625 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 09:34:22.334625 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 09:34:22.334625 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 09:34:22.334625 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 09:34:22.334625 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 09:34:22.334625 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 31 09:34:22.334625 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application Aug 31 09:34:22.337588 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 275/551] 10.222.0.13 () {56 vars in 1102 bytes} [Wed Aug 31 09:34:22 2022] GET /identity/v3/roles/service => generated 85 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 09:34:22.339091 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43410c42-2816-435f-be99-ea388db7a32f None None] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:22.340225 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43410c42-2816-435f-be99-ea388db7a32f None None] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:22.340745 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43410c42-2816-435f-be99-ea388db7a32f None None] PATH_INFO: `/` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:22.342559 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 276/552] 10.222.0.13 () {54 vars in 927 bytes} [Wed Aug 31 09:34:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:34:22.344982 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43410c42-2816-435f-be99-ea388db7a32f None None] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:22.401718 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5486dd19-12ec-40a2-9ba1-8759107e8b8a None admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:22.457923 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-402ce5f1-fb59-4349-8278-02a31bfbdaf7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:22.459373 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402ce5f1-fb59-4349-8278-02a31bfbdaf7 None admin] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:22.459838 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402ce5f1-fb59-4349-8278-02a31bfbdaf7 None admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:22.460361 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402ce5f1-fb59-4349-8278-02a31bfbdaf7 None admin] PATH_INFO: `/v3/users` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:22.461959 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-402ce5f1-fb59-4349-8278-02a31bfbdaf7 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=88113) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:34:22.462448 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-402ce5f1-fb59-4349-8278-02a31bfbdaf7 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:22.465109 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-402ce5f1-fb59-4349-8278-02a31bfbdaf7 None admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:22.494128 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 277/553] 10.222.0.13 () {56 vars in 1133 bytes} [Wed Aug 31 09:34:22 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 362 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:34:22.496225 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154a269e-3a36-422a-bed3-c71bdf02a376 None None] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:22.496708 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154a269e-3a36-422a-bed3-c71bdf02a376 None None] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:22.497179 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154a269e-3a36-422a-bed3-c71bdf02a376 None None] PATH_INFO: `/` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:22.499742 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 278/554] 10.222.0.13 () {54 vars in 927 bytes} [Wed Aug 31 09:34:22 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:34:22.505449 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b4d39d-a2a9-4cab-b7d5-4ef6624a9b3b None None] REQUEST_METHOD: `POST` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:22.506232 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b4d39d-a2a9-4cab-b7d5-4ef6624a9b3b None None] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:22.506717 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b4d39d-a2a9-4cab-b7d5-4ef6624a9b3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:22.512024 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=100 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:22.533582 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.auth.core [None req-07b4d39d-a2a9-4cab-b7d5-4ef6624a9b3b None None] MFA Rules not processed for user `fd836cf590a145df9acf19cf3ae2ac56`. Rule list: `[]` (Enabled: `True`). {{(pid=88113) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:34:22.533845 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43410c42-2816-435f-be99-ea388db7a32f None None] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:22.534893 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.fernet_utils [None req-07b4d39d-a2a9-4cab-b7d5-4ef6624a9b3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:34:22.590804 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4430f597-384d-4744-b9ee-e8b3b0cccb30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:22.591769 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4430f597-384d-4744-b9ee-e8b3b0cccb30 None admin] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:22.592182 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4430f597-384d-4744-b9ee-e8b3b0cccb30 None admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:22.592643 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4430f597-384d-4744-b9ee-e8b3b0cccb30 None admin] PATH_INFO: `/v3/roles` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:22.594074 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4430f597-384d-4744-b9ee-e8b3b0cccb30 None admin] RBAC: Adding query filter params (name=service) {{(pid=88114) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:34:22.594534 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4430f597-384d-4744-b9ee-e8b3b0cccb30 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:22.596213 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4430f597-384d-4744-b9ee-e8b3b0cccb30 None admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:22.613986 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 277/555] 10.222.0.13 () {56 vars in 1103 bytes} [Wed Aug 31 09:34:22 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:34:22.616787 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4430f597-384d-4744-b9ee-e8b3b0cccb30 None admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:22.663497 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 279/556] 10.222.0.13 () {58 vars in 1024 bytes} [Wed Aug 31 09:34:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 09:34:22.665727 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0763d0b-1f43-45bf-9483-b9fdbe567f3c None None] REQUEST_METHOD: `POST` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:22.666211 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0763d0b-1f43-45bf-9483-b9fdbe567f3c None None] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:22.666660 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0763d0b-1f43-45bf-9483-b9fdbe567f3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:22.699175 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.auth.core [None req-c0763d0b-1f43-45bf-9483-b9fdbe567f3c None None] MFA Rules not processed for user `fd836cf590a145df9acf19cf3ae2ac56`. Rule list: `[]` (Enabled: `True`). {{(pid=88113) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:34:22.701258 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.fernet_utils [None req-c0763d0b-1f43-45bf-9483-b9fdbe567f3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:34:22.771424 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=100 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:22.821237 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4430f597-384d-4744-b9ee-e8b3b0cccb30 None admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:22.849079 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 280/557] 10.222.0.13 () {58 vars in 1024 bytes} [Wed Aug 31 09:34:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 09:34:22.850987 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0763d0b-1f43-45bf-9483-b9fdbe567f3c None None] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:22.964208 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f03618a2-29e9-4c9b-96e8-26061c33bdca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:22.965367 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03618a2-29e9-4c9b-96e8-26061c33bdca None admin] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:22.965759 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03618a2-29e9-4c9b-96e8-26061c33bdca None admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:22.966245 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03618a2-29e9-4c9b-96e8-26061c33bdca None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:23.015773 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f03618a2-29e9-4c9b-96e8-26061c33bdca None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:23.019175 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f03618a2-29e9-4c9b-96e8-26061c33bdca None admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:23.058311 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application [None req-f03618a2-29e9-4c9b-96e8-26061c33bdca None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 09:34:23.058311 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:34:23.058311 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:34:23.058311 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:34:23.058311 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:34:23.058311 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:34:23.058311 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:34:23.058311 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:34:23.058311 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:34:23.058311 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:34:23.058311 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:34:23.058311 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:34:23.058311 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 09:34:23.058311 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 09:34:23.058311 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 09:34:23.058311 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 09:34:23.059927 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:34:23.059927 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:34:23.059927 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:34:23.059927 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:34:23.059927 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:34:23.059927 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:34:23.059927 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:34:23.059927 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application with Lock( Aug 31 09:34:23.059927 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:34:23.059927 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:34:23.059927 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:34:23.059927 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:34:23.059927 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:34:23.059927 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:34:23.059927 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:34:23.059927 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:34:23.059927 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 09:34:23.061678 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 09:34:23.061678 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 09:34:23.061678 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application Aug 31 09:34:23.061678 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 278/558] 10.222.0.13 () {56 vars in 1108 bytes} [Wed Aug 31 09:34:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 445 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:34:23.071324 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f03618a2-29e9-4c9b-96e8-26061c33bdca None admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:23.077118 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0763d0b-1f43-45bf-9483-b9fdbe567f3c None None] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:23.079889 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=100 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:23.141668 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c3b01c-733c-4805-b19d-61a4f72c247b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:23.143699 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c3b01c-733c-4805-b19d-61a4f72c247b None admin] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:23.144828 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c3b01c-733c-4805-b19d-61a4f72c247b None admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:23.145595 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c3b01c-733c-4805-b19d-61a4f72c247b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:23.165510 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8c3b01c-733c-4805-b19d-61a4f72c247b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:23.168657 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8c3b01c-733c-4805-b19d-61a4f72c247b None admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:23.196446 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application [None req-a8c3b01c-733c-4805-b19d-61a4f72c247b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 09:34:23.196446 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:34:23.196446 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:34:23.196446 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:34:23.196446 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:34:23.196446 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:34:23.196446 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:34:23.196446 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:34:23.196446 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:34:23.196446 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:34:23.196446 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:34:23.196446 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:34:23.196446 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 09:34:23.196446 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 09:34:23.196446 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 09:34:23.196446 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 09:34:23.198893 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:34:23.198893 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:34:23.198893 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:34:23.198893 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:34:23.198893 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:34:23.198893 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:34:23.198893 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:34:23.198893 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application with Lock( Aug 31 09:34:23.198893 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:34:23.198893 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:34:23.198893 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:34:23.198893 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:34:23.198893 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:34:23.198893 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:34:23.198893 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:34:23.198893 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:34:23.198893 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 09:34:23.200553 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 09:34:23.200553 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 09:34:23.200553 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application Aug 31 09:34:23.204023 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 281/559] 10.222.0.13 () {56 vars in 1108 bytes} [Wed Aug 31 09:34:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 354 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:34:23.210885 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c3b01c-733c-4805-b19d-61a4f72c247b None admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:23.259593 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=100 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:23.279025 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f03618a2-29e9-4c9b-96e8-26061c33bdca None admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:23.362004 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea0a610-e78d-4e72-9fe6-2aaef92623ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:23.363249 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea0a610-e78d-4e72-9fe6-2aaef92623ed None admin] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:23.363721 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea0a610-e78d-4e72-9fe6-2aaef92623ed None admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:23.364936 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea0a610-e78d-4e72-9fe6-2aaef92623ed None admin] PATH_INFO: `/v3/domains` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:23.366206 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aea0a610-e78d-4e72-9fe6-2aaef92623ed None admin] RBAC: Adding query filter params (name=Default) {{(pid=88114) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:34:23.366789 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aea0a610-e78d-4e72-9fe6-2aaef92623ed None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:23.370271 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aea0a610-e78d-4e72-9fe6-2aaef92623ed None admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:23.401686 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 279/560] 10.222.0.13 () {56 vars in 1109 bytes} [Wed Aug 31 09:34:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 332 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 09:34:23.402687 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea0a610-e78d-4e72-9fe6-2aaef92623ed None admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:23.414653 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c3b01c-733c-4805-b19d-61a4f72c247b None admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:23.479021 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=100 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:23.480315 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5605ee0-8ec8-4326-b556-79b7c82cb342 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:23.481324 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5605ee0-8ec8-4326-b556-79b7c82cb342 None admin] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:23.481827 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5605ee0-8ec8-4326-b556-79b7c82cb342 None admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:23.482279 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5605ee0-8ec8-4326-b556-79b7c82cb342 None admin] PATH_INFO: `/v3/roles/service` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:23.496160 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5605ee0-8ec8-4326-b556-79b7c82cb342 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 09:34:23.502191 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application [None req-c5605ee0-8ec8-4326-b556-79b7c82cb342 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 31 09:34:23.502191 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:34:23.502191 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:34:23.502191 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:34:23.502191 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:34:23.502191 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:34:23.502191 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:34:23.502191 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:34:23.502191 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:34:23.502191 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:34:23.502191 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:34:23.502191 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:34:23.502191 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 09:34:23.502191 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 09:34:23.502191 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 09:34:23.502191 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 09:34:23.504017 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 09:34:23.504017 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 09:34:23.504017 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 09:34:23.504017 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 09:34:23.504017 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:34:23.504017 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:34:23.504017 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:34:23.504017 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:34:23.504017 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:34:23.504017 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:34:23.504017 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:34:23.504017 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application with Lock( Aug 31 09:34:23.504017 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:34:23.504017 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:34:23.504017 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:34:23.504017 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:34:23.505666 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:34:23.505666 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:34:23.505666 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:34:23.505666 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:34:23.505666 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 09:34:23.505666 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 09:34:23.505666 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 09:34:23.505666 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 09:34:23.505666 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 09:34:23.505666 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 09:34:23.505666 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 31 09:34:23.505666 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application Aug 31 09:34:23.505666 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 282/561] 10.222.0.13 () {56 vars in 1102 bytes} [Wed Aug 31 09:34:23 2022] GET /identity/v3/roles/service => generated 85 bytes in 299 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:34:23.505666 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b4de2f-9be5-4ddd-9ee9-2dd2bc385b82 None None] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:23.507152 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b4de2f-9be5-4ddd-9ee9-2dd2bc385b82 None None] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:23.507152 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b4de2f-9be5-4ddd-9ee9-2dd2bc385b82 None None] PATH_INFO: `/` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:23.507152 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 283/562] 10.222.0.13 () {54 vars in 927 bytes} [Wed Aug 31 09:34:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:34:23.507660 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b4de2f-9be5-4ddd-9ee9-2dd2bc385b82 None None] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:23.544685 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea0a610-e78d-4e72-9fe6-2aaef92623ed None admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:23.593142 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c534a4d-9b6c-4d2c-8525-35bdcec700da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:23.594479 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c534a4d-9b6c-4d2c-8525-35bdcec700da None admin] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:23.594852 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c534a4d-9b6c-4d2c-8525-35bdcec700da None admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:23.595262 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c534a4d-9b6c-4d2c-8525-35bdcec700da None admin] PATH_INFO: `/v3/domains` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:23.596034 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c534a4d-9b6c-4d2c-8525-35bdcec700da None admin] RBAC: Adding query filter params (name=Default) {{(pid=88114) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:34:23.596396 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c534a4d-9b6c-4d2c-8525-35bdcec700da None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:23.598476 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c534a4d-9b6c-4d2c-8525-35bdcec700da None admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:23.612387 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=100 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:23.612654 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 280/563] 10.222.0.13 () {56 vars in 1109 bytes} [Wed Aug 31 09:34:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:34:23.613491 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c534a4d-9b6c-4d2c-8525-35bdcec700da None admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:23.642553 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b4de2f-9be5-4ddd-9ee9-2dd2bc385b82 None None] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:23.680710 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e1a5b7-c8df-453b-a1c4-7c2bc27e67c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:23.681262 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e1a5b7-c8df-453b-a1c4-7c2bc27e67c2 None admin] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:23.681575 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e1a5b7-c8df-453b-a1c4-7c2bc27e67c2 None admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:23.681846 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e1a5b7-c8df-453b-a1c4-7c2bc27e67c2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:23.689509 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9e1a5b7-c8df-453b-a1c4-7c2bc27e67c2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:23.690983 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9e1a5b7-c8df-453b-a1c4-7c2bc27e67c2 None admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:23.702809 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application [None req-f9e1a5b7-c8df-453b-a1c4-7c2bc27e67c2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 09:34:23.702809 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:34:23.702809 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:34:23.702809 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:34:23.702809 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:34:23.702809 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:34:23.702809 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:34:23.702809 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:34:23.702809 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:34:23.702809 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:34:23.702809 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:34:23.702809 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:34:23.702809 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 31 09:34:23.702809 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 31 09:34:23.702809 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 31 09:34:23.703948 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 31 09:34:23.703948 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:34:23.703948 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:34:23.703948 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:34:23.703948 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:34:23.703948 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:34:23.703948 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:34:23.703948 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:34:23.703948 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application with Lock( Aug 31 09:34:23.703948 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:34:23.703948 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:34:23.703948 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:34:23.703948 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:34:23.703948 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:34:23.703948 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:34:23.703948 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:34:23.703948 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:34:23.704898 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 31 09:34:23.704898 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 31 09:34:23.704898 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 31 09:34:23.704898 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 31 09:34:23.704898 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 31 09:34:23.704898 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 31 09:34:23.704898 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 09:34:23.704898 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application Aug 31 09:34:23.704898 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 284/564] 10.222.0.13 () {56 vars in 1111 bytes} [Wed Aug 31 09:34:23 2022] GET /identity/v3/projects/service => generated 88 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:34:23.704898 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e1a5b7-c8df-453b-a1c4-7c2bc27e67c2 None admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:23.766570 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c534a4d-9b6c-4d2c-8525-35bdcec700da None admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:23.804595 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=100 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:23.818474 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d99df3d8-d92d-4dd4-8fe6-28708923cf0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:23.819115 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99df3d8-d92d-4dd4-8fe6-28708923cf0a None admin] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:23.819393 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99df3d8-d92d-4dd4-8fe6-28708923cf0a None admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:23.819552 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e1a5b7-c8df-453b-a1c4-7c2bc27e67c2 None admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:23.819718 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99df3d8-d92d-4dd4-8fe6-28708923cf0a None admin] PATH_INFO: `/v3/roles/service` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:23.832092 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d99df3d8-d92d-4dd4-8fe6-28708923cf0a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 09:34:23.841956 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application [None req-d99df3d8-d92d-4dd4-8fe6-28708923cf0a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 31 09:34:23.841956 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:34:23.841956 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:34:23.841956 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:34:23.841956 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:34:23.841956 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:34:23.841956 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:34:23.841956 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:34:23.841956 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:34:23.841956 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:34:23.841956 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:34:23.841956 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:34:23.841956 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 09:34:23.841956 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 09:34:23.841956 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 09:34:23.841956 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 09:34:23.843923 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 09:34:23.843923 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 09:34:23.843923 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 09:34:23.843923 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 09:34:23.843923 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:34:23.843923 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:34:23.843923 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:34:23.843923 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:34:23.843923 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:34:23.843923 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:34:23.843923 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:34:23.843923 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application with Lock( Aug 31 09:34:23.843923 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:34:23.843923 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:34:23.843923 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:34:23.843923 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:34:23.846164 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:34:23.846164 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:34:23.846164 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:34:23.846164 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:34:23.846164 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 09:34:23.846164 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 09:34:23.846164 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 09:34:23.846164 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 09:34:23.846164 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 09:34:23.846164 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 09:34:23.846164 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 31 09:34:23.846164 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application Aug 31 09:34:23.846164 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 281/565] 10.222.0.13 () {56 vars in 1102 bytes} [Wed Aug 31 09:34:23 2022] GET /identity/v3/roles/service => generated 85 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:34:23.846164 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb8f0d6-6cd1-4380-9042-995c94682362 None None] REQUEST_METHOD: `POST` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:23.848417 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb8f0d6-6cd1-4380-9042-995c94682362 None None] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:23.848417 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb8f0d6-6cd1-4380-9042-995c94682362 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:23.869690 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.auth.core [None req-1cb8f0d6-6cd1-4380-9042-995c94682362 None None] MFA Rules not processed for user `fd836cf590a145df9acf19cf3ae2ac56`. Rule list: `[]` (Enabled: `True`). {{(pid=88114) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:34:23.871156 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffabf6ec-487b-44a5-aa5a-d90db9d898a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:23.871747 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.fernet_utils [None req-1cb8f0d6-6cd1-4380-9042-995c94682362 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88114) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:34:23.871868 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffabf6ec-487b-44a5-aa5a-d90db9d898a4 None admin] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:23.872040 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffabf6ec-487b-44a5-aa5a-d90db9d898a4 None admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:23.872313 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffabf6ec-487b-44a5-aa5a-d90db9d898a4 None admin] PATH_INFO: `/v3/roles` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:23.872998 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffabf6ec-487b-44a5-aa5a-d90db9d898a4 None admin] RBAC: Adding query filter params (name=service) {{(pid=88113) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:34:23.873305 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffabf6ec-487b-44a5-aa5a-d90db9d898a4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:23.874792 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffabf6ec-487b-44a5-aa5a-d90db9d898a4 None admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:23.883550 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 285/566] 10.222.0.13 () {56 vars in 1103 bytes} [Wed Aug 31 09:34:23 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:34:23.884405 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffabf6ec-487b-44a5-aa5a-d90db9d898a4 None admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:23.889945 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=100 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:23.989482 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 282/567] 10.222.0.13 () {58 vars in 1024 bytes} [Wed Aug 31 09:34:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 09:34:23.990357 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb8f0d6-6cd1-4380-9042-995c94682362 None None] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:24.008380 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffabf6ec-487b-44a5-aa5a-d90db9d898a4 None admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:24.058206 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d34869-6ce4-4692-808c-d4b147615232 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:24.059293 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d34869-6ce4-4692-808c-d4b147615232 None admin] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:24.059748 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d34869-6ce4-4692-808c-d4b147615232 None admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:24.060215 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d34869-6ce4-4692-808c-d4b147615232 None admin] PATH_INFO: `/v3/users/swift` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:24.074927 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63d34869-6ce4-4692-808c-d4b147615232 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:24.077582 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63d34869-6ce4-4692-808c-d4b147615232 None admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:24.100238 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application [None req-63d34869-6ce4-4692-808c-d4b147615232 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 31 09:34:24.100238 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:34:24.100238 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:34:24.100238 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:34:24.100238 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:34:24.100238 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:34:24.100238 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:34:24.100238 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:34:24.100238 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:34:24.100238 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:34:24.100238 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:34:24.100238 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:34:24.100238 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 31 09:34:24.100238 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 31 09:34:24.100238 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 31 09:34:24.100238 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 31 09:34:24.102306 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:34:24.102306 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:34:24.102306 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 31 09:34:24.102306 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 09:34:24.102306 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 31 09:34:24.102306 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 09:34:24.102306 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:34:24.102306 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:34:24.102306 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:34:24.102306 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:34:24.102306 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:34:24.102306 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application with Lock( Aug 31 09:34:24.102306 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:34:24.102306 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:34:24.102306 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:34:24.102306 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:34:24.102306 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:34:24.102306 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:34:24.104526 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:34:24.104526 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:34:24.104526 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 31 09:34:24.104526 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 31 09:34:24.104526 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 31 09:34:24.104526 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 31 09:34:24.104526 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 31 09:34:24.104526 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 31 09:34:24.104526 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 31 09:34:24.104526 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application Aug 31 09:34:24.110516 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 286/568] 10.222.0.13 () {56 vars in 1096 bytes} [Wed Aug 31 09:34:23 2022] GET /identity/v3/users/swift => generated 83 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:34:24.111563 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d34869-6ce4-4692-808c-d4b147615232 None admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:24.177852 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb8f0d6-6cd1-4380-9042-995c94682362 None None] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:24.195857 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=100 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:24.200416 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=100 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:24.235070 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-569ce5ba-49fe-4ded-9c43-5b6f0f8956c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:24.236009 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569ce5ba-49fe-4ded-9c43-5b6f0f8956c7 None admin] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:24.236499 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569ce5ba-49fe-4ded-9c43-5b6f0f8956c7 None admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:24.236996 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569ce5ba-49fe-4ded-9c43-5b6f0f8956c7 None admin] PATH_INFO: `/v3/projects` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:24.238352 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-569ce5ba-49fe-4ded-9c43-5b6f0f8956c7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88114) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:34:24.238907 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-569ce5ba-49fe-4ded-9c43-5b6f0f8956c7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:24.241587 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-569ce5ba-49fe-4ded-9c43-5b6f0f8956c7 None admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:24.248772 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d34869-6ce4-4692-808c-d4b147615232 None admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:24.263278 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 283/569] 10.222.0.13 () {56 vars in 1148 bytes} [Wed Aug 31 09:34:23 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:34:24.264855 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-569ce5ba-49fe-4ded-9c43-5b6f0f8956c7 None admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:24.306552 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f47a31b6-37af-46f9-8b80-902091f8f543 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:24.307199 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47a31b6-37af-46f9-8b80-902091f8f543 None admin] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:24.307475 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47a31b6-37af-46f9-8b80-902091f8f543 None admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:24.308264 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47a31b6-37af-46f9-8b80-902091f8f543 None admin] PATH_INFO: `/v3/roles` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:24.308542 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f47a31b6-37af-46f9-8b80-902091f8f543 None admin] RBAC: Adding query filter params (name=service) {{(pid=88113) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:34:24.308764 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f47a31b6-37af-46f9-8b80-902091f8f543 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:24.310286 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f47a31b6-37af-46f9-8b80-902091f8f543 None admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:24.326399 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 287/570] 10.222.0.13 () {56 vars in 1103 bytes} [Wed Aug 31 09:34:24 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:34:24.327446 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f47a31b6-37af-46f9-8b80-902091f8f543 None admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:24.412150 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-569ce5ba-49fe-4ded-9c43-5b6f0f8956c7 None admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:24.451999 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e411916-4a2c-4c0c-82da-98bbcacb376b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:24.452754 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e411916-4a2c-4c0c-82da-98bbcacb376b None admin] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:24.453106 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e411916-4a2c-4c0c-82da-98bbcacb376b None admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:24.453372 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e411916-4a2c-4c0c-82da-98bbcacb376b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:24.465101 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e411916-4a2c-4c0c-82da-98bbcacb376b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:24.466589 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e411916-4a2c-4c0c-82da-98bbcacb376b None admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:24.479574 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f47a31b6-37af-46f9-8b80-902091f8f543 None admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:24.481873 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application [None req-2e411916-4a2c-4c0c-82da-98bbcacb376b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 09:34:24.481873 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:34:24.481873 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:34:24.481873 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:34:24.481873 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:34:24.481873 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:34:24.481873 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:34:24.481873 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:34:24.481873 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:34:24.481873 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:34:24.481873 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:34:24.481873 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:34:24.481873 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 09:34:24.481873 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 09:34:24.481873 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 09:34:24.481873 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 09:34:24.484340 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:34:24.484340 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:34:24.484340 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:34:24.484340 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:34:24.484340 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:34:24.484340 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:34:24.484340 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:34:24.484340 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application with Lock( Aug 31 09:34:24.484340 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:34:24.484340 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:34:24.484340 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:34:24.484340 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:34:24.484340 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:34:24.484340 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:34:24.484340 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:34:24.484340 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:34:24.484340 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 09:34:24.485974 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 09:34:24.485974 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 09:34:24.485974 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application Aug 31 09:34:24.485974 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 284/571] 10.222.0.13 () {56 vars in 1108 bytes} [Wed Aug 31 09:34:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:34:24.485974 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e411916-4a2c-4c0c-82da-98bbcacb376b None admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:24.521615 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=100 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:24.526715 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=100 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:24.538415 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d66cbc-44b3-4bca-9ecf-aad1c3417578 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:24.539035 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d66cbc-44b3-4bca-9ecf-aad1c3417578 None admin] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:24.539307 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d66cbc-44b3-4bca-9ecf-aad1c3417578 None admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:24.539582 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d66cbc-44b3-4bca-9ecf-aad1c3417578 None admin] PATH_INFO: `/v3/users` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:24.540336 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92d66cbc-44b3-4bca-9ecf-aad1c3417578 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=88113) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:34:24.540624 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92d66cbc-44b3-4bca-9ecf-aad1c3417578 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:24.542142 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92d66cbc-44b3-4bca-9ecf-aad1c3417578 None admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:24.566405 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 288/572] 10.222.0.13 () {56 vars in 1135 bytes} [Wed Aug 31 09:34:24 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:34:24.567016 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d66cbc-44b3-4bca-9ecf-aad1c3417578 None admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:24.641242 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e411916-4a2c-4c0c-82da-98bbcacb376b None admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:24.688178 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d66cbc-44b3-4bca-9ecf-aad1c3417578 None admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:24.703133 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-239bcd24-6b0e-4539-9611-7c7d2fea1017 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:24.704196 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239bcd24-6b0e-4539-9611-7c7d2fea1017 None admin] REQUEST_METHOD: `PUT` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:24.704639 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239bcd24-6b0e-4539-9611-7c7d2fea1017 None admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:24.705079 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239bcd24-6b0e-4539-9611-7c7d2fea1017 None admin] PATH_INFO: `/v3/projects/49548eb44d62415582af820ab66decee/users/dc076493239d4ccdbdc37c0369c49529/roles/e24ba4fd91584883af350cc84bc96c1d` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:24.727246 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7049de0-ea80-4fb8-8e8b-3e7a97e65765 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:24.728250 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7049de0-ea80-4fb8-8e8b-3e7a97e65765 None admin] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:24.728692 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7049de0-ea80-4fb8-8e8b-3e7a97e65765 None admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:24.729114 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7049de0-ea80-4fb8-8e8b-3e7a97e65765 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:24.729369 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-239bcd24-6b0e-4539-9611-7c7d2fea1017 None admin] RBAC: Authorizing `identity:create_grant(project_id=49548eb44d62415582af820ab66decee, user_id=dc076493239d4ccdbdc37c0369c49529, role_id=e24ba4fd91584883af350cc84bc96c1d)` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:24.731938 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-239bcd24-6b0e-4539-9611-7c7d2fea1017 None admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:24.744788 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7049de0-ea80-4fb8-8e8b-3e7a97e65765 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:24.746261 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7049de0-ea80-4fb8-8e8b-3e7a97e65765 None admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:24.752018 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 285/573] 10.222.0.13 () {58 vars in 1439 bytes} [Wed Aug 31 09:34:24 2022] PUT /identity/v3/projects/49548eb44d62415582af820ab66decee/users/dc076493239d4ccdbdc37c0369c49529/roles/e24ba4fd91584883af350cc84bc96c1d => generated 0 bytes in 268 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:34:24.753451 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-239bcd24-6b0e-4539-9611-7c7d2fea1017 None admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:24.765487 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application [None req-f7049de0-ea80-4fb8-8e8b-3e7a97e65765 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 09:34:24.765487 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:34:24.765487 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:34:24.765487 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:34:24.765487 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:34:24.765487 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:34:24.765487 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:34:24.765487 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:34:24.765487 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:34:24.765487 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:34:24.765487 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:34:24.765487 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:34:24.765487 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 09:34:24.765487 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 09:34:24.765487 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 09:34:24.765487 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 09:34:24.775825 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:34:24.775825 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:34:24.775825 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:34:24.775825 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:34:24.775825 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:34:24.775825 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:34:24.775825 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:34:24.775825 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application with Lock( Aug 31 09:34:24.775825 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:34:24.775825 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:34:24.775825 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:34:24.775825 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:34:24.775825 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:34:24.775825 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:34:24.775825 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:34:24.775825 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:34:24.775825 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 09:34:24.777563 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 09:34:24.777563 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 09:34:24.777563 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application Aug 31 09:34:24.777563 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 289/574] 10.222.0.13 () {56 vars in 1108 bytes} [Wed Aug 31 09:34:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:34:24.777563 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7049de0-ea80-4fb8-8e8b-3e7a97e65765 None admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:24.825946 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=100 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:24.889543 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:24.930980 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7049de0-ea80-4fb8-8e8b-3e7a97e65765 None admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:24.942523 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-239bcd24-6b0e-4539-9611-7c7d2fea1017 None admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:24.990637 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab45da3-4ab7-4436-b7e4-97ed31aa928a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:24.991603 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab45da3-4ab7-4436-b7e4-97ed31aa928a None admin] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:24.992105 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab45da3-4ab7-4436-b7e4-97ed31aa928a None admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:24.992701 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab45da3-4ab7-4436-b7e4-97ed31aa928a None admin] PATH_INFO: `/v3/domains` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:24.993654 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ab45da3-4ab7-4436-b7e4-97ed31aa928a None admin] RBAC: Adding query filter params (name=Default) {{(pid=88113) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:34:24.994340 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ab45da3-4ab7-4436-b7e4-97ed31aa928a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:24.996088 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ab45da3-4ab7-4436-b7e4-97ed31aa928a None admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:25.013351 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 290/575] 10.222.0.13 () {56 vars in 1109 bytes} [Wed Aug 31 09:34:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:34:25.014367 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab45da3-4ab7-4436-b7e4-97ed31aa928a None admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:25.019190 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e381d84-8104-4eca-996a-127772793811 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:25.021910 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e381d84-8104-4eca-996a-127772793811 None admin] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:25.022156 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e381d84-8104-4eca-996a-127772793811 None admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:25.022596 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e381d84-8104-4eca-996a-127772793811 None admin] PATH_INFO: `/v3/roles/service` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:25.036854 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e381d84-8104-4eca-996a-127772793811 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 09:34:25.042808 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application [None req-5e381d84-8104-4eca-996a-127772793811 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 31 09:34:25.042808 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:34:25.042808 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:34:25.042808 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:34:25.042808 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:34:25.042808 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:34:25.042808 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:34:25.042808 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:34:25.042808 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:34:25.042808 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:34:25.042808 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:34:25.042808 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:34:25.042808 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 09:34:25.042808 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 09:34:25.042808 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 09:34:25.042808 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 09:34:25.044658 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 09:34:25.044658 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 09:34:25.044658 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 09:34:25.044658 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 09:34:25.044658 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:34:25.044658 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:34:25.044658 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:34:25.044658 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:34:25.044658 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:34:25.044658 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:34:25.044658 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:34:25.044658 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application with Lock( Aug 31 09:34:25.044658 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:34:25.044658 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:34:25.044658 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:34:25.044658 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:34:25.048950 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:34:25.048950 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:34:25.048950 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:34:25.048950 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:34:25.048950 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 09:34:25.048950 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 09:34:25.048950 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 09:34:25.048950 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 09:34:25.048950 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 09:34:25.048950 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 09:34:25.048950 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 31 09:34:25.048950 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application Aug 31 09:34:25.048950 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 286/576] 10.222.0.13 () {56 vars in 1102 bytes} [Wed Aug 31 09:34:24 2022] GET /identity/v3/roles/service => generated 85 bytes in 292 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:34:25.048950 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e381d84-8104-4eca-996a-127772793811 None admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:25.191156 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=100 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:25.201732 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab45da3-4ab7-4436-b7e4-97ed31aa928a None admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:25.224460 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e381d84-8104-4eca-996a-127772793811 None admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:25.267502 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a574b28-e818-4b7f-9573-2f682d7e2ce1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:25.268407 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a574b28-e818-4b7f-9573-2f682d7e2ce1 None admin] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:25.268888 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a574b28-e818-4b7f-9573-2f682d7e2ce1 None admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:25.269643 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a574b28-e818-4b7f-9573-2f682d7e2ce1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:25.272795 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f968971-5202-4fd0-ac50-2373f72c9078 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:25.273588 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f968971-5202-4fd0-ac50-2373f72c9078 None admin] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:25.273966 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f968971-5202-4fd0-ac50-2373f72c9078 None admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:25.274365 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f968971-5202-4fd0-ac50-2373f72c9078 None admin] PATH_INFO: `/v3/domains` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:25.275426 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f968971-5202-4fd0-ac50-2373f72c9078 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88114) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:34:25.275842 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f968971-5202-4fd0-ac50-2373f72c9078 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:25.277534 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f968971-5202-4fd0-ac50-2373f72c9078 None admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:25.282289 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=100 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:25.299545 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 287/577] 10.222.0.13 () {56 vars in 1109 bytes} [Wed Aug 31 09:34:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:34:25.301388 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f968971-5202-4fd0-ac50-2373f72c9078 None admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:25.399323 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a574b28-e818-4b7f-9573-2f682d7e2ce1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:25.402103 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a574b28-e818-4b7f-9573-2f682d7e2ce1 None admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:25.485843 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f968971-5202-4fd0-ac50-2373f72c9078 None admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:25.548743 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb7a490-c2ae-424d-8fba-0975eeb43353 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:25.549778 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb7a490-c2ae-424d-8fba-0975eeb43353 None admin] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:25.550252 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb7a490-c2ae-424d-8fba-0975eeb43353 None admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:25.550705 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb7a490-c2ae-424d-8fba-0975eeb43353 None admin] PATH_INFO: `/v3/users/glance` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:25.565328 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bb7a490-c2ae-424d-8fba-0975eeb43353 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:25.568095 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bb7a490-c2ae-424d-8fba-0975eeb43353 None admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:25.577255 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=100 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:25.595065 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 291/578] 10.222.0.13 () {58 vars in 1282 bytes} [Wed Aug 31 09:34:25 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 576 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 31 09:34:25.595065 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a574b28-e818-4b7f-9573-2f682d7e2ce1 None admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:25.595374 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application [None req-1bb7a490-c2ae-424d-8fba-0975eeb43353 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Aug 31 09:34:25.595374 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:34:25.595374 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:34:25.595374 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:34:25.595374 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:34:25.595374 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:34:25.595374 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:34:25.595374 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:34:25.595374 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:34:25.595374 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:34:25.595374 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:34:25.595374 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:34:25.595374 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 31 09:34:25.595374 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 31 09:34:25.595374 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 31 09:34:25.595374 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 31 09:34:25.596225 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:34:25.596225 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:34:25.596225 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 31 09:34:25.596225 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 09:34:25.596225 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 31 09:34:25.596225 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 09:34:25.596225 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:34:25.596225 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:34:25.596225 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:34:25.596225 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:34:25.596225 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:34:25.596225 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application with Lock( Aug 31 09:34:25.596225 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:34:25.596225 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:34:25.596225 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:34:25.596225 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:34:25.596225 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:34:25.596225 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:34:25.597173 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:34:25.597173 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:34:25.597173 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 31 09:34:25.597173 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 31 09:34:25.597173 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 31 09:34:25.597173 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 31 09:34:25.597173 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 31 09:34:25.597173 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 31 09:34:25.597173 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Aug 31 09:34:25.597173 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application Aug 31 09:34:25.603030 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 288/579] 10.222.0.13 () {56 vars in 1099 bytes} [Wed Aug 31 09:34:25 2022] GET /identity/v3/users/glance => generated 84 bytes in 301 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:34:25.603030 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb7a490-c2ae-424d-8fba-0975eeb43353 None admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:25.681897 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=100 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:25.753943 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb7a490-c2ae-424d-8fba-0975eeb43353 None admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:25.801614 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a4e763e-87da-4c87-8007-0def11bfdbe5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:25.802170 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4e763e-87da-4c87-8007-0def11bfdbe5 None admin] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:25.802482 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4e763e-87da-4c87-8007-0def11bfdbe5 None admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:25.802702 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4e763e-87da-4c87-8007-0def11bfdbe5 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:25.808035 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a574b28-e818-4b7f-9573-2f682d7e2ce1 None admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:25.811244 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a4e763e-87da-4c87-8007-0def11bfdbe5 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:25.813835 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a4e763e-87da-4c87-8007-0def11bfdbe5 None admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:25.827068 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application [None req-0a4e763e-87da-4c87-8007-0def11bfdbe5 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 31 09:34:25.827068 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:34:25.827068 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:34:25.827068 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:34:25.827068 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:34:25.827068 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:34:25.827068 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:34:25.827068 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:34:25.827068 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:34:25.827068 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:34:25.827068 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:34:25.827068 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:34:25.827068 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 31 09:34:25.827068 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 31 09:34:25.827068 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 31 09:34:25.827068 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 31 09:34:25.829573 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:34:25.829573 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:34:25.829573 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 31 09:34:25.829573 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 09:34:25.829573 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 31 09:34:25.829573 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 09:34:25.829573 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:34:25.829573 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:34:25.829573 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:34:25.829573 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:34:25.829573 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:34:25.829573 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application with Lock( Aug 31 09:34:25.829573 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:34:25.829573 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:34:25.829573 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:34:25.829573 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:34:25.829573 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:34:25.829573 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:34:25.831949 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:34:25.831949 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:34:25.831949 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 31 09:34:25.831949 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 31 09:34:25.831949 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 31 09:34:25.831949 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 31 09:34:25.831949 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 31 09:34:25.831949 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 31 09:34:25.831949 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 31 09:34:25.831949 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application Aug 31 09:34:25.831949 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 289/580] 10.222.0.13 () {56 vars in 1099 bytes} [Wed Aug 31 09:34:25 2022] GET /identity/v3/users/cinder => generated 84 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:34:25.831949 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a4e763e-87da-4c87-8007-0def11bfdbe5 None admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:25.889273 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-371640e5-0f8c-461c-93e5-7b79223a14ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:25.890515 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371640e5-0f8c-461c-93e5-7b79223a14ac None admin] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:25.891097 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371640e5-0f8c-461c-93e5-7b79223a14ac None admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:25.891674 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371640e5-0f8c-461c-93e5-7b79223a14ac None admin] PATH_INFO: `/v3/roles` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:25.893035 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-371640e5-0f8c-461c-93e5-7b79223a14ac None admin] RBAC: Adding query filter params (name=service) {{(pid=88113) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:34:25.893677 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-371640e5-0f8c-461c-93e5-7b79223a14ac None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:25.896615 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-371640e5-0f8c-461c-93e5-7b79223a14ac None admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:25.914403 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 292/581] 10.222.0.13 () {56 vars in 1103 bytes} [Wed Aug 31 09:34:25 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 324 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:34:25.916001 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-371640e5-0f8c-461c-93e5-7b79223a14ac None admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:25.992796 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a4e763e-87da-4c87-8007-0def11bfdbe5 None admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:26.016362 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=100 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:26.031708 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=100 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:26.042960 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f17769f0-373c-4932-bde2-09790d7f2c3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:26.043653 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17769f0-373c-4932-bde2-09790d7f2c3d None admin] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:26.043933 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17769f0-373c-4932-bde2-09790d7f2c3d None admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:26.044203 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17769f0-373c-4932-bde2-09790d7f2c3d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:26.059610 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f17769f0-373c-4932-bde2-09790d7f2c3d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:26.060993 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f17769f0-373c-4932-bde2-09790d7f2c3d None admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:26.080226 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application [None req-f17769f0-373c-4932-bde2-09790d7f2c3d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 09:34:26.080226 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:34:26.080226 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:34:26.080226 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:34:26.080226 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:34:26.080226 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:34:26.080226 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:34:26.080226 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:34:26.080226 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:34:26.080226 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:34:26.080226 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:34:26.080226 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:34:26.080226 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 09:34:26.080226 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 09:34:26.080226 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 09:34:26.080226 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 09:34:26.082435 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:34:26.082435 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:34:26.082435 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:34:26.082435 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:34:26.082435 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:34:26.082435 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:34:26.082435 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:34:26.082435 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application with Lock( Aug 31 09:34:26.082435 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:34:26.082435 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:34:26.082435 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:34:26.082435 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:34:26.082435 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:34:26.082435 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:34:26.082435 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:34:26.082435 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:34:26.082435 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 09:34:26.084615 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 09:34:26.084615 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 09:34:26.084615 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application Aug 31 09:34:26.084615 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 290/582] 10.222.0.13 () {56 vars in 1108 bytes} [Wed Aug 31 09:34:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 252 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:34:26.084615 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f17769f0-373c-4932-bde2-09790d7f2c3d None admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:26.108161 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-371640e5-0f8c-461c-93e5-7b79223a14ac None admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:26.157228 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38813c9a-0bd1-45e4-9e9f-d77aa92730ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:26.158323 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38813c9a-0bd1-45e4-9e9f-d77aa92730ce None admin] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:26.158725 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38813c9a-0bd1-45e4-9e9f-d77aa92730ce None admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:26.159147 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38813c9a-0bd1-45e4-9e9f-d77aa92730ce None admin] PATH_INFO: `/v3/users` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:26.160029 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38813c9a-0bd1-45e4-9e9f-d77aa92730ce None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=88113) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:34:26.160460 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38813c9a-0bd1-45e4-9e9f-d77aa92730ce None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:26.162186 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38813c9a-0bd1-45e4-9e9f-d77aa92730ce None admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:26.189425 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 293/583] 10.222.0.13 () {56 vars in 1137 bytes} [Wed Aug 31 09:34:25 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:34:26.189425 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38813c9a-0bd1-45e4-9e9f-d77aa92730ce None admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:26.273399 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f17769f0-373c-4932-bde2-09790d7f2c3d None admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:26.339548 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38813c9a-0bd1-45e4-9e9f-d77aa92730ce None admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:26.354322 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a55f3fe2-837e-4bd8-94a5-67a40f0e786c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:26.355321 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55f3fe2-837e-4bd8-94a5-67a40f0e786c None admin] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:26.356012 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55f3fe2-837e-4bd8-94a5-67a40f0e786c None admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:26.357047 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55f3fe2-837e-4bd8-94a5-67a40f0e786c None admin] PATH_INFO: `/v3/users` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:26.358810 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a55f3fe2-837e-4bd8-94a5-67a40f0e786c None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=88114) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:34:26.360052 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a55f3fe2-837e-4bd8-94a5-67a40f0e786c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:26.360260 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=100 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:26.363080 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a55f3fe2-837e-4bd8-94a5-67a40f0e786c None admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:26.388051 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e7d1ee7-c954-402c-a7e9-1f3b1d9d0827 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:26.388358 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 291/584] 10.222.0.13 () {56 vars in 1137 bytes} [Wed Aug 31 09:34:26 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 306 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:34:26.389013 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a55f3fe2-837e-4bd8-94a5-67a40f0e786c None admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:26.390745 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7d1ee7-c954-402c-a7e9-1f3b1d9d0827 None admin] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:26.391502 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7d1ee7-c954-402c-a7e9-1f3b1d9d0827 None admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:26.392098 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7d1ee7-c954-402c-a7e9-1f3b1d9d0827 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:26.416177 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e7d1ee7-c954-402c-a7e9-1f3b1d9d0827 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:26.418941 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e7d1ee7-c954-402c-a7e9-1f3b1d9d0827 None admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:26.429842 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=100 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:26.447493 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application [None req-8e7d1ee7-c954-402c-a7e9-1f3b1d9d0827 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 09:34:26.447493 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:34:26.447493 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:34:26.447493 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:34:26.447493 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:34:26.447493 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:34:26.447493 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:34:26.447493 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:34:26.447493 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:34:26.447493 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:34:26.447493 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:34:26.447493 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:34:26.447493 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 09:34:26.447493 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 09:34:26.447493 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 09:34:26.447493 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 09:34:26.449295 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:34:26.449295 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:34:26.449295 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:34:26.449295 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:34:26.449295 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:34:26.449295 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:34:26.449295 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:34:26.449295 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application with Lock( Aug 31 09:34:26.449295 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:34:26.449295 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:34:26.449295 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:34:26.449295 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:34:26.449295 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:34:26.449295 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:34:26.449295 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:34:26.449295 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:34:26.449295 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 09:34:26.451924 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 09:34:26.451924 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 09:34:26.451924 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application Aug 31 09:34:26.451924 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 294/585] 10.222.0.13 () {56 vars in 1108 bytes} [Wed Aug 31 09:34:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 264 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:34:26.451924 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e7d1ee7-c954-402c-a7e9-1f3b1d9d0827 None admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:26.575180 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a55f3fe2-837e-4bd8-94a5-67a40f0e786c None admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:26.601131 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e7d1ee7-c954-402c-a7e9-1f3b1d9d0827 None admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:26.639682 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0136d04-b4be-408a-97bc-66f6123d9578 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:26.641374 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0136d04-b4be-408a-97bc-66f6123d9578 None admin] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:26.641981 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0136d04-b4be-408a-97bc-66f6123d9578 None admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:26.642450 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0136d04-b4be-408a-97bc-66f6123d9578 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:26.643869 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17946d31-9f35-424d-98d1-ba7fff7564ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:26.644744 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17946d31-9f35-424d-98d1-ba7fff7564ad None admin] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:26.645016 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17946d31-9f35-424d-98d1-ba7fff7564ad None admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:26.645287 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17946d31-9f35-424d-98d1-ba7fff7564ad None admin] PATH_INFO: `/v3/domains` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:26.645980 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17946d31-9f35-424d-98d1-ba7fff7564ad None admin] RBAC: Adding query filter params (name=Default) {{(pid=88114) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:34:26.646283 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17946d31-9f35-424d-98d1-ba7fff7564ad None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:26.647657 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17946d31-9f35-424d-98d1-ba7fff7564ad None admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:26.662217 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 292/586] 10.222.0.13 () {56 vars in 1109 bytes} [Wed Aug 31 09:34:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:34:26.663078 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17946d31-9f35-424d-98d1-ba7fff7564ad None admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:26.671397 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0136d04-b4be-408a-97bc-66f6123d9578 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:26.675096 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0136d04-b4be-408a-97bc-66f6123d9578 None admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:26.695110 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application [None req-b0136d04-b4be-408a-97bc-66f6123d9578 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 09:34:26.695110 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:34:26.695110 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:34:26.695110 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:34:26.695110 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:34:26.695110 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:34:26.695110 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:34:26.695110 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:34:26.695110 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:34:26.695110 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:34:26.695110 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:34:26.695110 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:34:26.695110 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 09:34:26.695110 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 09:34:26.695110 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 09:34:26.695110 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 09:34:26.705643 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:34:26.705643 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:34:26.705643 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:34:26.705643 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:34:26.705643 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:34:26.705643 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:34:26.705643 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:34:26.705643 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application with Lock( Aug 31 09:34:26.705643 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:34:26.705643 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:34:26.705643 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:34:26.705643 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:34:26.705643 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:34:26.705643 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:34:26.705643 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:34:26.705643 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:34:26.705643 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 09:34:26.707361 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 09:34:26.707361 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 09:34:26.707361 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application Aug 31 09:34:26.707361 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 295/587] 10.222.0.13 () {56 vars in 1108 bytes} [Wed Aug 31 09:34:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 247 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:34:26.707361 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0136d04-b4be-408a-97bc-66f6123d9578 None admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:26.732217 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=100 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:26.822314 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=100 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:26.845477 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0136d04-b4be-408a-97bc-66f6123d9578 None admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:26.856519 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17946d31-9f35-424d-98d1-ba7fff7564ad None admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:26.890833 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0828cca7-1422-4f8e-8ff6-e715c648d617 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:26.891628 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0828cca7-1422-4f8e-8ff6-e715c648d617 None admin] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:26.891939 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0828cca7-1422-4f8e-8ff6-e715c648d617 None admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:26.892183 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0828cca7-1422-4f8e-8ff6-e715c648d617 None admin] PATH_INFO: `/v3/domains` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:26.892824 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0828cca7-1422-4f8e-8ff6-e715c648d617 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88113) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:34:26.893200 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0828cca7-1422-4f8e-8ff6-e715c648d617 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:26.894899 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0828cca7-1422-4f8e-8ff6-e715c648d617 None admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:26.915956 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 296/588] 10.222.0.13 () {56 vars in 1109 bytes} [Wed Aug 31 09:34:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:34:26.915956 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0828cca7-1422-4f8e-8ff6-e715c648d617 None admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:26.942721 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdbc6dbe-9e89-45ad-bcf6-e34237f56ad5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:26.943783 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdbc6dbe-9e89-45ad-bcf6-e34237f56ad5 None admin] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:26.944221 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdbc6dbe-9e89-45ad-bcf6-e34237f56ad5 None admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:26.944662 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdbc6dbe-9e89-45ad-bcf6-e34237f56ad5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:26.969366 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdbc6dbe-9e89-45ad-bcf6-e34237f56ad5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:26.974529 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdbc6dbe-9e89-45ad-bcf6-e34237f56ad5 None admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:26.995789 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application [None req-bdbc6dbe-9e89-45ad-bcf6-e34237f56ad5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 09:34:26.995789 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:34:26.995789 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:34:26.995789 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:34:26.995789 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:34:26.995789 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:34:26.995789 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:34:26.995789 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:34:26.995789 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:34:26.995789 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:34:26.995789 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:34:26.995789 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:34:26.995789 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 09:34:26.995789 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 09:34:26.995789 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 09:34:26.995789 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 09:34:26.997598 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:34:26.997598 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:34:26.997598 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:34:26.997598 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:34:26.997598 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:34:26.997598 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:34:26.997598 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:34:26.997598 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application with Lock( Aug 31 09:34:26.997598 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:34:26.997598 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:34:26.997598 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:34:26.997598 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:34:26.997598 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:34:26.997598 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:34:26.997598 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:34:26.997598 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:34:26.997598 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 09:34:26.999316 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 09:34:26.999316 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 09:34:26.999316 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application Aug 31 09:34:26.999316 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 293/589] 10.222.0.13 () {56 vars in 1108 bytes} [Wed Aug 31 09:34:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 335 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:34:26.999316 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdbc6dbe-9e89-45ad-bcf6-e34237f56ad5 None admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:27.043038 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=100 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:27.066204 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0828cca7-1422-4f8e-8ff6-e715c648d617 None admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:27.105935 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d814e8d2-121a-46f3-83c8-3742f5cbf860 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:27.106760 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d814e8d2-121a-46f3-83c8-3742f5cbf860 None admin] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:27.107198 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d814e8d2-121a-46f3-83c8-3742f5cbf860 None admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:27.107556 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d814e8d2-121a-46f3-83c8-3742f5cbf860 None admin] PATH_INFO: `/v3/projects/service` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:27.116158 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d814e8d2-121a-46f3-83c8-3742f5cbf860 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:27.120048 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d814e8d2-121a-46f3-83c8-3742f5cbf860 None admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:27.136234 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application [None req-d814e8d2-121a-46f3-83c8-3742f5cbf860 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 09:34:27.136234 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:34:27.136234 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:34:27.136234 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:34:27.136234 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:34:27.136234 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:34:27.136234 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:34:27.136234 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:34:27.136234 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:34:27.136234 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:34:27.136234 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:34:27.136234 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:34:27.136234 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 31 09:34:27.136234 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 31 09:34:27.136234 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 31 09:34:27.138252 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 31 09:34:27.138252 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:34:27.138252 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:34:27.138252 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:34:27.138252 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:34:27.138252 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:34:27.138252 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:34:27.138252 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:34:27.138252 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application with Lock( Aug 31 09:34:27.138252 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:34:27.138252 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:34:27.138252 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:34:27.138252 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:34:27.138252 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:34:27.138252 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:34:27.138252 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:34:27.138252 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:34:27.140135 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 31 09:34:27.140135 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 31 09:34:27.140135 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 31 09:34:27.140135 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 31 09:34:27.140135 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 31 09:34:27.140135 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 31 09:34:27.140135 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 09:34:27.140135 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application Aug 31 09:34:27.140135 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 297/590] 10.222.0.13 () {56 vars in 1111 bytes} [Wed Aug 31 09:34:26 2022] GET /identity/v3/projects/service => generated 88 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:34:27.140135 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d814e8d2-121a-46f3-83c8-3742f5cbf860 None admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:27.159793 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdbc6dbe-9e89-45ad-bcf6-e34237f56ad5 None admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:27.211443 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed355ae3-9cf5-4768-bb36-1ed6aaf09c48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:27.212455 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed355ae3-9cf5-4768-bb36-1ed6aaf09c48 None admin] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:27.213103 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed355ae3-9cf5-4768-bb36-1ed6aaf09c48 None admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:27.213621 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed355ae3-9cf5-4768-bb36-1ed6aaf09c48 None admin] PATH_INFO: `/v3/domains` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:27.214831 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed355ae3-9cf5-4768-bb36-1ed6aaf09c48 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88114) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:34:27.215375 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed355ae3-9cf5-4768-bb36-1ed6aaf09c48 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:27.217971 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed355ae3-9cf5-4768-bb36-1ed6aaf09c48 None admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:27.231293 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=100 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:27.242526 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 294/591] 10.222.0.13 () {56 vars in 1109 bytes} [Wed Aug 31 09:34:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:34:27.245451 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19839f04-d3a5-47db-af41-00149f1ffdd3 None None] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:27.246169 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19839f04-d3a5-47db-af41-00149f1ffdd3 None None] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:27.246648 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19839f04-d3a5-47db-af41-00149f1ffdd3 None None] PATH_INFO: `/` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:27.253433 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 295/592] 10.222.0.13 () {54 vars in 927 bytes} [Wed Aug 31 09:34:27 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:34:27.253433 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19839f04-d3a5-47db-af41-00149f1ffdd3 None None] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:27.274110 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d814e8d2-121a-46f3-83c8-3742f5cbf860 None admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:27.312720 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0949cbc2-fce3-4471-a6f7-17711946a3aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:27.313527 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0949cbc2-fce3-4471-a6f7-17711946a3aa None admin] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:27.313896 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0949cbc2-fce3-4471-a6f7-17711946a3aa None admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:27.314255 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0949cbc2-fce3-4471-a6f7-17711946a3aa None admin] PATH_INFO: `/v3/users/neutron` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:27.326978 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0949cbc2-fce3-4471-a6f7-17711946a3aa None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:27.330098 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0949cbc2-fce3-4471-a6f7-17711946a3aa None admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:27.334927 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=100 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:27.347154 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application [None req-0949cbc2-fce3-4471-a6f7-17711946a3aa None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Aug 31 09:34:27.347154 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:34:27.347154 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:34:27.347154 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:34:27.347154 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:34:27.347154 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:34:27.347154 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:34:27.347154 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:34:27.347154 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:34:27.347154 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:34:27.347154 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:34:27.347154 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:34:27.347154 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 31 09:34:27.347154 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 31 09:34:27.347154 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 31 09:34:27.347154 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 31 09:34:27.348847 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:34:27.348847 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:34:27.348847 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 31 09:34:27.348847 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 09:34:27.348847 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 31 09:34:27.348847 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 09:34:27.348847 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:34:27.348847 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:34:27.348847 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:34:27.348847 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:34:27.348847 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:34:27.348847 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application with Lock( Aug 31 09:34:27.348847 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:34:27.348847 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:34:27.348847 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:34:27.348847 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:34:27.348847 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:34:27.348847 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:34:27.349835 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:34:27.349835 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:34:27.349835 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 31 09:34:27.349835 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 31 09:34:27.349835 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 31 09:34:27.349835 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 31 09:34:27.349835 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 31 09:34:27.349835 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 31 09:34:27.349835 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Aug 31 09:34:27.349835 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application Aug 31 09:34:27.349835 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 298/593] 10.222.0.13 () {56 vars in 1102 bytes} [Wed Aug 31 09:34:27 2022] GET /identity/v3/users/neutron => generated 85 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:34:27.350429 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0949cbc2-fce3-4471-a6f7-17711946a3aa None admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:27.416724 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19839f04-d3a5-47db-af41-00149f1ffdd3 None None] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:27.463392 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0949cbc2-fce3-4471-a6f7-17711946a3aa None admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:27.479893 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df6b163e-a3c2-4924-8d35-8b089e6602c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:27.480918 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6b163e-a3c2-4924-8d35-8b089e6602c1 None admin] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:27.481426 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6b163e-a3c2-4924-8d35-8b089e6602c1 None admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:27.481901 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6b163e-a3c2-4924-8d35-8b089e6602c1 None admin] PATH_INFO: `/v3/domains` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:27.483087 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df6b163e-a3c2-4924-8d35-8b089e6602c1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88114) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:34:27.483622 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df6b163e-a3c2-4924-8d35-8b089e6602c1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:27.487042 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df6b163e-a3c2-4924-8d35-8b089e6602c1 None admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:27.502478 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-577543ed-b8c0-4d62-9e3a-d01ed366bffe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:27.503110 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577543ed-b8c0-4d62-9e3a-d01ed366bffe None admin] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:27.503385 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577543ed-b8c0-4d62-9e3a-d01ed366bffe None admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:27.503653 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577543ed-b8c0-4d62-9e3a-d01ed366bffe None admin] PATH_INFO: `/v3/projects` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:27.504363 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-577543ed-b8c0-4d62-9e3a-d01ed366bffe None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88113) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:34:27.504672 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-577543ed-b8c0-4d62-9e3a-d01ed366bffe None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:27.506157 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-577543ed-b8c0-4d62-9e3a-d01ed366bffe None admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:27.506708 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 296/594] 10.222.0.13 () {56 vars in 1109 bytes} [Wed Aug 31 09:34:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:34:27.508104 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df6b163e-a3c2-4924-8d35-8b089e6602c1 None admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:27.520364 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 299/595] 10.222.0.13 () {56 vars in 1148 bytes} [Wed Aug 31 09:34:27 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:34:27.521739 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c6f27f-a26d-40f5-b539-75da0d5a828e None None] REQUEST_METHOD: `POST` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:27.522072 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c6f27f-a26d-40f5-b539-75da0d5a828e None None] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:27.522357 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c6f27f-a26d-40f5-b539-75da0d5a828e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:27.544434 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.auth.core [None req-f1c6f27f-a26d-40f5-b539-75da0d5a828e None None] MFA Rules not processed for user `fd836cf590a145df9acf19cf3ae2ac56`. Rule list: `[]` (Enabled: `True`). {{(pid=88113) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:34:27.545898 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.fernet_utils [None req-f1c6f27f-a26d-40f5-b539-75da0d5a828e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:34:27.584156 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=100 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:27.620537 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=100 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:27.634851 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 300/596] 10.222.0.13 () {58 vars in 1024 bytes} [Wed Aug 31 09:34:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 09:34:27.635708 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c6f27f-a26d-40f5-b539-75da0d5a828e None None] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:27.658045 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df6b163e-a3c2-4924-8d35-8b089e6602c1 None admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:27.711167 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db7caf34-b61f-442f-82ee-76f3464ccfee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:27.711800 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7caf34-b61f-442f-82ee-76f3464ccfee None admin] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:27.712071 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7caf34-b61f-442f-82ee-76f3464ccfee None admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:27.712337 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7caf34-b61f-442f-82ee-76f3464ccfee None admin] PATH_INFO: `/v3/projects/service` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:27.720979 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db7caf34-b61f-442f-82ee-76f3464ccfee None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:27.722481 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db7caf34-b61f-442f-82ee-76f3464ccfee None admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:27.736090 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application [None req-db7caf34-b61f-442f-82ee-76f3464ccfee None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 09:34:27.736090 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:34:27.736090 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:34:27.736090 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:34:27.736090 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:34:27.736090 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:34:27.736090 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:34:27.736090 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:34:27.736090 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:34:27.736090 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:34:27.736090 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:34:27.736090 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:34:27.736090 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 31 09:34:27.736090 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 31 09:34:27.736090 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 31 09:34:27.738039 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 31 09:34:27.738039 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:34:27.738039 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:34:27.738039 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:34:27.738039 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:34:27.738039 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:34:27.738039 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:34:27.738039 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:34:27.738039 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application with Lock( Aug 31 09:34:27.738039 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:34:27.738039 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:34:27.738039 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:34:27.738039 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:34:27.738039 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:34:27.738039 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:34:27.738039 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:34:27.738039 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:34:27.739975 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 31 09:34:27.739975 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 31 09:34:27.739975 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 31 09:34:27.739975 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 31 09:34:27.739975 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 31 09:34:27.739975 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 31 09:34:27.739975 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 09:34:27.739975 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application Aug 31 09:34:27.742011 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 297/597] 10.222.0.13 () {56 vars in 1111 bytes} [Wed Aug 31 09:34:27 2022] GET /identity/v3/projects/service => generated 88 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:34:27.742948 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db7caf34-b61f-442f-82ee-76f3464ccfee None admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:27.801335 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c6f27f-a26d-40f5-b539-75da0d5a828e None None] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:27.868323 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd2bb81f-c0cc-45bb-93b0-165d11fdca57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:27.871417 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2bb81f-c0cc-45bb-93b0-165d11fdca57 None admin] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:27.871913 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2bb81f-c0cc-45bb-93b0-165d11fdca57 None admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:27.872382 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2bb81f-c0cc-45bb-93b0-165d11fdca57 None admin] PATH_INFO: `/v3/users` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:27.873695 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd2bb81f-c0cc-45bb-93b0-165d11fdca57 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=88113) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:34:27.874178 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd2bb81f-c0cc-45bb-93b0-165d11fdca57 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:27.880269 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd2bb81f-c0cc-45bb-93b0-165d11fdca57 None admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:27.903518 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db7caf34-b61f-442f-82ee-76f3464ccfee None admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:27.920791 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 301/598] 10.222.0.13 () {56 vars in 1139 bytes} [Wed Aug 31 09:34:27 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 280 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:34:27.920791 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd2bb81f-c0cc-45bb-93b0-165d11fdca57 None admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:27.933209 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=100 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:27.957356 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba72614c-8c88-467e-ac2f-57af8827cfd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:27.958097 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba72614c-8c88-467e-ac2f-57af8827cfd4 None admin] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:27.958498 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba72614c-8c88-467e-ac2f-57af8827cfd4 None admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:27.958884 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba72614c-8c88-467e-ac2f-57af8827cfd4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:27.973304 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba72614c-8c88-467e-ac2f-57af8827cfd4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:27.973304 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba72614c-8c88-467e-ac2f-57af8827cfd4 None admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:27.989174 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application [None req-ba72614c-8c88-467e-ac2f-57af8827cfd4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 09:34:27.989174 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:34:27.989174 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:34:27.989174 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:34:27.989174 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:34:27.989174 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:34:27.989174 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:34:27.989174 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:34:27.989174 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:34:27.989174 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:34:27.989174 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:34:27.989174 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:34:27.989174 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 31 09:34:27.989174 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 31 09:34:27.989174 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 31 09:34:27.990174 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 31 09:34:27.990174 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:34:27.990174 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:34:27.990174 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:34:27.990174 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:34:27.990174 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:34:27.990174 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:34:27.990174 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:34:27.990174 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application with Lock( Aug 31 09:34:27.990174 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:34:27.990174 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:34:27.990174 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:34:27.990174 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:34:27.990174 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:34:27.990174 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:34:27.990174 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:34:27.990174 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:34:27.991152 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 31 09:34:27.991152 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 31 09:34:27.991152 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 31 09:34:27.991152 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 31 09:34:27.991152 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 31 09:34:27.991152 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 31 09:34:27.991152 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 09:34:27.991152 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application Aug 31 09:34:27.993744 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 298/599] 10.222.0.13 () {56 vars in 1111 bytes} [Wed Aug 31 09:34:27 2022] GET /identity/v3/projects/service => generated 88 bytes in 251 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:34:27.995300 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba72614c-8c88-467e-ac2f-57af8827cfd4 None admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:27.997642 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=100 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:28.058705 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd2bb81f-c0cc-45bb-93b0-165d11fdca57 None admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:28.097146 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d1b1ec3-e325-4e82-b8a5-fe6022d4b279 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:28.097793 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1b1ec3-e325-4e82-b8a5-fe6022d4b279 None admin] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:28.098070 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1b1ec3-e325-4e82-b8a5-fe6022d4b279 None admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:28.098341 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1b1ec3-e325-4e82-b8a5-fe6022d4b279 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:28.099072 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d1b1ec3-e325-4e82-b8a5-fe6022d4b279 None admin] RBAC: Adding query filter params (role.id=e24ba4fd91584883af350cc84bc96c1d, scope.project.id=49548eb44d62415582af820ab66decee, user.id=202bb9db7a8a4977ba788a290ed9a3fa) {{(pid=88113) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:34:28.099379 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d1b1ec3-e325-4e82-b8a5-fe6022d4b279 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:28.100824 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d1b1ec3-e325-4e82-b8a5-fe6022d4b279 None admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:28.106686 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 302/600] 10.222.0.13 () {56 vars in 1374 bytes} [Wed Aug 31 09:34:27 2022] GET /identity/v3/role_assignments?user.id=202bb9db7a8a4977ba788a290ed9a3fa&scope.project.id=49548eb44d62415582af820ab66decee&role.id=e24ba4fd91584883af350cc84bc96c1d => generated 259 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:34:28.107536 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d1b1ec3-e325-4e82-b8a5-fe6022d4b279 None admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:28.133153 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba72614c-8c88-467e-ac2f-57af8827cfd4 None admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:28.188876 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27519f92-0ed3-4a18-a3ac-4cf055b2811a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:28.189720 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27519f92-0ed3-4a18-a3ac-4cf055b2811a None admin] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:28.190322 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27519f92-0ed3-4a18-a3ac-4cf055b2811a None admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:28.190911 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27519f92-0ed3-4a18-a3ac-4cf055b2811a None admin] PATH_INFO: `/v3/projects` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:28.191908 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27519f92-0ed3-4a18-a3ac-4cf055b2811a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88114) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:34:28.192341 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27519f92-0ed3-4a18-a3ac-4cf055b2811a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:28.194024 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27519f92-0ed3-4a18-a3ac-4cf055b2811a None admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:28.216797 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 299/601] 10.222.0.13 () {56 vars in 1148 bytes} [Wed Aug 31 09:34:27 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:34:28.217601 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27519f92-0ed3-4a18-a3ac-4cf055b2811a None admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:28.273629 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=100 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:28.310402 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d1b1ec3-e325-4e82-b8a5-fe6022d4b279 None admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:28.329342 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=100 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:28.362605 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2699666b-0154-4adf-9535-f6117feece51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:28.363649 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2699666b-0154-4adf-9535-f6117feece51 None admin] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:28.364091 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2699666b-0154-4adf-9535-f6117feece51 None admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:28.364547 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2699666b-0154-4adf-9535-f6117feece51 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:28.368418 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27519f92-0ed3-4a18-a3ac-4cf055b2811a None admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:28.382676 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2699666b-0154-4adf-9535-f6117feece51 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:28.385028 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2699666b-0154-4adf-9535-f6117feece51 None admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:28.410975 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application [None req-2699666b-0154-4adf-9535-f6117feece51 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 09:34:28.410975 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:34:28.410975 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:34:28.410975 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:34:28.410975 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:34:28.410975 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:34:28.410975 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:34:28.410975 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:34:28.410975 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:34:28.410975 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:34:28.410975 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:34:28.410975 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:34:28.410975 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 09:34:28.410975 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 09:34:28.410975 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 09:34:28.410975 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 09:34:28.413033 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:34:28.413033 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:34:28.413033 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:34:28.413033 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:34:28.413033 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:34:28.413033 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:34:28.413033 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:34:28.413033 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application with Lock( Aug 31 09:34:28.413033 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:34:28.413033 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:34:28.413033 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:34:28.413033 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:34:28.413033 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:34:28.413033 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:34:28.413033 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:34:28.413033 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:34:28.413033 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 09:34:28.414773 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 09:34:28.414773 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 09:34:28.414773 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application Aug 31 09:34:28.414773 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 303/602] 10.222.0.13 () {56 vars in 1108 bytes} [Wed Aug 31 09:34:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 306 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:34:28.414773 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2699666b-0154-4adf-9535-f6117feece51 None admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:28.431345 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a9b714e-288a-4f4e-a05b-e3c3e8376c87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:28.431620 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9b714e-288a-4f4e-a05b-e3c3e8376c87 None admin] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:28.431805 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9b714e-288a-4f4e-a05b-e3c3e8376c87 None admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:28.432094 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9b714e-288a-4f4e-a05b-e3c3e8376c87 None admin] PATH_INFO: `/v3/roles/service` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:28.451562 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a9b714e-288a-4f4e-a05b-e3c3e8376c87 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 09:34:28.461235 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application [None req-4a9b714e-288a-4f4e-a05b-e3c3e8376c87 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 31 09:34:28.461235 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:34:28.461235 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:34:28.461235 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:34:28.461235 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:34:28.461235 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:34:28.461235 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:34:28.461235 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:34:28.461235 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:34:28.461235 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:34:28.461235 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:34:28.461235 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:34:28.461235 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 09:34:28.461235 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 09:34:28.461235 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 09:34:28.461235 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 09:34:28.462456 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 09:34:28.462456 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 09:34:28.462456 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 09:34:28.462456 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 09:34:28.462456 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:34:28.462456 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:34:28.462456 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:34:28.462456 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:34:28.462456 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:34:28.462456 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:34:28.462456 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:34:28.462456 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application with Lock( Aug 31 09:34:28.462456 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:34:28.462456 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:34:28.462456 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:34:28.462456 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:34:28.463406 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:34:28.463406 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:34:28.463406 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:34:28.463406 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:34:28.463406 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 09:34:28.463406 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 09:34:28.463406 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 09:34:28.463406 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 09:34:28.463406 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 09:34:28.463406 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 09:34:28.463406 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 31 09:34:28.463406 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application Aug 31 09:34:28.463406 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 300/603] 10.222.0.13 () {56 vars in 1102 bytes} [Wed Aug 31 09:34:28 2022] GET /identity/v3/roles/service => generated 85 bytes in 246 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:34:28.464894 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a9b714e-288a-4f4e-a05b-e3c3e8376c87 None admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:28.599199 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2699666b-0154-4adf-9535-f6117feece51 None admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:28.611588 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=100 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:28.618681 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a9b714e-288a-4f4e-a05b-e3c3e8376c87 None admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:28.661182 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-118b8891-d7de-4638-a266-026b595d53d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:28.662189 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118b8891-d7de-4638-a266-026b595d53d6 None admin] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:28.662589 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118b8891-d7de-4638-a266-026b595d53d6 None admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:28.663171 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118b8891-d7de-4638-a266-026b595d53d6 None admin] PATH_INFO: `/v3/projects` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:28.664082 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-118b8891-d7de-4638-a266-026b595d53d6 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88113) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:34:28.664522 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-118b8891-d7de-4638-a266-026b595d53d6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:28.666667 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-118b8891-d7de-4638-a266-026b595d53d6 None admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:28.672495 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75db3ef2-5a40-4405-9e01-e85c420d1159 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:28.673263 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75db3ef2-5a40-4405-9e01-e85c420d1159 None admin] REQUEST_METHOD: `PUT` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:28.673637 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75db3ef2-5a40-4405-9e01-e85c420d1159 None admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:28.673917 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75db3ef2-5a40-4405-9e01-e85c420d1159 None admin] PATH_INFO: `/v3/projects/49548eb44d62415582af820ab66decee/users/be216d156b54452cbbf978c7734343aa/roles/e24ba4fd91584883af350cc84bc96c1d` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:28.685987 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 304/604] 10.222.0.13 () {56 vars in 1148 bytes} [Wed Aug 31 09:34:28 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:34:28.687237 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-118b8891-d7de-4638-a266-026b595d53d6 None admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:28.697247 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75db3ef2-5a40-4405-9e01-e85c420d1159 None admin] RBAC: Authorizing `identity:create_grant(project_id=49548eb44d62415582af820ab66decee, user_id=be216d156b54452cbbf978c7734343aa, role_id=e24ba4fd91584883af350cc84bc96c1d)` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:28.699780 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75db3ef2-5a40-4405-9e01-e85c420d1159 None admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:28.717292 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 301/605] 10.222.0.13 () {58 vars in 1439 bytes} [Wed Aug 31 09:34:28 2022] PUT /identity/v3/projects/49548eb44d62415582af820ab66decee/users/be216d156b54452cbbf978c7734343aa/roles/e24ba4fd91584883af350cc84bc96c1d => generated 0 bytes in 253 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:34:28.718973 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75db3ef2-5a40-4405-9e01-e85c420d1159 None admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:28.738435 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=100 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:28.895201 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75db3ef2-5a40-4405-9e01-e85c420d1159 None admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:28.902516 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-118b8891-d7de-4638-a266-026b595d53d6 None admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:28.946501 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f1eb968-5571-4a33-ac6f-f567930e3d15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:28.947443 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f1eb968-5571-4a33-ac6f-f567930e3d15 None admin] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:28.947726 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f1eb968-5571-4a33-ac6f-f567930e3d15 None admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:28.948002 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f1eb968-5571-4a33-ac6f-f567930e3d15 None admin] PATH_INFO: `/v3/roles` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:28.949326 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f1eb968-5571-4a33-ac6f-f567930e3d15 None admin] RBAC: Adding query filter params (name=service) {{(pid=88114) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:34:28.949671 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f1eb968-5571-4a33-ac6f-f567930e3d15 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:28.951378 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f1eb968-5571-4a33-ac6f-f567930e3d15 None admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:28.965373 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 302/606] 10.222.0.13 () {56 vars in 1103 bytes} [Wed Aug 31 09:34:28 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:34:28.966258 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f1eb968-5571-4a33-ac6f-f567930e3d15 None admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:28.966914 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cecc1557-56c4-4445-ae07-a4b81ffb71fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:28.967932 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cecc1557-56c4-4445-ae07-a4b81ffb71fe None admin] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:28.968439 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cecc1557-56c4-4445-ae07-a4b81ffb71fe None admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:28.968885 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cecc1557-56c4-4445-ae07-a4b81ffb71fe None admin] PATH_INFO: `/v3/domains` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:28.974359 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cecc1557-56c4-4445-ae07-a4b81ffb71fe None admin] RBAC: Adding query filter params (name=Default) {{(pid=88113) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:34:28.975059 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cecc1557-56c4-4445-ae07-a4b81ffb71fe None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:28.977760 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cecc1557-56c4-4445-ae07-a4b81ffb71fe None admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:28.986154 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:29.003217 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 305/607] 10.222.0.13 () {56 vars in 1109 bytes} [Wed Aug 31 09:34:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 316 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:34:29.004736 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cecc1557-56c4-4445-ae07-a4b81ffb71fe None admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:29.136694 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=100 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:29.142565 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f1eb968-5571-4a33-ac6f-f567930e3d15 None admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:29.151752 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cecc1557-56c4-4445-ae07-a4b81ffb71fe None admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:29.184997 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e404b73-d23e-4833-847d-dd0dee69cbb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:29.185806 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e404b73-d23e-4833-847d-dd0dee69cbb6 None admin] REQUEST_METHOD: `PUT` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:29.186099 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e404b73-d23e-4833-847d-dd0dee69cbb6 None admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:29.186404 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e404b73-d23e-4833-847d-dd0dee69cbb6 None admin] PATH_INFO: `/v3/projects/49548eb44d62415582af820ab66decee/users/0d52b69c8e94441dacfcff944326f169/roles/e24ba4fd91584883af350cc84bc96c1d` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:29.195280 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b306e0d4-3d58-4c5e-87ce-8d1d51857815 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:29.196842 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b306e0d4-3d58-4c5e-87ce-8d1d51857815 None admin] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:29.197786 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b306e0d4-3d58-4c5e-87ce-8d1d51857815 None admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:29.198569 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b306e0d4-3d58-4c5e-87ce-8d1d51857815 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:29.206203 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e404b73-d23e-4833-847d-dd0dee69cbb6 None admin] RBAC: Authorizing `identity:create_grant(project_id=49548eb44d62415582af820ab66decee, user_id=0d52b69c8e94441dacfcff944326f169, role_id=e24ba4fd91584883af350cc84bc96c1d)` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:29.207684 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e404b73-d23e-4833-847d-dd0dee69cbb6 None admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:29.223065 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b306e0d4-3d58-4c5e-87ce-8d1d51857815 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:29.224561 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b306e0d4-3d58-4c5e-87ce-8d1d51857815 None admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:29.229701 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 303/608] 10.222.0.13 () {58 vars in 1439 bytes} [Wed Aug 31 09:34:28 2022] PUT /identity/v3/projects/49548eb44d62415582af820ab66decee/users/0d52b69c8e94441dacfcff944326f169/roles/e24ba4fd91584883af350cc84bc96c1d => generated 0 bytes in 264 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:34:29.230582 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e404b73-d23e-4833-847d-dd0dee69cbb6 None admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:29.247665 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application [None req-b306e0d4-3d58-4c5e-87ce-8d1d51857815 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 09:34:29.247665 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:34:29.247665 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:34:29.247665 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:34:29.247665 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:34:29.247665 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:34:29.247665 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:34:29.247665 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:34:29.247665 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:34:29.247665 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:34:29.247665 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:34:29.247665 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:34:29.247665 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 09:34:29.247665 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 09:34:29.247665 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 09:34:29.247665 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 09:34:29.250295 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:34:29.250295 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:34:29.250295 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:34:29.250295 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:34:29.250295 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:34:29.250295 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:34:29.250295 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:34:29.250295 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application with Lock( Aug 31 09:34:29.250295 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:34:29.250295 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:34:29.250295 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:34:29.250295 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:34:29.250295 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:34:29.250295 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:34:29.250295 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:34:29.250295 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:34:29.250295 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 09:34:29.252301 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 09:34:29.252301 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 09:34:29.252301 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application Aug 31 09:34:29.257952 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 306/609] 10.222.0.13 () {56 vars in 1108 bytes} [Wed Aug 31 09:34:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 245 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:34:29.257952 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b306e0d4-3d58-4c5e-87ce-8d1d51857815 None admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:29.357929 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:29.419434 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e404b73-d23e-4833-847d-dd0dee69cbb6 None admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:29.491915 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c3f92ab-1de9-49ad-a6c5-932eef2b3ee4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:29.493692 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3f92ab-1de9-49ad-a6c5-932eef2b3ee4 None admin] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:29.494525 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3f92ab-1de9-49ad-a6c5-932eef2b3ee4 None admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:29.495303 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3f92ab-1de9-49ad-a6c5-932eef2b3ee4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:29.502814 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b306e0d4-3d58-4c5e-87ce-8d1d51857815 None admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:29.512364 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c3f92ab-1de9-49ad-a6c5-932eef2b3ee4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:29.516304 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c3f92ab-1de9-49ad-a6c5-932eef2b3ee4 None admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:29.540823 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application [None req-0c3f92ab-1de9-49ad-a6c5-932eef2b3ee4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 09:34:29.540823 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:34:29.540823 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:34:29.540823 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:34:29.540823 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:34:29.540823 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:34:29.540823 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:34:29.540823 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:34:29.540823 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:34:29.540823 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:34:29.540823 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:34:29.540823 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:34:29.540823 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 31 09:34:29.540823 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 31 09:34:29.540823 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 31 09:34:29.542792 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 31 09:34:29.542792 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:34:29.542792 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:34:29.542792 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:34:29.542792 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:34:29.542792 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:34:29.542792 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:34:29.542792 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:34:29.542792 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application with Lock( Aug 31 09:34:29.542792 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:34:29.542792 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:34:29.542792 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:34:29.542792 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:34:29.542792 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:34:29.542792 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:34:29.542792 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:34:29.542792 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:34:29.544337 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 31 09:34:29.544337 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 31 09:34:29.544337 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 31 09:34:29.544337 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 31 09:34:29.544337 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 31 09:34:29.544337 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 31 09:34:29.544337 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 09:34:29.544337 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application Aug 31 09:34:29.548556 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f50a1a-aeeb-41d7-965d-b022556ed9f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:29.549057 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f50a1a-aeeb-41d7-965d-b022556ed9f1 None admin] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:29.549334 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f50a1a-aeeb-41d7-965d-b022556ed9f1 None admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:29.549643 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f50a1a-aeeb-41d7-965d-b022556ed9f1 None admin] PATH_INFO: `/v3/domains` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:29.550164 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 304/610] 10.222.0.13 () {56 vars in 1111 bytes} [Wed Aug 31 09:34:29 2022] GET /identity/v3/projects/service => generated 88 bytes in 320 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:34:29.550326 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2f50a1a-aeeb-41d7-965d-b022556ed9f1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88113) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:34:29.550632 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2f50a1a-aeeb-41d7-965d-b022556ed9f1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:29.553433 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2f50a1a-aeeb-41d7-965d-b022556ed9f1 None admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:29.555940 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=100 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:29.565481 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c3f92ab-1de9-49ad-a6c5-932eef2b3ee4 None admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:29.567894 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 307/611] 10.222.0.13 () {56 vars in 1109 bytes} [Wed Aug 31 09:34:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 311 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:34:29.573354 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f50a1a-aeeb-41d7-965d-b022556ed9f1 None admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:29.696380 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f50a1a-aeeb-41d7-965d-b022556ed9f1 None admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:29.731203 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-005ce0ca-23d4-4712-b874-aae5642ef946 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:29.732016 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005ce0ca-23d4-4712-b874-aae5642ef946 None admin] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:29.732401 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005ce0ca-23d4-4712-b874-aae5642ef946 None admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:29.732775 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005ce0ca-23d4-4712-b874-aae5642ef946 None admin] PATH_INFO: `/v3/users/nova` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:29.744520 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-005ce0ca-23d4-4712-b874-aae5642ef946 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:29.744520 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-005ce0ca-23d4-4712-b874-aae5642ef946 None admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:29.755998 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application [None req-005ce0ca-23d4-4712-b874-aae5642ef946 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 31 09:34:29.755998 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:34:29.755998 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:34:29.755998 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:34:29.755998 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:34:29.755998 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:34:29.755998 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:34:29.755998 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:34:29.755998 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:34:29.755998 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:34:29.755998 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:34:29.755998 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:34:29.755998 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 31 09:34:29.755998 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 31 09:34:29.755998 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 31 09:34:29.755998 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 31 09:34:29.758023 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:34:29.758023 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:34:29.758023 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 31 09:34:29.758023 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 09:34:29.758023 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 31 09:34:29.758023 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 09:34:29.758023 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:34:29.758023 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:34:29.758023 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:34:29.758023 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:34:29.758023 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:34:29.758023 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application with Lock( Aug 31 09:34:29.758023 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:34:29.758023 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:34:29.758023 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:34:29.758023 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:34:29.758023 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:34:29.758023 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:34:29.760059 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:34:29.760059 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:34:29.760059 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 31 09:34:29.760059 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 31 09:34:29.760059 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 31 09:34:29.760059 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 31 09:34:29.760059 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 31 09:34:29.760059 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 31 09:34:29.760059 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 31 09:34:29.760059 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application Aug 31 09:34:29.760059 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 308/612] 10.222.0.13 () {56 vars in 1093 bytes} [Wed Aug 31 09:34:29 2022] GET /identity/v3/users/nova => generated 82 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:34:29.767553 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-005ce0ca-23d4-4712-b874-aae5642ef946 None admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:29.836485 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c3f92ab-1de9-49ad-a6c5-932eef2b3ee4 None admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:29.879227 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=100 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:29.907666 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=100 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:29.914926 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e5d515-2e06-482e-b444-90380739c81d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:29.915625 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e5d515-2e06-482e-b444-90380739c81d None admin] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:29.915904 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e5d515-2e06-482e-b444-90380739c81d None admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:29.916174 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e5d515-2e06-482e-b444-90380739c81d None admin] PATH_INFO: `/v3/projects` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:29.916888 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21e5d515-2e06-482e-b444-90380739c81d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88114) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:34:29.917194 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21e5d515-2e06-482e-b444-90380739c81d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:29.918685 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21e5d515-2e06-482e-b444-90380739c81d None admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:29.939502 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 305/613] 10.222.0.13 () {56 vars in 1148 bytes} [Wed Aug 31 09:34:29 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 375 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:34:29.949307 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e5d515-2e06-482e-b444-90380739c81d None admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:30.027510 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-005ce0ca-23d4-4712-b874-aae5642ef946 None admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:30.067691 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e5dc24-a52a-4a15-9772-afd59c05ce5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:30.068485 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e5dc24-a52a-4a15-9772-afd59c05ce5c None admin] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:30.068860 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e5dc24-a52a-4a15-9772-afd59c05ce5c None admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:30.069256 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e5dc24-a52a-4a15-9772-afd59c05ce5c None admin] PATH_INFO: `/v3/users` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:30.070140 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71e5dc24-a52a-4a15-9772-afd59c05ce5c None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=88113) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:34:30.070548 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71e5dc24-a52a-4a15-9772-afd59c05ce5c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:30.072155 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71e5dc24-a52a-4a15-9772-afd59c05ce5c None admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:30.093547 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 309/614] 10.222.0.13 () {56 vars in 1133 bytes} [Wed Aug 31 09:34:29 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 326 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:34:30.099420 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e5dc24-a52a-4a15-9772-afd59c05ce5c None admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:30.223253 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e5d515-2e06-482e-b444-90380739c81d None admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:30.240688 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e5dc24-a52a-4a15-9772-afd59c05ce5c None admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:30.261795 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=100 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:30.283034 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf17c4d1-2013-4808-b47d-243c5b4b7501 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:30.284248 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf17c4d1-2013-4808-b47d-243c5b4b7501 None admin] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:30.284633 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf17c4d1-2013-4808-b47d-243c5b4b7501 None admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:30.285022 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf17c4d1-2013-4808-b47d-243c5b4b7501 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:30.362449 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e15da43-60ba-4a43-b729-3a74e847356c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:30.363150 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e15da43-60ba-4a43-b729-3a74e847356c None admin] REQUEST_METHOD: `PUT` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:30.363427 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e15da43-60ba-4a43-b729-3a74e847356c None admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:30.363702 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e15da43-60ba-4a43-b729-3a74e847356c None admin] PATH_INFO: `/v3/projects/49548eb44d62415582af820ab66decee/users/a9bdaacd597442c2bcde2c555f2cc140/roles/e24ba4fd91584883af350cc84bc96c1d` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:30.364328 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf17c4d1-2013-4808-b47d-243c5b4b7501 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:30.365871 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf17c4d1-2013-4808-b47d-243c5b4b7501 None admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:30.398158 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e15da43-60ba-4a43-b729-3a74e847356c None admin] RBAC: Authorizing `identity:create_grant(project_id=49548eb44d62415582af820ab66decee, user_id=a9bdaacd597442c2bcde2c555f2cc140, role_id=e24ba4fd91584883af350cc84bc96c1d)` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:30.399617 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e15da43-60ba-4a43-b729-3a74e847356c None admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:30.441201 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 306/615] 10.222.0.13 () {58 vars in 1439 bytes} [Wed Aug 31 09:34:29 2022] PUT /identity/v3/projects/49548eb44d62415582af820ab66decee/users/a9bdaacd597442c2bcde2c555f2cc140/roles/e24ba4fd91584883af350cc84bc96c1d => generated 0 bytes in 493 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 09:34:30.487230 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 310/616] 10.222.0.13 () {58 vars in 1282 bytes} [Wed Aug 31 09:34:30 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 388 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 31 09:34:30.494669 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e15da43-60ba-4a43-b729-3a74e847356c None admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:30.562322 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:30.610239 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7f9a80-04cf-4a0d-a315-a4386d1ca7a2 None None] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:30.611021 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7f9a80-04cf-4a0d-a315-a4386d1ca7a2 None None] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:30.611275 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7f9a80-04cf-4a0d-a315-a4386d1ca7a2 None None] PATH_INFO: `/` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:30.613278 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 311/617] 10.222.0.13 () {54 vars in 927 bytes} [Wed Aug 31 09:34:30 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 31 09:34:30.629430 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef127527-171d-40b4-a583-e2790e94f3dd None None] REQUEST_METHOD: `POST` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:30.629430 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef127527-171d-40b4-a583-e2790e94f3dd None None] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:30.629430 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef127527-171d-40b4-a583-e2790e94f3dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:30.672584 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e15da43-60ba-4a43-b729-3a74e847356c None admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:30.683194 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.auth.core [None req-ef127527-171d-40b4-a583-e2790e94f3dd None None] MFA Rules not processed for user `fd836cf590a145df9acf19cf3ae2ac56`. Rule list: `[]` (Enabled: `True`). {{(pid=88113) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:34:30.685263 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.fernet_utils [None req-ef127527-171d-40b4-a583-e2790e94f3dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:34:30.705794 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=100 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:30.720011 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee3a495-1554-4999-a1fa-340ccbccb42c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:30.720674 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee3a495-1554-4999-a1fa-340ccbccb42c None admin] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:30.720954 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee3a495-1554-4999-a1fa-340ccbccb42c None admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:30.721233 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee3a495-1554-4999-a1fa-340ccbccb42c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:30.732828 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ee3a495-1554-4999-a1fa-340ccbccb42c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:30.734969 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ee3a495-1554-4999-a1fa-340ccbccb42c None admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:30.762096 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application [None req-5ee3a495-1554-4999-a1fa-340ccbccb42c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 09:34:30.762096 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:34:30.762096 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:34:30.762096 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:34:30.762096 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:34:30.762096 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:34:30.762096 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:34:30.762096 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:34:30.762096 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:34:30.762096 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:34:30.762096 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:34:30.762096 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:34:30.762096 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 09:34:30.762096 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 09:34:30.762096 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 09:34:30.762096 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 09:34:30.763851 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:34:30.763851 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:34:30.763851 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:34:30.763851 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:34:30.763851 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:34:30.763851 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:34:30.763851 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:34:30.763851 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application with Lock( Aug 31 09:34:30.763851 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:34:30.763851 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:34:30.763851 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:34:30.763851 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:34:30.763851 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:34:30.763851 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:34:30.763851 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:34:30.763851 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:34:30.763851 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 09:34:30.766505 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 09:34:30.766505 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 09:34:30.766505 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application Aug 31 09:34:30.766505 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 307/618] 10.222.0.13 () {56 vars in 1108 bytes} [Wed Aug 31 09:34:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 269 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:34:30.771066 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee3a495-1554-4999-a1fa-340ccbccb42c None admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:30.858478 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 312/619] 10.222.0.13 () {58 vars in 1024 bytes} [Wed Aug 31 09:34:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 240 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 09:34:30.922079 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=100 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:30.934890 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee3a495-1554-4999-a1fa-340ccbccb42c None admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:31.007954 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfdd0809-7577-4b3a-93cc-5159c7ed37fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:31.008944 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfdd0809-7577-4b3a-93cc-5159c7ed37fd None admin] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:31.008944 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfdd0809-7577-4b3a-93cc-5159c7ed37fd None admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:31.009228 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfdd0809-7577-4b3a-93cc-5159c7ed37fd None admin] PATH_INFO: `/v3/domains` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:31.009782 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfdd0809-7577-4b3a-93cc-5159c7ed37fd None admin] RBAC: Adding query filter params (name=Default) {{(pid=88114) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:34:31.010087 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfdd0809-7577-4b3a-93cc-5159c7ed37fd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:31.011494 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfdd0809-7577-4b3a-93cc-5159c7ed37fd None admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:31.036061 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 308/620] 10.222.0.13 () {56 vars in 1109 bytes} [Wed Aug 31 09:34:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:34:31.041802 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef127527-171d-40b4-a583-e2790e94f3dd None None] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:31.206473 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2893d90c-e74a-429e-b387-e10042f07c15 None None] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:31.206896 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2893d90c-e74a-429e-b387-e10042f07c15 None None] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:31.207184 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2893d90c-e74a-429e-b387-e10042f07c15 None None] PATH_INFO: `/` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:31.208900 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 309/621] 10.222.0.13 () {54 vars in 927 bytes} [Wed Aug 31 09:34:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 31 09:34:31.214614 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace35651-18e0-4844-a3ac-e0cba22b6d24 None None] REQUEST_METHOD: `POST` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:31.214889 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace35651-18e0-4844-a3ac-e0cba22b6d24 None None] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:31.215160 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace35651-18e0-4844-a3ac-e0cba22b6d24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:31.235554 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.auth.core [None req-ace35651-18e0-4844-a3ac-e0cba22b6d24 None None] MFA Rules not processed for user `fd836cf590a145df9acf19cf3ae2ac56`. Rule list: `[]` (Enabled: `True`). {{(pid=88114) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:34:31.236925 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.fernet_utils [None req-ace35651-18e0-4844-a3ac-e0cba22b6d24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88114) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:34:31.255747 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef127527-171d-40b4-a583-e2790e94f3dd None None] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:31.303756 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b788a5-7fcb-4046-b2b8-5ad136aa656c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:31.304309 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b788a5-7fcb-4046-b2b8-5ad136aa656c None admin] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:31.304589 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b788a5-7fcb-4046-b2b8-5ad136aa656c None admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:31.304862 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b788a5-7fcb-4046-b2b8-5ad136aa656c None admin] PATH_INFO: `/v3/projects/service` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:31.313267 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77b788a5-7fcb-4046-b2b8-5ad136aa656c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:31.314896 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77b788a5-7fcb-4046-b2b8-5ad136aa656c None admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:31.329786 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application [None req-77b788a5-7fcb-4046-b2b8-5ad136aa656c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 09:34:31.329786 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:34:31.329786 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:34:31.329786 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:34:31.329786 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:34:31.329786 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:34:31.329786 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:34:31.329786 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:34:31.329786 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:34:31.329786 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:34:31.329786 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:34:31.329786 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:34:31.329786 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 31 09:34:31.329786 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 31 09:34:31.329786 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 31 09:34:31.332690 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 31 09:34:31.332690 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:34:31.332690 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:34:31.332690 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:34:31.332690 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:34:31.332690 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:34:31.332690 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:34:31.332690 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:34:31.332690 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application with Lock( Aug 31 09:34:31.332690 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:34:31.332690 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:34:31.332690 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:34:31.332690 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:34:31.332690 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:34:31.332690 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:34:31.332690 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:34:31.332690 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:34:31.335720 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 31 09:34:31.335720 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 31 09:34:31.335720 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 31 09:34:31.335720 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 31 09:34:31.335720 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 31 09:34:31.335720 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 31 09:34:31.335720 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 09:34:31.335720 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application Aug 31 09:34:31.337200 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 313/622] 10.222.0.13 () {56 vars in 1111 bytes} [Wed Aug 31 09:34:31 2022] GET /identity/v3/projects/service => generated 88 bytes in 290 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:34:31.337200 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b788a5-7fcb-4046-b2b8-5ad136aa656c None admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:31.345480 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 310/623] 10.222.0.13 () {58 vars in 1024 bytes} [Wed Aug 31 09:34:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 31 09:34:31.346594 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace35651-18e0-4844-a3ac-e0cba22b6d24 None None] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:31.351692 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=100 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:31.486015 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b788a5-7fcb-4046-b2b8-5ad136aa656c None admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:31.528326 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d8b9b2-582c-4ae1-bdc6-6d3e67e988bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:31.529009 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d8b9b2-582c-4ae1-bdc6-6d3e67e988bf None admin] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:31.529288 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d8b9b2-582c-4ae1-bdc6-6d3e67e988bf None admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:31.529581 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d8b9b2-582c-4ae1-bdc6-6d3e67e988bf None admin] PATH_INFO: `/v3/roles/service` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:31.544208 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5d8b9b2-582c-4ae1-bdc6-6d3e67e988bf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 09:34:31.551377 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application [None req-d5d8b9b2-582c-4ae1-bdc6-6d3e67e988bf None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 31 09:34:31.551377 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:34:31.551377 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:34:31.551377 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:34:31.551377 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:34:31.551377 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:34:31.551377 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:34:31.551377 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:34:31.551377 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:34:31.551377 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:34:31.551377 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:34:31.551377 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:34:31.551377 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 09:34:31.551377 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 09:34:31.551377 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 09:34:31.551377 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 09:34:31.552405 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 09:34:31.552405 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 09:34:31.552405 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 09:34:31.552405 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 09:34:31.552405 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:34:31.552405 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:34:31.552405 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:34:31.552405 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:34:31.552405 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:34:31.552405 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:34:31.552405 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:34:31.552405 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application with Lock( Aug 31 09:34:31.552405 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:34:31.552405 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:34:31.552405 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:34:31.552405 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:34:31.553719 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:34:31.553719 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:34:31.553719 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:34:31.553719 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:34:31.553719 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 09:34:31.553719 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 09:34:31.553719 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 09:34:31.553719 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 09:34:31.553719 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 09:34:31.553719 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 09:34:31.553719 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 31 09:34:31.553719 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application Aug 31 09:34:31.553719 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 314/624] 10.222.0.13 () {56 vars in 1102 bytes} [Wed Aug 31 09:34:31 2022] GET /identity/v3/roles/service => generated 85 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:34:31.560089 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d8b9b2-582c-4ae1-bdc6-6d3e67e988bf None admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:31.636390 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=100 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:31.651067 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=100 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:31.705214 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace35651-18e0-4844-a3ac-e0cba22b6d24 None None] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:31.732573 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d8b9b2-582c-4ae1-bdc6-6d3e67e988bf None admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:31.752346 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62effb8d-02e6-4cdc-be82-495554920558 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:31.752980 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62effb8d-02e6-4cdc-be82-495554920558 None admin] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:31.753588 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62effb8d-02e6-4cdc-be82-495554920558 None admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:31.753880 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62effb8d-02e6-4cdc-be82-495554920558 None admin] PATH_INFO: `/v3/projects` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:31.754610 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62effb8d-02e6-4cdc-be82-495554920558 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88114) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:34:31.754921 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62effb8d-02e6-4cdc-be82-495554920558 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:31.757461 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62effb8d-02e6-4cdc-be82-495554920558 None admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:31.774694 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 311/625] 10.222.0.13 () {56 vars in 1148 bytes} [Wed Aug 31 09:34:31 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 428 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:34:31.775943 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62effb8d-02e6-4cdc-be82-495554920558 None admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:31.789245 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2bff662-adff-47ee-b856-0db81650f54f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:31.790400 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2bff662-adff-47ee-b856-0db81650f54f None admin] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:31.790970 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2bff662-adff-47ee-b856-0db81650f54f None admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:31.791568 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2bff662-adff-47ee-b856-0db81650f54f None admin] PATH_INFO: `/v3/roles` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:31.792560 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2bff662-adff-47ee-b856-0db81650f54f None admin] RBAC: Adding query filter params (name=service) {{(pid=88113) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:34:31.793173 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2bff662-adff-47ee-b856-0db81650f54f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:31.794994 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2bff662-adff-47ee-b856-0db81650f54f None admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:31.809697 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 315/626] 10.222.0.13 () {56 vars in 1103 bytes} [Wed Aug 31 09:34:31 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 09:34:31.811246 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2bff662-adff-47ee-b856-0db81650f54f None admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:31.919134 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62effb8d-02e6-4cdc-be82-495554920558 None admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:31.954556 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=100 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:31.954873 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2bff662-adff-47ee-b856-0db81650f54f None admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:31.987685 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c2e8d25-fc87-4b41-889a-03c0e3b16b61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:31.988340 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2e8d25-fc87-4b41-889a-03c0e3b16b61 None admin] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:31.988619 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2e8d25-fc87-4b41-889a-03c0e3b16b61 None admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:31.988889 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2e8d25-fc87-4b41-889a-03c0e3b16b61 None admin] PATH_INFO: `/v3/roles/service` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:32.003184 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c2e8d25-fc87-4b41-889a-03c0e3b16b61 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 09:34:32.008633 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application [None req-7c2e8d25-fc87-4b41-889a-03c0e3b16b61 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 31 09:34:32.008633 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:34:32.008633 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:34:32.008633 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:34:32.008633 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:34:32.008633 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:34:32.008633 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:34:32.008633 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:34:32.008633 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:34:32.008633 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:34:32.008633 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:34:32.008633 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:34:32.008633 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 09:34:32.008633 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 09:34:32.008633 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 09:34:32.008633 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 09:34:32.010052 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 09:34:32.010052 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 09:34:32.010052 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 09:34:32.010052 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 09:34:32.010052 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:34:32.010052 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:34:32.010052 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:34:32.010052 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:34:32.010052 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:34:32.010052 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:34:32.010052 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:34:32.010052 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application with Lock( Aug 31 09:34:32.010052 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:34:32.010052 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:34:32.010052 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:34:32.010052 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:34:32.010884 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:34:32.010884 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:34:32.010884 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:34:32.010884 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:34:32.010884 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 09:34:32.010884 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 09:34:32.010884 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 09:34:32.010884 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 09:34:32.010884 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 09:34:32.010884 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 09:34:32.010884 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 31 09:34:32.010884 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application Aug 31 09:34:32.015524 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 312/627] 10.222.0.13 () {56 vars in 1102 bytes} [Wed Aug 31 09:34:31 2022] GET /identity/v3/roles/service => generated 85 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:34:32.016635 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c2e8d25-fc87-4b41-889a-03c0e3b16b61 None admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:32.017925 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=100 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:32.032024 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d18b78a9-1669-44f5-b3c5-fbb358193536 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:32.033270 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18b78a9-1669-44f5-b3c5-fbb358193536 None admin] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:32.033792 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18b78a9-1669-44f5-b3c5-fbb358193536 None admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:32.034242 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18b78a9-1669-44f5-b3c5-fbb358193536 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:32.035941 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d18b78a9-1669-44f5-b3c5-fbb358193536 None admin] RBAC: Adding query filter params (role.id=e24ba4fd91584883af350cc84bc96c1d, scope.project.id=49548eb44d62415582af820ab66decee, user.id=dc076493239d4ccdbdc37c0369c49529) {{(pid=88113) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:34:32.036485 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d18b78a9-1669-44f5-b3c5-fbb358193536 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:32.039076 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d18b78a9-1669-44f5-b3c5-fbb358193536 None admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:32.053746 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 316/628] 10.222.0.13 () {56 vars in 1374 bytes} [Wed Aug 31 09:34:31 2022] GET /identity/v3/role_assignments?user.id=dc076493239d4ccdbdc37c0369c49529&scope.project.id=49548eb44d62415582af820ab66decee&role.id=e24ba4fd91584883af350cc84bc96c1d => generated 608 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:34:32.055025 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d18b78a9-1669-44f5-b3c5-fbb358193536 None admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:32.222753 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c2e8d25-fc87-4b41-889a-03c0e3b16b61 None admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:32.281719 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d18b78a9-1669-44f5-b3c5-fbb358193536 None admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:32.285680 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c8c2cd-71d2-4113-9e3c-5c4ae9f03c92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:32.286657 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c8c2cd-71d2-4113-9e3c-5c4ae9f03c92 None admin] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:32.287142 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c8c2cd-71d2-4113-9e3c-5c4ae9f03c92 None admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:32.287616 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c8c2cd-71d2-4113-9e3c-5c4ae9f03c92 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:32.306350 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32c8c2cd-71d2-4113-9e3c-5c4ae9f03c92 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:32.308548 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32c8c2cd-71d2-4113-9e3c-5c4ae9f03c92 None admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:32.333761 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2fb3aa2-1d58-4e4e-903a-489c8538b821 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:32.333761 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2fb3aa2-1d58-4e4e-903a-489c8538b821 None admin] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:32.333761 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2fb3aa2-1d58-4e4e-903a-489c8538b821 None admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:32.333761 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2fb3aa2-1d58-4e4e-903a-489c8538b821 None admin] PATH_INFO: `/v3/roles` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:32.334173 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2fb3aa2-1d58-4e4e-903a-489c8538b821 None admin] RBAC: Adding query filter params (name=service) {{(pid=88113) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:34:32.334173 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2fb3aa2-1d58-4e4e-903a-489c8538b821 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:32.335478 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2fb3aa2-1d58-4e4e-903a-489c8538b821 None admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:32.336564 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application [None req-32c8c2cd-71d2-4113-9e3c-5c4ae9f03c92 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 09:34:32.336564 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:34:32.336564 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:34:32.336564 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:34:32.336564 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:34:32.336564 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:34:32.336564 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:34:32.336564 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:34:32.336564 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:34:32.336564 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:34:32.336564 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:34:32.336564 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:34:32.336564 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 09:34:32.336564 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 09:34:32.336564 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 09:34:32.336564 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 09:34:32.337536 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:34:32.337536 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:34:32.337536 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:34:32.337536 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:34:32.337536 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:34:32.337536 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:34:32.337536 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:34:32.337536 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application with Lock( Aug 31 09:34:32.337536 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:34:32.337536 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:34:32.337536 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:34:32.337536 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:34:32.337536 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:34:32.337536 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:34:32.337536 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:34:32.337536 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:34:32.337536 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 09:34:32.338423 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 09:34:32.338423 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 09:34:32.338423 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application Aug 31 09:34:32.340881 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 313/629] 10.222.0.13 () {56 vars in 1108 bytes} [Wed Aug 31 09:34:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 325 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:34:32.348255 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c8c2cd-71d2-4113-9e3c-5c4ae9f03c92 None admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:32.357066 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 317/630] 10.222.0.13 () {56 vars in 1103 bytes} [Wed Aug 31 09:34:32 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 303 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:34:32.365072 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2fb3aa2-1d58-4e4e-903a-489c8538b821 None admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:32.420022 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=100 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:32.448021 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=100 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:32.492744 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c8c2cd-71d2-4113-9e3c-5c4ae9f03c92 None admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:32.508676 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2fb3aa2-1d58-4e4e-903a-489c8538b821 None admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:32.538742 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-187a4256-d82a-40c0-aca9-f813a520f385 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:32.539382 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187a4256-d82a-40c0-aca9-f813a520f385 None admin] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:32.539661 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187a4256-d82a-40c0-aca9-f813a520f385 None admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:32.539934 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187a4256-d82a-40c0-aca9-f813a520f385 None admin] PATH_INFO: `/v3/domains` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:32.541696 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-187a4256-d82a-40c0-aca9-f813a520f385 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88114) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:34:32.542033 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-187a4256-d82a-40c0-aca9-f813a520f385 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:32.543542 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-187a4256-d82a-40c0-aca9-f813a520f385 None admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:32.555255 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4542e1f8-93cb-4368-a1c2-37a8a5cb8c27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:32.556293 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4542e1f8-93cb-4368-a1c2-37a8a5cb8c27 None admin] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:32.556777 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4542e1f8-93cb-4368-a1c2-37a8a5cb8c27 None admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:32.557291 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4542e1f8-93cb-4368-a1c2-37a8a5cb8c27 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:32.563178 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 314/631] 10.222.0.13 () {56 vars in 1109 bytes} [Wed Aug 31 09:34:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:34:32.564994 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-187a4256-d82a-40c0-aca9-f813a520f385 None admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:32.571940 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4542e1f8-93cb-4368-a1c2-37a8a5cb8c27 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:32.573856 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4542e1f8-93cb-4368-a1c2-37a8a5cb8c27 None admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:32.591553 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application [None req-4542e1f8-93cb-4368-a1c2-37a8a5cb8c27 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 09:34:32.591553 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:34:32.591553 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:34:32.591553 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:34:32.591553 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:34:32.591553 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:34:32.591553 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:34:32.591553 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:34:32.591553 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:34:32.591553 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:34:32.591553 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:34:32.591553 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:34:32.591553 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 09:34:32.591553 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 09:34:32.591553 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 09:34:32.591553 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 09:34:32.592579 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:34:32.592579 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:34:32.592579 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:34:32.592579 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:34:32.592579 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:34:32.592579 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:34:32.592579 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:34:32.592579 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application with Lock( Aug 31 09:34:32.592579 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:34:32.592579 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:34:32.592579 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:34:32.592579 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:34:32.592579 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:34:32.592579 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:34:32.592579 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:34:32.592579 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:34:32.592579 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 09:34:32.594279 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 09:34:32.594279 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 09:34:32.594279 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application Aug 31 09:34:32.596702 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 318/632] 10.222.0.13 () {56 vars in 1108 bytes} [Wed Aug 31 09:34:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 09:34:32.603838 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4542e1f8-93cb-4368-a1c2-37a8a5cb8c27 None admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:32.717775 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-187a4256-d82a-40c0-aca9-f813a520f385 None admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:32.721844 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=100 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:32.760777 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4542e1f8-93cb-4368-a1c2-37a8a5cb8c27 None admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:32.763809 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef1c2f96-54ec-45c9-bc63-ce01ec5d7db9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:32.764342 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=100 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:32.764453 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1c2f96-54ec-45c9-bc63-ce01ec5d7db9 None admin] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:32.764717 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1c2f96-54ec-45c9-bc63-ce01ec5d7db9 None admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:32.765546 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1c2f96-54ec-45c9-bc63-ce01ec5d7db9 None admin] PATH_INFO: `/v3/users/swift` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:32.774690 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef1c2f96-54ec-45c9-bc63-ce01ec5d7db9 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:32.776658 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef1c2f96-54ec-45c9-bc63-ce01ec5d7db9 None admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:32.790192 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application [None req-ef1c2f96-54ec-45c9-bc63-ce01ec5d7db9 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 31 09:34:32.790192 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:34:32.790192 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:34:32.790192 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:34:32.790192 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:34:32.790192 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:34:32.790192 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:34:32.790192 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:34:32.790192 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:34:32.790192 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:34:32.790192 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:34:32.790192 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:34:32.790192 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 31 09:34:32.790192 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 31 09:34:32.790192 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 31 09:34:32.790192 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 31 09:34:32.791228 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:34:32.791228 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:34:32.791228 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 31 09:34:32.791228 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 09:34:32.791228 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 31 09:34:32.791228 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 09:34:32.791228 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:34:32.791228 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:34:32.791228 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:34:32.791228 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:34:32.791228 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:34:32.791228 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application with Lock( Aug 31 09:34:32.791228 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:34:32.791228 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:34:32.791228 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:34:32.791228 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:34:32.791228 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:34:32.791228 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:34:32.792830 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:34:32.792830 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:34:32.792830 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 31 09:34:32.792830 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 31 09:34:32.792830 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 31 09:34:32.792830 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 31 09:34:32.792830 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 31 09:34:32.792830 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 31 09:34:32.792830 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 31 09:34:32.792830 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application Aug 31 09:34:32.792830 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 315/633] 10.222.0.13 () {56 vars in 1096 bytes} [Wed Aug 31 09:34:32 2022] GET /identity/v3/users/swift => generated 83 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:34:32.798632 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef1c2f96-54ec-45c9-bc63-ce01ec5d7db9 None admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:32.816103 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2caceecd-65fc-4211-9c93-a60edd851f82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:32.816681 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2caceecd-65fc-4211-9c93-a60edd851f82 None admin] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:32.816961 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2caceecd-65fc-4211-9c93-a60edd851f82 None admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:32.817238 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2caceecd-65fc-4211-9c93-a60edd851f82 None admin] PATH_INFO: `/v3/domains` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:32.817941 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2caceecd-65fc-4211-9c93-a60edd851f82 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88113) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:34:32.818297 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2caceecd-65fc-4211-9c93-a60edd851f82 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:32.819701 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2caceecd-65fc-4211-9c93-a60edd851f82 None admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:32.837746 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 319/634] 10.222.0.13 () {56 vars in 1109 bytes} [Wed Aug 31 09:34:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:34:32.839574 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2caceecd-65fc-4211-9c93-a60edd851f82 None admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:32.941827 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef1c2f96-54ec-45c9-bc63-ce01ec5d7db9 None admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:32.985855 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d615c43-8a02-48a5-80e8-98c473a733c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:32.986979 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d615c43-8a02-48a5-80e8-98c473a733c2 None admin] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:32.988214 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d615c43-8a02-48a5-80e8-98c473a733c2 None admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:32.991120 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2caceecd-65fc-4211-9c93-a60edd851f82 None admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:32.991346 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d615c43-8a02-48a5-80e8-98c473a733c2 None admin] PATH_INFO: `/v3/users` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:32.992164 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d615c43-8a02-48a5-80e8-98c473a733c2 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=88114) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:34:32.992484 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d615c43-8a02-48a5-80e8-98c473a733c2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:32.994297 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d615c43-8a02-48a5-80e8-98c473a733c2 None admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:33.018531 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 316/635] 10.222.0.13 () {56 vars in 1135 bytes} [Wed Aug 31 09:34:32 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:34:33.026545 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d615c43-8a02-48a5-80e8-98c473a733c2 None admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:33.088746 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c259ab79-10c0-4642-a4db-8fd67011d535 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:33.089437 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c259ab79-10c0-4642-a4db-8fd67011d535 None admin] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:33.092158 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c259ab79-10c0-4642-a4db-8fd67011d535 None admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:33.092527 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c259ab79-10c0-4642-a4db-8fd67011d535 None admin] PATH_INFO: `/v3/users/glance` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:33.104783 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=100 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:33.114048 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c259ab79-10c0-4642-a4db-8fd67011d535 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:33.118995 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c259ab79-10c0-4642-a4db-8fd67011d535 None admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:33.160105 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application [None req-c259ab79-10c0-4642-a4db-8fd67011d535 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Aug 31 09:34:33.160105 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:34:33.160105 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:34:33.160105 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:34:33.160105 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:34:33.160105 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:34:33.160105 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:34:33.160105 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:34:33.160105 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:34:33.160105 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:34:33.160105 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:34:33.160105 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:34:33.160105 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 31 09:34:33.160105 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 31 09:34:33.160105 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 31 09:34:33.160105 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 31 09:34:33.163681 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:34:33.163681 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:34:33.163681 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 31 09:34:33.163681 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 09:34:33.163681 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 31 09:34:33.163681 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 09:34:33.163681 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:34:33.163681 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:34:33.163681 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:34:33.163681 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:34:33.163681 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:34:33.163681 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application with Lock( Aug 31 09:34:33.163681 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:34:33.163681 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:34:33.163681 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:34:33.163681 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:34:33.163681 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:34:33.163681 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:34:33.167303 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:34:33.167303 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:34:33.167303 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 31 09:34:33.167303 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 31 09:34:33.167303 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 31 09:34:33.167303 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 31 09:34:33.167303 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 31 09:34:33.167303 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 31 09:34:33.167303 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Aug 31 09:34:33.167303 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application Aug 31 09:34:33.167303 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 320/636] 10.222.0.13 () {56 vars in 1099 bytes} [Wed Aug 31 09:34:32 2022] GET /identity/v3/users/glance => generated 84 bytes in 324 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:34:33.178167 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c259ab79-10c0-4642-a4db-8fd67011d535 None admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:33.215064 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d615c43-8a02-48a5-80e8-98c473a733c2 None admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:33.259250 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a76261-ebe1-4623-bdce-9f570251a0b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:33.259962 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a76261-ebe1-4623-bdce-9f570251a0b8 None admin] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:33.260241 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a76261-ebe1-4623-bdce-9f570251a0b8 None admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:33.260531 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a76261-ebe1-4623-bdce-9f570251a0b8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:33.263758 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=100 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:33.277826 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88a76261-ebe1-4623-bdce-9f570251a0b8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:33.279472 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88a76261-ebe1-4623-bdce-9f570251a0b8 None admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:33.310394 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application [None req-88a76261-ebe1-4623-bdce-9f570251a0b8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 09:34:33.310394 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:34:33.310394 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:34:33.310394 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:34:33.310394 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:34:33.310394 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:34:33.310394 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:34:33.310394 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:34:33.310394 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:34:33.310394 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:34:33.310394 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:34:33.310394 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:34:33.310394 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 09:34:33.310394 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 09:34:33.310394 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 09:34:33.310394 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 09:34:33.311697 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:34:33.311697 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:34:33.311697 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:34:33.311697 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:34:33.311697 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:34:33.311697 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:34:33.311697 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:34:33.311697 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application with Lock( Aug 31 09:34:33.311697 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:34:33.311697 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:34:33.311697 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:34:33.311697 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:34:33.311697 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:34:33.311697 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:34:33.311697 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:34:33.311697 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:34:33.311697 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 09:34:33.313079 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 09:34:33.313079 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 09:34:33.313079 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application Aug 31 09:34:33.315774 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 317/637] 10.222.0.13 () {56 vars in 1108 bytes} [Wed Aug 31 09:34:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 289 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:34:33.316877 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66e5b0b-98cc-4cbd-9f3b-767acca4ff58 None None] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:33.317153 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66e5b0b-98cc-4cbd-9f3b-767acca4ff58 None None] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:33.320158 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66e5b0b-98cc-4cbd-9f3b-767acca4ff58 None None] PATH_INFO: `/` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:33.328952 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 318/638] 10.222.0.13 () {54 vars in 927 bytes} [Wed Aug 31 09:34:33 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:34:33.331328 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d66e5b0b-98cc-4cbd-9f3b-767acca4ff58 None None] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:33.392213 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c259ab79-10c0-4642-a4db-8fd67011d535 None admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:33.450161 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=100 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:33.459871 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c668cf-2e30-42cb-b12c-ad4ddff9507c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:33.461027 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c668cf-2e30-42cb-b12c-ad4ddff9507c None admin] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:33.461618 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c668cf-2e30-42cb-b12c-ad4ddff9507c None admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:33.462184 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c668cf-2e30-42cb-b12c-ad4ddff9507c None admin] PATH_INFO: `/v3/users` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:33.463821 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10c668cf-2e30-42cb-b12c-ad4ddff9507c None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=88113) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:34:33.464451 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10c668cf-2e30-42cb-b12c-ad4ddff9507c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:33.467428 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10c668cf-2e30-42cb-b12c-ad4ddff9507c None admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:33.489052 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d66e5b0b-98cc-4cbd-9f3b-767acca4ff58 None None] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:33.507830 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 321/639] 10.222.0.13 () {56 vars in 1137 bytes} [Wed Aug 31 09:34:33 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 329 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:34:33.507830 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c6b82d-ead7-4518-88b7-e1ed29efc605 None None] REQUEST_METHOD: `POST` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:33.507830 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c6b82d-ead7-4518-88b7-e1ed29efc605 None None] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:33.507830 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c6b82d-ead7-4518-88b7-e1ed29efc605 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:33.537805 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c81c7f-324b-4f86-8bf7-561b21090752 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:33.538723 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c81c7f-324b-4f86-8bf7-561b21090752 None admin] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:33.539177 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c81c7f-324b-4f86-8bf7-561b21090752 None admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:33.539621 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c81c7f-324b-4f86-8bf7-561b21090752 None admin] PATH_INFO: `/v3/domains` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:33.540074 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.auth.core [None req-16c6b82d-ead7-4518-88b7-e1ed29efc605 None None] MFA Rules not processed for user `fd836cf590a145df9acf19cf3ae2ac56`. Rule list: `[]` (Enabled: `True`). {{(pid=88113) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:34:33.540831 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5c81c7f-324b-4f86-8bf7-561b21090752 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88114) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:34:33.541314 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5c81c7f-324b-4f86-8bf7-561b21090752 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:33.542653 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.fernet_utils [None req-16c6b82d-ead7-4518-88b7-e1ed29efc605 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:34:33.543179 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5c81c7f-324b-4f86-8bf7-561b21090752 None admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:33.568315 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 319/640] 10.222.0.13 () {56 vars in 1109 bytes} [Wed Aug 31 09:34:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:34:33.568315 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c81c7f-324b-4f86-8bf7-561b21090752 None admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:33.647955 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 322/641] 10.222.0.13 () {58 vars in 1024 bytes} [Wed Aug 31 09:34:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 09:34:33.649119 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c6b82d-ead7-4518-88b7-e1ed29efc605 None None] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:33.667143 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=100 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:33.761567 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c81c7f-324b-4f86-8bf7-561b21090752 None admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:33.806114 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad5afe9-65a9-4b33-8c64-5dd30ae90fa5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:33.806649 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad5afe9-65a9-4b33-8c64-5dd30ae90fa5 None admin] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:33.806931 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad5afe9-65a9-4b33-8c64-5dd30ae90fa5 None admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:33.807211 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad5afe9-65a9-4b33-8c64-5dd30ae90fa5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:33.810152 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=100 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:33.835064 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c6b82d-ead7-4518-88b7-e1ed29efc605 None None] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:33.885178 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d739d3a4-2f16-4255-827b-57751f3ba10c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:33.885850 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d739d3a4-2f16-4255-827b-57751f3ba10c None admin] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:33.886131 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d739d3a4-2f16-4255-827b-57751f3ba10c None admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:33.886402 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d739d3a4-2f16-4255-827b-57751f3ba10c None admin] PATH_INFO: `/v3/projects/service` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:33.894543 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d739d3a4-2f16-4255-827b-57751f3ba10c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:33.896029 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d739d3a4-2f16-4255-827b-57751f3ba10c None admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:33.910159 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application [None req-d739d3a4-2f16-4255-827b-57751f3ba10c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 09:34:33.910159 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:34:33.910159 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:34:33.910159 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:34:33.910159 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:34:33.910159 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:34:33.910159 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:34:33.910159 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:34:33.910159 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:34:33.910159 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:34:33.910159 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:34:33.910159 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:34:33.910159 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 31 09:34:33.910159 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 31 09:34:33.910159 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 31 09:34:33.911903 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 31 09:34:33.911903 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:34:33.911903 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:34:33.911903 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:34:33.911903 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:34:33.911903 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:34:33.911903 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:34:33.911903 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:34:33.911903 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application with Lock( Aug 31 09:34:33.911903 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:34:33.911903 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:34:33.911903 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:34:33.911903 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:34:33.911903 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:34:33.911903 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:34:33.911903 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:34:33.911903 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:34:33.913525 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 31 09:34:33.913525 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 31 09:34:33.913525 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 31 09:34:33.913525 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 31 09:34:33.913525 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 31 09:34:33.913525 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 31 09:34:33.913525 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 09:34:33.913525 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application Aug 31 09:34:33.913525 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 323/642] 10.222.0.13 () {56 vars in 1111 bytes} [Wed Aug 31 09:34:33 2022] GET /identity/v3/projects/service => generated 88 bytes in 263 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:34:33.913525 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ea368c-dd6b-4cbd-a283-075b0dd8c4e7 None None] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:33.913525 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ea368c-dd6b-4cbd-a283-075b0dd8c4e7 None None] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:33.914442 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ea368c-dd6b-4cbd-a283-075b0dd8c4e7 None None] PATH_INFO: `/` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:33.920924 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 324/643] 10.222.0.13 () {54 vars in 927 bytes} [Wed Aug 31 09:34:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:34:33.920924 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ea368c-dd6b-4cbd-a283-075b0dd8c4e7 None None] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:33.924147 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ad5afe9-65a9-4b33-8c64-5dd30ae90fa5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:33.927018 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ad5afe9-65a9-4b33-8c64-5dd30ae90fa5 None admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:34.017573 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=100 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:34.063102 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ea368c-dd6b-4cbd-a283-075b0dd8c4e7 None None] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:34.085432 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 320/644] 10.222.0.13 () {58 vars in 1282 bytes} [Wed Aug 31 09:34:33 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 517 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 31 09:34:34.085432 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e599a61d-e437-4ff9-aaa5-6af6d37fbbee None None] REQUEST_METHOD: `POST` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:34.085432 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e599a61d-e437-4ff9-aaa5-6af6d37fbbee None None] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:34.085432 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e599a61d-e437-4ff9-aaa5-6af6d37fbbee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:34.109406 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f35def7f-28ed-4584-acc0-4a427ce05ade None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:34.110052 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35def7f-28ed-4584-acc0-4a427ce05ade None admin] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:34.110334 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35def7f-28ed-4584-acc0-4a427ce05ade None admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:34.110604 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35def7f-28ed-4584-acc0-4a427ce05ade None admin] PATH_INFO: `/v3/projects` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:34.111320 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f35def7f-28ed-4584-acc0-4a427ce05ade None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88113) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:34:34.111624 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f35def7f-28ed-4584-acc0-4a427ce05ade None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:34.113130 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f35def7f-28ed-4584-acc0-4a427ce05ade None admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:34.115024 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.auth.core [None req-e599a61d-e437-4ff9-aaa5-6af6d37fbbee None None] MFA Rules not processed for user `fd836cf590a145df9acf19cf3ae2ac56`. Rule list: `[]` (Enabled: `True`). {{(pid=88114) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:34:34.116278 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.fernet_utils [None req-e599a61d-e437-4ff9-aaa5-6af6d37fbbee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88114) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:34:34.128419 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 325/645] 10.222.0.13 () {56 vars in 1148 bytes} [Wed Aug 31 09:34:33 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:34:34.129442 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f35def7f-28ed-4584-acc0-4a427ce05ade None admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:34.153607 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=100 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:34.205374 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 321/646] 10.222.0.13 () {58 vars in 1024 bytes} [Wed Aug 31 09:34:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 09:34:34.206304 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e599a61d-e437-4ff9-aaa5-6af6d37fbbee None None] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:34.299589 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f35def7f-28ed-4584-acc0-4a427ce05ade None admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:34.356819 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=100 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:34.367480 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e599a61d-e437-4ff9-aaa5-6af6d37fbbee None None] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:34.368867 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9edd863a-cd2a-4635-8699-4d8117896ffd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:34.370285 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9edd863a-cd2a-4635-8699-4d8117896ffd None admin] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:34.371225 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9edd863a-cd2a-4635-8699-4d8117896ffd None admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:34.372019 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9edd863a-cd2a-4635-8699-4d8117896ffd None admin] PATH_INFO: `/v3/roles/service` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:34.393978 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9edd863a-cd2a-4635-8699-4d8117896ffd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 09:34:34.403943 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application [None req-9edd863a-cd2a-4635-8699-4d8117896ffd None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 31 09:34:34.403943 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:34:34.403943 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:34:34.403943 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:34:34.403943 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:34:34.403943 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:34:34.403943 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:34:34.403943 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:34:34.403943 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:34:34.403943 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:34:34.403943 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:34:34.403943 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:34:34.403943 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 09:34:34.403943 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 09:34:34.403943 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 09:34:34.403943 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 09:34:34.406580 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 09:34:34.406580 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 09:34:34.406580 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 09:34:34.406580 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 09:34:34.406580 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:34:34.406580 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:34:34.406580 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:34:34.406580 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:34:34.406580 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:34:34.406580 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:34:34.406580 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:34:34.406580 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application with Lock( Aug 31 09:34:34.406580 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:34:34.406580 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:34:34.406580 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:34:34.406580 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:34:34.408156 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:34:34.408156 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:34:34.408156 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:34:34.408156 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:34:34.408156 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 09:34:34.408156 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 09:34:34.408156 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 09:34:34.408156 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 09:34:34.408156 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 09:34:34.408156 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 09:34:34.408156 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 31 09:34:34.408156 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application Aug 31 09:34:34.412335 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 326/647] 10.222.0.13 () {56 vars in 1102 bytes} [Wed Aug 31 09:34:34 2022] GET /identity/v3/roles/service => generated 85 bytes in 282 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:34:34.412335 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9edd863a-cd2a-4635-8699-4d8117896ffd None admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:34.427973 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae797a18-0e7b-44e8-b1a3-5d46d4c4bcdd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:34.429619 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae797a18-0e7b-44e8-b1a3-5d46d4c4bcdd None admin] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:34.430445 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae797a18-0e7b-44e8-b1a3-5d46d4c4bcdd None admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:34.431246 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae797a18-0e7b-44e8-b1a3-5d46d4c4bcdd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:34.455121 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae797a18-0e7b-44e8-b1a3-5d46d4c4bcdd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:34.458146 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae797a18-0e7b-44e8-b1a3-5d46d4c4bcdd None admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:34.497324 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application [None req-ae797a18-0e7b-44e8-b1a3-5d46d4c4bcdd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 09:34:34.497324 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:34:34.497324 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:34:34.497324 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:34:34.497324 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:34:34.497324 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:34:34.497324 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:34:34.497324 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:34:34.497324 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:34:34.497324 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:34:34.497324 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:34:34.497324 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:34:34.497324 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 09:34:34.497324 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 09:34:34.497324 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 09:34:34.497324 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 09:34:34.499145 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:34:34.499145 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:34:34.499145 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:34:34.499145 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:34:34.499145 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:34:34.499145 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:34:34.499145 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:34:34.499145 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application with Lock( Aug 31 09:34:34.499145 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:34:34.499145 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:34:34.499145 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:34:34.499145 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:34:34.499145 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:34:34.499145 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:34:34.499145 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:34:34.499145 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:34:34.499145 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 09:34:34.500917 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 09:34:34.500917 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 09:34:34.500917 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application Aug 31 09:34:34.509565 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 322/648] 10.222.0.13 () {56 vars in 1108 bytes} [Wed Aug 31 09:34:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 299 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:34:34.511405 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae797a18-0e7b-44e8-b1a3-5d46d4c4bcdd None admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:34.555417 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=100 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:34.623768 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9edd863a-cd2a-4635-8699-4d8117896ffd None admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:34.667052 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db695a5-dbdd-42c1-90aa-d5ecc79a30e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:34.667737 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db695a5-dbdd-42c1-90aa-d5ecc79a30e1 None admin] REQUEST_METHOD: `PUT` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:34.668049 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db695a5-dbdd-42c1-90aa-d5ecc79a30e1 None admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:34.668341 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db695a5-dbdd-42c1-90aa-d5ecc79a30e1 None admin] PATH_INFO: `/v3/projects/49548eb44d62415582af820ab66decee/users/202bb9db7a8a4977ba788a290ed9a3fa/roles/e24ba4fd91584883af350cc84bc96c1d` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:34.687319 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5db695a5-dbdd-42c1-90aa-d5ecc79a30e1 None admin] RBAC: Authorizing `identity:create_grant(project_id=49548eb44d62415582af820ab66decee, user_id=202bb9db7a8a4977ba788a290ed9a3fa, role_id=e24ba4fd91584883af350cc84bc96c1d)` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:34.689045 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5db695a5-dbdd-42c1-90aa-d5ecc79a30e1 None admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:34.711505 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae797a18-0e7b-44e8-b1a3-5d46d4c4bcdd None admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:34.713594 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 327/649] 10.222.0.13 () {58 vars in 1439 bytes} [Wed Aug 31 09:34:34 2022] PUT /identity/v3/projects/49548eb44d62415582af820ab66decee/users/202bb9db7a8a4977ba788a290ed9a3fa/roles/e24ba4fd91584883af350cc84bc96c1d => generated 0 bytes in 294 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:34:34.715914 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db695a5-dbdd-42c1-90aa-d5ecc79a30e1 None admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:34.793546 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e766c69-45f7-431d-9c10-29e089834f54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:34.795167 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e766c69-45f7-431d-9c10-29e089834f54 None admin] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:34.796769 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e766c69-45f7-431d-9c10-29e089834f54 None admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:34.797613 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e766c69-45f7-431d-9c10-29e089834f54 None admin] PATH_INFO: `/v3/roles` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:34.799283 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e766c69-45f7-431d-9c10-29e089834f54 None admin] RBAC: Adding query filter params (name=service) {{(pid=88114) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:34:34.800042 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e766c69-45f7-431d-9c10-29e089834f54 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:34.803138 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e766c69-45f7-431d-9c10-29e089834f54 None admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:34.828507 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 323/650] 10.222.0.13 () {56 vars in 1103 bytes} [Wed Aug 31 09:34:34 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 318 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:34:34.829671 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:34.832831 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e766c69-45f7-431d-9c10-29e089834f54 None admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:34.955458 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db695a5-dbdd-42c1-90aa-d5ecc79a30e1 None admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:35.023221 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=100 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:35.032548 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32269313-e0c6-48ef-a2b1-6f7ec5dee559 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:35.034075 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32269313-e0c6-48ef-a2b1-6f7ec5dee559 None admin] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:35.034871 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32269313-e0c6-48ef-a2b1-6f7ec5dee559 None admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:35.035595 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32269313-e0c6-48ef-a2b1-6f7ec5dee559 None admin] PATH_INFO: `/v3/domains` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:35.037044 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32269313-e0c6-48ef-a2b1-6f7ec5dee559 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88113) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:34:35.037870 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32269313-e0c6-48ef-a2b1-6f7ec5dee559 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:35.040635 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32269313-e0c6-48ef-a2b1-6f7ec5dee559 None admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:35.069522 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 328/651] 10.222.0.13 () {56 vars in 1109 bytes} [Wed Aug 31 09:34:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 349 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:34:35.071553 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32269313-e0c6-48ef-a2b1-6f7ec5dee559 None admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:35.073659 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e766c69-45f7-431d-9c10-29e089834f54 None admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:35.151383 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d808621-1f74-463d-9bf8-b1776acca159 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:35.152452 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d808621-1f74-463d-9bf8-b1776acca159 None admin] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:35.152874 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d808621-1f74-463d-9bf8-b1776acca159 None admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:35.153333 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d808621-1f74-463d-9bf8-b1776acca159 None admin] PATH_INFO: `/v3/roles/service` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:35.175821 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d808621-1f74-463d-9bf8-b1776acca159 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 09:34:35.185792 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application [None req-4d808621-1f74-463d-9bf8-b1776acca159 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 31 09:34:35.185792 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:34:35.185792 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:34:35.185792 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:34:35.185792 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:34:35.185792 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:34:35.185792 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:34:35.185792 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:34:35.185792 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:34:35.185792 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:34:35.185792 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:34:35.185792 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:34:35.185792 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 09:34:35.185792 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 09:34:35.185792 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 09:34:35.185792 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 09:34:35.188206 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 09:34:35.188206 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 09:34:35.188206 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 09:34:35.188206 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 09:34:35.188206 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:34:35.188206 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:34:35.188206 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:34:35.188206 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:34:35.188206 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:34:35.188206 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:34:35.188206 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:34:35.188206 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application with Lock( Aug 31 09:34:35.188206 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:34:35.188206 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:34:35.188206 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:34:35.188206 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:34:35.189803 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:34:35.189803 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:34:35.189803 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:34:35.189803 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:34:35.189803 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 09:34:35.189803 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 09:34:35.189803 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 09:34:35.189803 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 09:34:35.189803 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 09:34:35.189803 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 09:34:35.189803 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 31 09:34:35.189803 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application Aug 31 09:34:35.189803 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 324/652] 10.222.0.13 () {56 vars in 1102 bytes} [Wed Aug 31 09:34:34 2022] GET /identity/v3/roles/service => generated 85 bytes in 359 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:34:35.192441 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d808621-1f74-463d-9bf8-b1776acca159 None admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:35.278865 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32269313-e0c6-48ef-a2b1-6f7ec5dee559 None admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:35.280857 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=100 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:35.327316 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b38b0d61-a5ee-4176-9cbe-90d7a7d0c729 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:35.329096 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38b0d61-a5ee-4176-9cbe-90d7a7d0c729 None admin] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:35.329603 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38b0d61-a5ee-4176-9cbe-90d7a7d0c729 None admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:35.330086 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38b0d61-a5ee-4176-9cbe-90d7a7d0c729 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:35.349180 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b38b0d61-a5ee-4176-9cbe-90d7a7d0c729 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:35.352290 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b38b0d61-a5ee-4176-9cbe-90d7a7d0c729 None admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:35.367333 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d808621-1f74-463d-9bf8-b1776acca159 None admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:35.381908 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application [None req-b38b0d61-a5ee-4176-9cbe-90d7a7d0c729 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 09:34:35.381908 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:34:35.381908 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:34:35.381908 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:34:35.381908 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:34:35.381908 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:34:35.381908 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:34:35.381908 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:34:35.381908 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:34:35.381908 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:34:35.381908 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:34:35.381908 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:34:35.381908 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 09:34:35.381908 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 09:34:35.381908 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 09:34:35.381908 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 09:34:35.382972 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:34:35.382972 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:34:35.382972 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:34:35.382972 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:34:35.382972 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:34:35.382972 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:34:35.382972 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:34:35.382972 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application with Lock( Aug 31 09:34:35.382972 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:34:35.382972 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:34:35.382972 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:34:35.382972 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:34:35.382972 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:34:35.382972 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:34:35.382972 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:34:35.382972 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:34:35.382972 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 09:34:35.389133 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 09:34:35.389133 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 09:34:35.389133 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application Aug 31 09:34:35.389133 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 329/653] 10.222.0.13 () {56 vars in 1108 bytes} [Wed Aug 31 09:34:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 314 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:34:35.389133 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7a880f-81ce-4501-9164-1959b9f0f9e4 None None] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:35.389133 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7a880f-81ce-4501-9164-1959b9f0f9e4 None None] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:35.389133 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7a880f-81ce-4501-9164-1959b9f0f9e4 None None] PATH_INFO: `/` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:35.393813 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 330/654] 10.222.0.13 () {54 vars in 927 bytes} [Wed Aug 31 09:34:35 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:34:35.395649 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e7a880f-81ce-4501-9164-1959b9f0f9e4 None None] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:35.435942 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce0e56c-8bce-48a4-87b5-4dd2bd67ec07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:35.437512 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce0e56c-8bce-48a4-87b5-4dd2bd67ec07 None admin] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:35.437673 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce0e56c-8bce-48a4-87b5-4dd2bd67ec07 None admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:35.437920 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce0e56c-8bce-48a4-87b5-4dd2bd67ec07 None admin] PATH_INFO: `/v3/projects/service` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:35.446190 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ce0e56c-8bce-48a4-87b5-4dd2bd67ec07 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:35.447014 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ce0e56c-8bce-48a4-87b5-4dd2bd67ec07 None admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:35.449261 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=100 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:35.469839 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application [None req-3ce0e56c-8bce-48a4-87b5-4dd2bd67ec07 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 09:34:35.469839 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:34:35.469839 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:34:35.469839 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:34:35.469839 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:34:35.469839 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:34:35.469839 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:34:35.469839 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:34:35.469839 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:34:35.469839 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:34:35.469839 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:34:35.469839 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:34:35.469839 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 31 09:34:35.469839 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 31 09:34:35.469839 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 31 09:34:35.471553 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 31 09:34:35.471553 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:34:35.471553 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:34:35.471553 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:34:35.471553 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:34:35.471553 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:34:35.471553 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:34:35.471553 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:34:35.471553 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application with Lock( Aug 31 09:34:35.471553 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:34:35.471553 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:34:35.471553 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:34:35.471553 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:34:35.471553 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:34:35.471553 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:34:35.471553 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:34:35.471553 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:34:35.473272 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 31 09:34:35.473272 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 31 09:34:35.473272 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 31 09:34:35.473272 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 31 09:34:35.473272 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 31 09:34:35.473272 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 31 09:34:35.473272 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 09:34:35.473272 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application Aug 31 09:34:35.476333 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 325/655] 10.222.0.13 () {56 vars in 1111 bytes} [Wed Aug 31 09:34:35 2022] GET /identity/v3/projects/service => generated 88 bytes in 287 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:34:35.478302 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce0e56c-8bce-48a4-87b5-4dd2bd67ec07 None admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:35.575998 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e7a880f-81ce-4501-9164-1959b9f0f9e4 None None] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:35.631116 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce0e56c-8bce-48a4-87b5-4dd2bd67ec07 None admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:35.640952 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52bfb359-2176-4b94-924a-d26f5bd02da0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:35.642287 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52bfb359-2176-4b94-924a-d26f5bd02da0 None admin] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:35.642916 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52bfb359-2176-4b94-924a-d26f5bd02da0 None admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:35.643539 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52bfb359-2176-4b94-924a-d26f5bd02da0 None admin] PATH_INFO: `/v3/roles` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:35.645003 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52bfb359-2176-4b94-924a-d26f5bd02da0 None admin] RBAC: Adding query filter params (name=service) {{(pid=88113) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:34:35.645462 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52bfb359-2176-4b94-924a-d26f5bd02da0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:35.647878 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52bfb359-2176-4b94-924a-d26f5bd02da0 None admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:35.669081 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 331/656] 10.222.0.13 () {56 vars in 1103 bytes} [Wed Aug 31 09:34:35 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:34:35.669081 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a89e4a1-4e22-4176-93ad-a18cd8aee6d8 None None] REQUEST_METHOD: `POST` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:35.669081 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a89e4a1-4e22-4176-93ad-a18cd8aee6d8 None None] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:35.669081 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a89e4a1-4e22-4176-93ad-a18cd8aee6d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:35.695103 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.auth.core [None req-6a89e4a1-4e22-4176-93ad-a18cd8aee6d8 None None] MFA Rules not processed for user `fd836cf590a145df9acf19cf3ae2ac56`. Rule list: `[]` (Enabled: `True`). {{(pid=88113) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:34:35.697031 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.fernet_utils [None req-6a89e4a1-4e22-4176-93ad-a18cd8aee6d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:34:35.701818 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-035eb411-b1bc-44f4-9079-327b7a5033c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:35.702881 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035eb411-b1bc-44f4-9079-327b7a5033c2 None admin] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:35.703507 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035eb411-b1bc-44f4-9079-327b7a5033c2 None admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:35.703841 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035eb411-b1bc-44f4-9079-327b7a5033c2 None admin] PATH_INFO: `/v3/domains` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:35.704941 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-035eb411-b1bc-44f4-9079-327b7a5033c2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88114) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:34:35.705470 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-035eb411-b1bc-44f4-9079-327b7a5033c2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:35.707834 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-035eb411-b1bc-44f4-9079-327b7a5033c2 None admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:35.720256 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=100 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:35.733969 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 326/657] 10.222.0.13 () {56 vars in 1109 bytes} [Wed Aug 31 09:34:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:34:35.733969 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-035eb411-b1bc-44f4-9079-327b7a5033c2 None admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:35.828247 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 332/658] 10.222.0.13 () {58 vars in 1024 bytes} [Wed Aug 31 09:34:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 09:34:35.830213 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a89e4a1-4e22-4176-93ad-a18cd8aee6d8 None None] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:35.854044 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=100 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:35.949061 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-035eb411-b1bc-44f4-9079-327b7a5033c2 None admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:36.025703 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a91238-73f7-4cf8-a0bc-31b4f0f333e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:36.027127 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a91238-73f7-4cf8-a0bc-31b4f0f333e3 None admin] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:36.027820 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a91238-73f7-4cf8-a0bc-31b4f0f333e3 None admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:36.028692 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a91238-73f7-4cf8-a0bc-31b4f0f333e3 None admin] PATH_INFO: `/v3/projects` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:36.030906 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11a91238-73f7-4cf8-a0bc-31b4f0f333e3 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88114) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:34:36.031675 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11a91238-73f7-4cf8-a0bc-31b4f0f333e3 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:36.034638 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11a91238-73f7-4cf8-a0bc-31b4f0f333e3 None admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:36.045748 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a89e4a1-4e22-4176-93ad-a18cd8aee6d8 None None] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:36.066005 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 327/659] 10.222.0.13 () {56 vars in 1148 bytes} [Wed Aug 31 09:34:35 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 336 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:34:36.068109 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a91238-73f7-4cf8-a0bc-31b4f0f333e3 None admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:36.118132 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc1aff1d-13ee-4484-b805-27fb230e4b1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:36.119113 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1aff1d-13ee-4484-b805-27fb230e4b1e None admin] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:36.119599 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1aff1d-13ee-4484-b805-27fb230e4b1e None admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:36.120056 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1aff1d-13ee-4484-b805-27fb230e4b1e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:36.142459 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc1aff1d-13ee-4484-b805-27fb230e4b1e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:36.145611 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc1aff1d-13ee-4484-b805-27fb230e4b1e None admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:36.159637 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=100 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:36.177335 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application [None req-cc1aff1d-13ee-4484-b805-27fb230e4b1e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 09:34:36.177335 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:34:36.177335 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:34:36.177335 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:34:36.177335 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:34:36.177335 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:34:36.177335 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:34:36.177335 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:34:36.177335 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:34:36.177335 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:34:36.177335 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:34:36.177335 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:34:36.177335 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 09:34:36.177335 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 09:34:36.177335 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 09:34:36.177335 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 09:34:36.179518 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:34:36.179518 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:34:36.179518 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:34:36.179518 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:34:36.179518 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:34:36.179518 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:34:36.179518 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:34:36.179518 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application with Lock( Aug 31 09:34:36.179518 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:34:36.179518 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:34:36.179518 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:34:36.179518 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:34:36.179518 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:34:36.179518 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:34:36.179518 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:34:36.179518 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:34:36.179518 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 09:34:36.181625 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 09:34:36.181625 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 09:34:36.181625 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application Aug 31 09:34:36.189899 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 333/660] 10.222.0.13 () {56 vars in 1108 bytes} [Wed Aug 31 09:34:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 356 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:34:36.192339 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc1aff1d-13ee-4484-b805-27fb230e4b1e None admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:36.282662 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a91238-73f7-4cf8-a0bc-31b4f0f333e3 None admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:36.315150 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=100 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:36.345921 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d56c636b-0686-4b62-bc7b-62266d57a020 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:36.346899 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56c636b-0686-4b62-bc7b-62266d57a020 None admin] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:36.347381 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56c636b-0686-4b62-bc7b-62266d57a020 None admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:36.347808 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56c636b-0686-4b62-bc7b-62266d57a020 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:36.362791 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d56c636b-0686-4b62-bc7b-62266d57a020 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:36.365907 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d56c636b-0686-4b62-bc7b-62266d57a020 None admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:36.386824 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application [None req-d56c636b-0686-4b62-bc7b-62266d57a020 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 31 09:34:36.386824 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:34:36.386824 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:34:36.386824 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:34:36.386824 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:34:36.386824 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:34:36.386824 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:34:36.386824 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:34:36.386824 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:34:36.386824 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:34:36.386824 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:34:36.386824 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:34:36.386824 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 31 09:34:36.386824 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 31 09:34:36.386824 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 31 09:34:36.386824 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 31 09:34:36.387804 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:34:36.387804 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:34:36.387804 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 31 09:34:36.387804 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 09:34:36.387804 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 31 09:34:36.387804 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 09:34:36.387804 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:34:36.387804 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:34:36.387804 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:34:36.387804 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:34:36.387804 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:34:36.387804 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application with Lock( Aug 31 09:34:36.387804 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:34:36.387804 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:34:36.387804 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:34:36.387804 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:34:36.387804 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:34:36.387804 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:34:36.388750 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:34:36.388750 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:34:36.388750 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 31 09:34:36.388750 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 31 09:34:36.388750 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 31 09:34:36.388750 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 31 09:34:36.388750 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 31 09:34:36.388750 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 31 09:34:36.388750 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 31 09:34:36.388750 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application Aug 31 09:34:36.388750 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 328/661] 10.222.0.13 () {56 vars in 1099 bytes} [Wed Aug 31 09:34:36 2022] GET /identity/v3/users/cinder => generated 84 bytes in 322 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:34:36.390987 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d56c636b-0686-4b62-bc7b-62266d57a020 None admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:36.398990 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc1aff1d-13ee-4484-b805-27fb230e4b1e None admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:36.447167 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc4708d5-3a1e-432f-8e60-8160c09aab12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:36.448133 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4708d5-3a1e-432f-8e60-8160c09aab12 None admin] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:36.448556 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4708d5-3a1e-432f-8e60-8160c09aab12 None admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:36.449206 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4708d5-3a1e-432f-8e60-8160c09aab12 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:36.450273 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc4708d5-3a1e-432f-8e60-8160c09aab12 None admin] RBAC: Adding query filter params (role.id=e24ba4fd91584883af350cc84bc96c1d, scope.project.id=49548eb44d62415582af820ab66decee, user.id=be216d156b54452cbbf978c7734343aa) {{(pid=88113) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:34:36.450761 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc4708d5-3a1e-432f-8e60-8160c09aab12 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:36.453730 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc4708d5-3a1e-432f-8e60-8160c09aab12 None admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:36.471014 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 334/662] 10.222.0.13 () {56 vars in 1374 bytes} [Wed Aug 31 09:34:36 2022] GET /identity/v3/role_assignments?user.id=be216d156b54452cbbf978c7734343aa&scope.project.id=49548eb44d62415582af820ab66decee&role.id=e24ba4fd91584883af350cc84bc96c1d => generated 608 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:34:36.471014 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc4708d5-3a1e-432f-8e60-8160c09aab12 None admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:36.589806 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=100 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:36.600760 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d56c636b-0686-4b62-bc7b-62266d57a020 None admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:36.650805 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-522872ac-88c2-4f42-b774-1fc87fdcdd3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:36.651869 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522872ac-88c2-4f42-b774-1fc87fdcdd3c None admin] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:36.652425 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522872ac-88c2-4f42-b774-1fc87fdcdd3c None admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:36.652970 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522872ac-88c2-4f42-b774-1fc87fdcdd3c None admin] PATH_INFO: `/v3/domains` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:36.654275 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-522872ac-88c2-4f42-b774-1fc87fdcdd3c None admin] RBAC: Adding query filter params (name=Default) {{(pid=88114) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:34:36.654705 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc4708d5-3a1e-432f-8e60-8160c09aab12 None admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:36.654981 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-522872ac-88c2-4f42-b774-1fc87fdcdd3c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:36.659211 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-522872ac-88c2-4f42-b774-1fc87fdcdd3c None admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:36.683785 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 329/663] 10.222.0.13 () {56 vars in 1109 bytes} [Wed Aug 31 09:34:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 295 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:34:36.686268 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-522872ac-88c2-4f42-b774-1fc87fdcdd3c None admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:36.700216 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9736776-4997-41c9-8d85-b43ee0ca5349 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:36.701151 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9736776-4997-41c9-8d85-b43ee0ca5349 None admin] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:36.701616 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9736776-4997-41c9-8d85-b43ee0ca5349 None admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:36.702195 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9736776-4997-41c9-8d85-b43ee0ca5349 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:36.716373 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9736776-4997-41c9-8d85-b43ee0ca5349 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 09:34:36.723244 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application [None req-b9736776-4997-41c9-8d85-b43ee0ca5349 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 09:34:36.723244 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:34:36.723244 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:34:36.723244 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:34:36.723244 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:34:36.723244 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:34:36.723244 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:34:36.723244 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:34:36.723244 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:34:36.723244 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:34:36.723244 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:34:36.723244 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:34:36.723244 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 09:34:36.723244 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 09:34:36.723244 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 09:34:36.723244 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 09:34:36.724377 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 09:34:36.724377 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 09:34:36.724377 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 09:34:36.724377 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 09:34:36.724377 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:34:36.724377 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:34:36.724377 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:34:36.724377 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:34:36.724377 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:34:36.724377 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:34:36.724377 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:34:36.724377 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application with Lock( Aug 31 09:34:36.724377 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:34:36.724377 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:34:36.724377 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:34:36.724377 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:34:36.725309 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:34:36.725309 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:34:36.725309 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:34:36.725309 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:34:36.725309 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 09:34:36.725309 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 09:34:36.725309 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 09:34:36.725309 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 09:34:36.725309 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 09:34:36.725309 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 09:34:36.725309 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 09:34:36.725309 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application Aug 31 09:34:36.726026 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 335/664] 10.222.0.13 () {56 vars in 1096 bytes} [Wed Aug 31 09:34:36 2022] GET /identity/v3/roles/admin => generated 83 bytes in 261 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:34:36.726288 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9736776-4997-41c9-8d85-b43ee0ca5349 None admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:36.757533 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=100 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:36.853663 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-522872ac-88c2-4f42-b774-1fc87fdcdd3c None admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:36.884233 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9736776-4997-41c9-8d85-b43ee0ca5349 None admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:36.915955 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7440d8d-47c0-4f27-a3bf-24216f0b03f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:36.916837 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7440d8d-47c0-4f27-a3bf-24216f0b03f3 None admin] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:36.917268 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7440d8d-47c0-4f27-a3bf-24216f0b03f3 None admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:36.917720 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7440d8d-47c0-4f27-a3bf-24216f0b03f3 None admin] PATH_INFO: `/v3/users` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:36.918607 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7440d8d-47c0-4f27-a3bf-24216f0b03f3 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=88114) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:34:36.919066 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7440d8d-47c0-4f27-a3bf-24216f0b03f3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:36.920690 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7440d8d-47c0-4f27-a3bf-24216f0b03f3 None admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:36.926292 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=100 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:36.944835 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 330/665] 10.222.0.13 () {56 vars in 1137 bytes} [Wed Aug 31 09:34:36 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:34:36.945747 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7440d8d-47c0-4f27-a3bf-24216f0b03f3 None admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:36.954986 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f566af1-2a66-4690-803c-5899c1bc5555 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:36.956069 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f566af1-2a66-4690-803c-5899c1bc5555 None admin] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:36.956628 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f566af1-2a66-4690-803c-5899c1bc5555 None admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:36.957116 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f566af1-2a66-4690-803c-5899c1bc5555 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:36.976802 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f566af1-2a66-4690-803c-5899c1bc5555 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:36.980556 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f566af1-2a66-4690-803c-5899c1bc5555 None admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:37.005969 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application [None req-0f566af1-2a66-4690-803c-5899c1bc5555 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Aug 31 09:34:37.005969 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:34:37.005969 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:34:37.005969 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:34:37.005969 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:34:37.005969 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:34:37.005969 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:34:37.005969 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:34:37.005969 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:34:37.005969 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:34:37.005969 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:34:37.005969 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:34:37.005969 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 31 09:34:37.005969 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 31 09:34:37.005969 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 31 09:34:37.005969 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 31 09:34:37.007778 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:34:37.007778 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:34:37.007778 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 31 09:34:37.007778 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 09:34:37.007778 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 31 09:34:37.007778 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 09:34:37.007778 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:34:37.007778 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:34:37.007778 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:34:37.007778 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:34:37.007778 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:34:37.007778 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application with Lock( Aug 31 09:34:37.007778 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:34:37.007778 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:34:37.007778 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:34:37.007778 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:34:37.007778 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:34:37.007778 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:34:37.009309 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:34:37.009309 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:34:37.009309 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 31 09:34:37.009309 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 31 09:34:37.009309 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 31 09:34:37.009309 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 31 09:34:37.009309 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 31 09:34:37.009309 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 31 09:34:37.009309 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Aug 31 09:34:37.009309 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application Aug 31 09:34:37.015308 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 336/666] 10.222.0.13 () {56 vars in 1102 bytes} [Wed Aug 31 09:34:36 2022] GET /identity/v3/users/neutron => generated 85 bytes in 289 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:34:37.018879 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6842355-3c63-4d97-b417-beae26472b57 None None] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:37.019602 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6842355-3c63-4d97-b417-beae26472b57 None None] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:37.020069 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6842355-3c63-4d97-b417-beae26472b57 None None] PATH_INFO: `/` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:37.022672 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 337/667] 10.222.0.13 () {54 vars in 927 bytes} [Wed Aug 31 09:34:37 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:34:37.027670 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6842355-3c63-4d97-b417-beae26472b57 None None] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:37.075527 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=100 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:37.083766 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7440d8d-47c0-4f27-a3bf-24216f0b03f3 None admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:37.127886 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66208f18-3125-4618-a149-7cef22adc07d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:37.128612 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66208f18-3125-4618-a149-7cef22adc07d None admin] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:37.128911 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66208f18-3125-4618-a149-7cef22adc07d None admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:37.129166 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66208f18-3125-4618-a149-7cef22adc07d None admin] PATH_INFO: `/v3/roles` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:37.129864 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66208f18-3125-4618-a149-7cef22adc07d None admin] RBAC: Adding query filter params (name=admin) {{(pid=88114) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:34:37.130225 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66208f18-3125-4618-a149-7cef22adc07d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:37.131745 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66208f18-3125-4618-a149-7cef22adc07d None admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:37.145860 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 331/668] 10.222.0.13 () {56 vars in 1099 bytes} [Wed Aug 31 09:34:36 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:34:37.148280 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66208f18-3125-4618-a149-7cef22adc07d None admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:37.223707 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6842355-3c63-4d97-b417-beae26472b57 None None] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:37.287957 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ad5c8a-f5f9-41e8-a519-035bacce2642 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:37.288927 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ad5c8a-f5f9-41e8-a519-035bacce2642 None admin] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:37.289431 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ad5c8a-f5f9-41e8-a519-035bacce2642 None admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:37.290292 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ad5c8a-f5f9-41e8-a519-035bacce2642 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:37.305487 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66208f18-3125-4618-a149-7cef22adc07d None admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:37.368433 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e6bed9c-320c-4862-a2c5-e4ee86ae6053 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:37.369258 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6bed9c-320c-4862-a2c5-e4ee86ae6053 None admin] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:37.369686 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6bed9c-320c-4862-a2c5-e4ee86ae6053 None admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:37.370127 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6bed9c-320c-4862-a2c5-e4ee86ae6053 None admin] PATH_INFO: `/v3/users` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:37.371355 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e6bed9c-320c-4862-a2c5-e4ee86ae6053 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=88114) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:34:37.371858 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e6bed9c-320c-4862-a2c5-e4ee86ae6053 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:37.374666 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e6bed9c-320c-4862-a2c5-e4ee86ae6053 None admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:37.376037 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=100 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:37.412675 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 332/669] 10.222.0.13 () {56 vars in 1139 bytes} [Wed Aug 31 09:34:37 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:34:37.412675 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51c4227-b538-48b5-9d8e-abc36b381dbf None None] REQUEST_METHOD: `POST` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:37.412675 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51c4227-b538-48b5-9d8e-abc36b381dbf None None] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:37.412675 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51c4227-b538-48b5-9d8e-abc36b381dbf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:37.416127 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2ad5c8a-f5f9-41e8-a519-035bacce2642 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:37.418772 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2ad5c8a-f5f9-41e8-a519-035bacce2642 None admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:37.438703 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.auth.core [None req-a51c4227-b538-48b5-9d8e-abc36b381dbf None None] MFA Rules not processed for user `fd836cf590a145df9acf19cf3ae2ac56`. Rule list: `[]` (Enabled: `True`). {{(pid=88114) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:34:37.440026 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.fernet_utils [None req-a51c4227-b538-48b5-9d8e-abc36b381dbf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88114) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:34:37.454107 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=100 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:37.545540 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 333/670] 10.222.0.13 () {58 vars in 1024 bytes} [Wed Aug 31 09:34:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 09:34:37.545540 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a51c4227-b538-48b5-9d8e-abc36b381dbf None None] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:37.633145 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 338/671] 10.222.0.13 () {58 vars in 1282 bytes} [Wed Aug 31 09:34:37 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 610 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 31 09:34:37.634439 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ad5c8a-f5f9-41e8-a519-035bacce2642 None admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:37.739469 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a51c4227-b538-48b5-9d8e-abc36b381dbf None None] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:37.764941 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=100 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:37.791076 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ad5c8a-f5f9-41e8-a519-035bacce2642 None admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:37.807216 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea5fcf8-d12f-4a15-9bd2-d1d9e8472796 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:37.808620 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea5fcf8-d12f-4a15-9bd2-d1d9e8472796 None admin] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:37.809474 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea5fcf8-d12f-4a15-9bd2-d1d9e8472796 None admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:37.810280 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea5fcf8-d12f-4a15-9bd2-d1d9e8472796 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:37.831044 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cea5fcf8-d12f-4a15-9bd2-d1d9e8472796 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:37.833950 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cea5fcf8-d12f-4a15-9bd2-d1d9e8472796 None admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:37.835947 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72576c5d-0925-4091-97a7-1d549c690b78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:37.836584 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72576c5d-0925-4091-97a7-1d549c690b78 None admin] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:37.836863 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72576c5d-0925-4091-97a7-1d549c690b78 None admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:37.837137 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72576c5d-0925-4091-97a7-1d549c690b78 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:37.856598 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=100 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:37.876154 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application [None req-cea5fcf8-d12f-4a15-9bd2-d1d9e8472796 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 09:34:37.876154 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:34:37.876154 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:34:37.876154 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:34:37.876154 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:34:37.876154 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:34:37.876154 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:34:37.876154 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:34:37.876154 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:34:37.876154 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:34:37.876154 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:34:37.876154 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:34:37.876154 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 09:34:37.876154 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 09:34:37.876154 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 09:34:37.876154 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 09:34:37.877846 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:34:37.877846 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:34:37.877846 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:34:37.877846 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:34:37.877846 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:34:37.877846 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:34:37.877846 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:34:37.877846 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application with Lock( Aug 31 09:34:37.877846 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:34:37.877846 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:34:37.877846 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:34:37.877846 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:34:37.877846 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:34:37.877846 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:34:37.877846 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:34:37.877846 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:34:37.877846 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 09:34:37.879636 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 09:34:37.879636 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 09:34:37.879636 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application Aug 31 09:34:37.882776 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 334/672] 10.222.0.13 () {56 vars in 1108 bytes} [Wed Aug 31 09:34:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 341 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:34:37.884730 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea5fcf8-d12f-4a15-9bd2-d1d9e8472796 None admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:37.943361 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72576c5d-0925-4091-97a7-1d549c690b78 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:37.946958 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72576c5d-0925-4091-97a7-1d549c690b78 None admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:38.065811 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea5fcf8-d12f-4a15-9bd2-d1d9e8472796 None admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:38.091657 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=100 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:38.108124 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa3abe6-9a30-40f2-a99e-3f5245fd0a42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:38.108962 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa3abe6-9a30-40f2-a99e-3f5245fd0a42 None admin] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:38.109253 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa3abe6-9a30-40f2-a99e-3f5245fd0a42 None admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:38.109549 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa3abe6-9a30-40f2-a99e-3f5245fd0a42 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:38.120372 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 339/673] 10.222.0.13 () {58 vars in 1282 bytes} [Wed Aug 31 09:34:37 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 486 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 31 09:34:38.121719 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72576c5d-0925-4091-97a7-1d549c690b78 None admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:38.126433 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fa3abe6-9a30-40f2-a99e-3f5245fd0a42 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:38.128019 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fa3abe6-9a30-40f2-a99e-3f5245fd0a42 None admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:38.151044 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application [None req-2fa3abe6-9a30-40f2-a99e-3f5245fd0a42 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 09:34:38.151044 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:34:38.151044 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:34:38.151044 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:34:38.151044 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:34:38.151044 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:34:38.151044 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:34:38.151044 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:34:38.151044 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:34:38.151044 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:34:38.151044 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:34:38.151044 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:34:38.151044 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 09:34:38.151044 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 09:34:38.151044 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 09:34:38.151044 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 09:34:38.153094 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:34:38.153094 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:34:38.153094 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:34:38.153094 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:34:38.153094 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:34:38.153094 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:34:38.153094 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:34:38.153094 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application with Lock( Aug 31 09:34:38.153094 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:34:38.153094 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:34:38.153094 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:34:38.153094 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:34:38.153094 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:34:38.153094 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:34:38.153094 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:34:38.153094 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:34:38.153094 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 09:34:38.156685 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 09:34:38.156685 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 09:34:38.156685 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application Aug 31 09:34:38.156685 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 335/674] 10.222.0.13 () {56 vars in 1108 bytes} [Wed Aug 31 09:34:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 269 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:34:38.156685 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa3abe6-9a30-40f2-a99e-3f5245fd0a42 None admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:38.251966 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=100 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:38.285325 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72576c5d-0925-4091-97a7-1d549c690b78 None admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:38.353182 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa3abe6-9a30-40f2-a99e-3f5245fd0a42 None admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:38.354029 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c4b34c8-14d1-407c-8a71-2e9f1796463d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:38.354666 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4b34c8-14d1-407c-8a71-2e9f1796463d None admin] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:38.355140 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4b34c8-14d1-407c-8a71-2e9f1796463d None admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:38.355640 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4b34c8-14d1-407c-8a71-2e9f1796463d None admin] PATH_INFO: `/v3/domains` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:38.356847 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c4b34c8-14d1-407c-8a71-2e9f1796463d None admin] RBAC: Adding query filter params (name=Default) {{(pid=88113) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:34:38.357337 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c4b34c8-14d1-407c-8a71-2e9f1796463d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:38.359825 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c4b34c8-14d1-407c-8a71-2e9f1796463d None admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:38.386360 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 340/675] 10.222.0.13 () {56 vars in 1109 bytes} [Wed Aug 31 09:34:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:34:38.386360 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c4b34c8-14d1-407c-8a71-2e9f1796463d None admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:38.417762 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d341f92-5658-4eb8-bfb3-f1e9851027fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:38.418408 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d341f92-5658-4eb8-bfb3-f1e9851027fe None admin] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:38.418687 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d341f92-5658-4eb8-bfb3-f1e9851027fe None admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:38.418958 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d341f92-5658-4eb8-bfb3-f1e9851027fe None admin] PATH_INFO: `/v3/roles/service` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:38.432377 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d341f92-5658-4eb8-bfb3-f1e9851027fe None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 09:34:38.438269 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application [None req-6d341f92-5658-4eb8-bfb3-f1e9851027fe None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 31 09:34:38.438269 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:34:38.438269 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:34:38.438269 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:34:38.438269 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:34:38.438269 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:34:38.438269 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:34:38.438269 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:34:38.438269 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:34:38.438269 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:34:38.438269 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:34:38.438269 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:34:38.438269 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 09:34:38.438269 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 09:34:38.438269 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 09:34:38.438269 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 09:34:38.440168 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 09:34:38.440168 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 09:34:38.440168 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 09:34:38.440168 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 09:34:38.440168 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:34:38.440168 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:34:38.440168 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:34:38.440168 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:34:38.440168 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:34:38.440168 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:34:38.440168 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:34:38.440168 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application with Lock( Aug 31 09:34:38.440168 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:34:38.440168 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:34:38.440168 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:34:38.440168 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:34:38.441953 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:34:38.441953 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:34:38.441953 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:34:38.441953 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:34:38.441953 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 09:34:38.441953 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 09:34:38.441953 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 09:34:38.441953 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 09:34:38.441953 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 09:34:38.441953 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 09:34:38.441953 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 31 09:34:38.441953 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application Aug 31 09:34:38.441953 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 336/676] 10.222.0.13 () {56 vars in 1102 bytes} [Wed Aug 31 09:34:38 2022] GET /identity/v3/roles/service => generated 85 bytes in 286 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:34:38.441953 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d341f92-5658-4eb8-bfb3-f1e9851027fe None admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:38.459421 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=100 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:38.529104 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c4b34c8-14d1-407c-8a71-2e9f1796463d None admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:38.574047 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d4b31eb-5148-4862-8617-929fecd00520 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:38.574706 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d341f92-5658-4eb8-bfb3-f1e9851027fe None admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:38.574970 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4b31eb-5148-4862-8617-929fecd00520 None admin] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:38.575366 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4b31eb-5148-4862-8617-929fecd00520 None admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:38.575819 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4b31eb-5148-4862-8617-929fecd00520 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:38.588026 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=100 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:38.595364 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d4b31eb-5148-4862-8617-929fecd00520 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:38.598155 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d4b31eb-5148-4862-8617-929fecd00520 None admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:38.621481 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application [None req-1d4b31eb-5148-4862-8617-929fecd00520 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 09:34:38.621481 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:34:38.621481 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:34:38.621481 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:34:38.621481 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:34:38.621481 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:34:38.621481 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:34:38.621481 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:34:38.621481 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:34:38.621481 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:34:38.621481 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:34:38.621481 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:34:38.621481 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 09:34:38.621481 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 09:34:38.621481 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 09:34:38.621481 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 09:34:38.631527 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:34:38.631527 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:34:38.631527 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:34:38.631527 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:34:38.631527 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:34:38.631527 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:34:38.631527 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:34:38.631527 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application with Lock( Aug 31 09:34:38.631527 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:34:38.631527 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:34:38.631527 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:34:38.631527 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:34:38.631527 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:34:38.631527 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:34:38.631527 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:34:38.631527 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:34:38.631527 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 09:34:38.632978 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 09:34:38.632978 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 09:34:38.632978 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application Aug 31 09:34:38.633221 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f107de0f-9e6b-43b4-a071-03568d18f163 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:38.633221 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f107de0f-9e6b-43b4-a071-03568d18f163 None admin] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:38.633221 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f107de0f-9e6b-43b4-a071-03568d18f163 None admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:38.633566 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 341/677] 10.222.0.13 () {56 vars in 1108 bytes} [Wed Aug 31 09:34:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:34:38.633652 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f107de0f-9e6b-43b4-a071-03568d18f163 None admin] PATH_INFO: `/v3/domains` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:38.633652 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f107de0f-9e6b-43b4-a071-03568d18f163 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88114) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:34:38.633807 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d4b31eb-5148-4862-8617-929fecd00520 None admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:38.636135 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f107de0f-9e6b-43b4-a071-03568d18f163 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:38.636135 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f107de0f-9e6b-43b4-a071-03568d18f163 None admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:38.649692 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 337/678] 10.222.0.13 () {56 vars in 1109 bytes} [Wed Aug 31 09:34:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:34:38.650844 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f107de0f-9e6b-43b4-a071-03568d18f163 None admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:38.791267 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=100 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:38.805774 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d4b31eb-5148-4862-8617-929fecd00520 None admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:38.815175 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f107de0f-9e6b-43b4-a071-03568d18f163 None admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:38.877909 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a4f690-f19a-4079-a228-c21d01aba971 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:38.878676 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ca01e5-73d2-446f-bc40-51f1a8be2758 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:38.878798 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a4f690-f19a-4079-a228-c21d01aba971 None admin] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:38.879199 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a4f690-f19a-4079-a228-c21d01aba971 None admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:38.879652 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a4f690-f19a-4079-a228-c21d01aba971 None admin] PATH_INFO: `/v3/roles` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:38.879929 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ca01e5-73d2-446f-bc40-51f1a8be2758 None admin] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:38.880573 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ca01e5-73d2-446f-bc40-51f1a8be2758 None admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:38.880803 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1a4f690-f19a-4079-a228-c21d01aba971 None admin] RBAC: Adding query filter params (name=service) {{(pid=88114) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:34:38.881217 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ca01e5-73d2-446f-bc40-51f1a8be2758 None admin] PATH_INFO: `/v3/users/nova` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:38.881313 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1a4f690-f19a-4079-a228-c21d01aba971 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:38.883728 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1a4f690-f19a-4079-a228-c21d01aba971 None admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:38.897714 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1ca01e5-73d2-446f-bc40-51f1a8be2758 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:38.904245 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 338/679] 10.222.0.13 () {56 vars in 1103 bytes} [Wed Aug 31 09:34:38 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:34:38.904245 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a4f690-f19a-4079-a228-c21d01aba971 None admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:38.904472 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1ca01e5-73d2-446f-bc40-51f1a8be2758 None admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:38.929723 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application [None req-e1ca01e5-73d2-446f-bc40-51f1a8be2758 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 31 09:34:38.929723 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:34:38.929723 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:34:38.929723 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:34:38.929723 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:34:38.929723 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:34:38.929723 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:34:38.929723 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:34:38.929723 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:34:38.929723 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:34:38.929723 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:34:38.929723 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:34:38.929723 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 31 09:34:38.929723 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 31 09:34:38.929723 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 31 09:34:38.929723 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 31 09:34:38.930751 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:34:38.930751 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:34:38.930751 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 31 09:34:38.930751 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 09:34:38.930751 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 31 09:34:38.930751 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 09:34:38.930751 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:34:38.930751 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:34:38.930751 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:34:38.930751 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:34:38.930751 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:34:38.930751 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application with Lock( Aug 31 09:34:38.930751 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:34:38.930751 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:34:38.930751 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:34:38.930751 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:34:38.930751 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:34:38.930751 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:34:38.931716 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:34:38.931716 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:34:38.931716 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 31 09:34:38.931716 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 31 09:34:38.931716 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 31 09:34:38.931716 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 31 09:34:38.931716 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 31 09:34:38.931716 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 31 09:34:38.931716 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 31 09:34:38.931716 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application Aug 31 09:34:38.932256 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 342/680] 10.222.0.13 () {56 vars in 1093 bytes} [Wed Aug 31 09:34:38 2022] GET /identity/v3/users/nova => generated 82 bytes in 308 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:34:38.934297 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ca01e5-73d2-446f-bc40-51f1a8be2758 None admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:38.960137 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=100 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:39.069061 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a4f690-f19a-4079-a228-c21d01aba971 None admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:39.104635 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a281f9e8-0298-414d-a718-5d6a338c36e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:39.105239 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a281f9e8-0298-414d-a718-5d6a338c36e9 None admin] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:39.105544 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a281f9e8-0298-414d-a718-5d6a338c36e9 None admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:39.105814 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a281f9e8-0298-414d-a718-5d6a338c36e9 None admin] PATH_INFO: `/v3/domains` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:39.106464 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a281f9e8-0298-414d-a718-5d6a338c36e9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88114) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:34:39.106765 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a281f9e8-0298-414d-a718-5d6a338c36e9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:39.108135 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a281f9e8-0298-414d-a718-5d6a338c36e9 None admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:39.115933 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ca01e5-73d2-446f-bc40-51f1a8be2758 None admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:39.122116 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 339/681] 10.222.0.13 () {56 vars in 1109 bytes} [Wed Aug 31 09:34:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:34:39.122978 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a281f9e8-0298-414d-a718-5d6a338c36e9 None admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:39.171700 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c389c5ab-6b58-456b-a596-57b61cea5571 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:39.172529 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c389c5ab-6b58-456b-a596-57b61cea5571 None admin] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:39.173080 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c389c5ab-6b58-456b-a596-57b61cea5571 None admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:39.173681 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c389c5ab-6b58-456b-a596-57b61cea5571 None admin] PATH_INFO: `/v3/projects/service` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:39.184144 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c389c5ab-6b58-456b-a596-57b61cea5571 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:39.186398 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c389c5ab-6b58-456b-a596-57b61cea5571 None admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:39.205468 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application [None req-c389c5ab-6b58-456b-a596-57b61cea5571 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 09:34:39.205468 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:34:39.205468 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:34:39.205468 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:34:39.205468 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:34:39.205468 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:34:39.205468 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:34:39.205468 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:34:39.205468 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:34:39.205468 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:34:39.205468 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:34:39.205468 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:34:39.205468 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 31 09:34:39.205468 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 31 09:34:39.205468 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 31 09:34:39.207570 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 31 09:34:39.207570 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:34:39.207570 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:34:39.207570 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:34:39.207570 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:34:39.207570 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:34:39.207570 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:34:39.207570 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:34:39.207570 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application with Lock( Aug 31 09:34:39.207570 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:34:39.207570 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:34:39.207570 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:34:39.207570 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:34:39.207570 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:34:39.207570 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:34:39.207570 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:34:39.207570 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:34:39.209445 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 31 09:34:39.209445 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 31 09:34:39.209445 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 31 09:34:39.209445 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 31 09:34:39.209445 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 31 09:34:39.209445 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 31 09:34:39.209445 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 09:34:39.209445 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application Aug 31 09:34:39.209445 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 343/682] 10.222.0.13 () {56 vars in 1111 bytes} [Wed Aug 31 09:34:38 2022] GET /identity/v3/projects/service => generated 88 bytes in 276 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:34:39.210602 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c389c5ab-6b58-456b-a596-57b61cea5571 None admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:39.237377 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=100 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:39.256247 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=100 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:39.271771 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a281f9e8-0298-414d-a718-5d6a338c36e9 None admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:39.315380 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce850d1-b6dc-4b17-9950-388e1178ae80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:39.316318 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce850d1-b6dc-4b17-9950-388e1178ae80 None admin] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:39.316736 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce850d1-b6dc-4b17-9950-388e1178ae80 None admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:39.317039 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce850d1-b6dc-4b17-9950-388e1178ae80 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:39.330056 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ce850d1-b6dc-4b17-9950-388e1178ae80 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:39.331852 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ce850d1-b6dc-4b17-9950-388e1178ae80 None admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:39.349217 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application [None req-8ce850d1-b6dc-4b17-9950-388e1178ae80 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 09:34:39.349217 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:34:39.349217 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:34:39.349217 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:34:39.349217 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:34:39.349217 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:34:39.349217 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:34:39.349217 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:34:39.349217 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:34:39.349217 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:34:39.349217 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:34:39.349217 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:34:39.349217 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 09:34:39.349217 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 09:34:39.349217 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 09:34:39.349217 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 09:34:39.351361 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:34:39.351361 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:34:39.351361 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:34:39.351361 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:34:39.351361 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:34:39.351361 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:34:39.351361 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:34:39.351361 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application with Lock( Aug 31 09:34:39.351361 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:34:39.351361 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:34:39.351361 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:34:39.351361 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:34:39.351361 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:34:39.351361 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:34:39.351361 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:34:39.351361 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:34:39.351361 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 09:34:39.355105 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 09:34:39.355105 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 09:34:39.355105 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application Aug 31 09:34:39.355105 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 340/683] 10.222.0.13 () {56 vars in 1108 bytes} [Wed Aug 31 09:34:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:34:39.355105 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5671626-ca07-48bd-b51c-bb5638c3aad0 None None] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:39.355105 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5671626-ca07-48bd-b51c-bb5638c3aad0 None None] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:39.355105 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5671626-ca07-48bd-b51c-bb5638c3aad0 None None] PATH_INFO: `/` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:39.355105 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 341/684] 10.222.0.13 () {54 vars in 927 bytes} [Wed Aug 31 09:34:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:34:39.355105 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5671626-ca07-48bd-b51c-bb5638c3aad0 None None] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:39.372529 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c389c5ab-6b58-456b-a596-57b61cea5571 None admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:39.431623 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1cfcaac-72bf-4ba3-b819-38ea7c07773b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:39.432561 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1cfcaac-72bf-4ba3-b819-38ea7c07773b None admin] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:39.433109 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1cfcaac-72bf-4ba3-b819-38ea7c07773b None admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:39.433727 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1cfcaac-72bf-4ba3-b819-38ea7c07773b None admin] PATH_INFO: `/v3/users` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:39.435349 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1cfcaac-72bf-4ba3-b819-38ea7c07773b None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=88113) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:34:39.435963 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1cfcaac-72bf-4ba3-b819-38ea7c07773b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:39.438805 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1cfcaac-72bf-4ba3-b819-38ea7c07773b None admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:39.473989 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 344/685] 10.222.0.13 () {56 vars in 1133 bytes} [Wed Aug 31 09:34:39 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:34:39.475288 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1cfcaac-72bf-4ba3-b819-38ea7c07773b None admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:39.493704 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5671626-ca07-48bd-b51c-bb5638c3aad0 None None] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:39.537700 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c09c2e-870b-4747-a59e-77a981aa3f9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:39.538688 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c09c2e-870b-4747-a59e-77a981aa3f9c None admin] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:39.539144 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c09c2e-870b-4747-a59e-77a981aa3f9c None admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:39.539594 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c09c2e-870b-4747-a59e-77a981aa3f9c None admin] PATH_INFO: `/v3/projects/service` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:39.550936 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80c09c2e-870b-4747-a59e-77a981aa3f9c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:39.553564 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80c09c2e-870b-4747-a59e-77a981aa3f9c None admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:39.575065 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application [None req-80c09c2e-870b-4747-a59e-77a981aa3f9c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 09:34:39.575065 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:34:39.575065 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:34:39.575065 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:34:39.575065 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:34:39.575065 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:34:39.575065 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:34:39.575065 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:34:39.575065 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:34:39.575065 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:34:39.575065 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:34:39.575065 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:34:39.575065 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 31 09:34:39.575065 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 31 09:34:39.575065 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 31 09:34:39.576761 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 31 09:34:39.576761 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:34:39.576761 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:34:39.576761 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:34:39.576761 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:34:39.576761 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:34:39.576761 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:34:39.576761 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:34:39.576761 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application with Lock( Aug 31 09:34:39.576761 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:34:39.576761 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:34:39.576761 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:34:39.576761 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:34:39.576761 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:34:39.576761 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:34:39.576761 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:34:39.576761 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:34:39.579497 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 31 09:34:39.579497 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 31 09:34:39.579497 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 31 09:34:39.579497 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 31 09:34:39.579497 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 31 09:34:39.579497 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 31 09:34:39.579497 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 09:34:39.579497 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application Aug 31 09:34:39.579497 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 342/686] 10.222.0.13 () {56 vars in 1111 bytes} [Wed Aug 31 09:34:39 2022] GET /identity/v3/projects/service => generated 88 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:34:39.579497 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c09c2e-870b-4747-a59e-77a981aa3f9c None admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:39.584472 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=100 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:39.605377 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=100 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:39.645808 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1cfcaac-72bf-4ba3-b819-38ea7c07773b None admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:39.701021 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1046e968-0534-4dae-b146-09d037f54397 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:39.702124 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1046e968-0534-4dae-b146-09d037f54397 None admin] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:39.702872 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1046e968-0534-4dae-b146-09d037f54397 None admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:39.703525 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1046e968-0534-4dae-b146-09d037f54397 None admin] PATH_INFO: `/v3/projects` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:39.705562 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1046e968-0534-4dae-b146-09d037f54397 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88113) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:34:39.706312 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1046e968-0534-4dae-b146-09d037f54397 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:39.709427 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1046e968-0534-4dae-b146-09d037f54397 None admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:39.734007 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 345/687] 10.222.0.13 () {56 vars in 1148 bytes} [Wed Aug 31 09:34:39 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:34:39.736052 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff67e5e7-a736-4bfe-a55f-69a76ff25248 None None] REQUEST_METHOD: `POST` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:39.736498 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff67e5e7-a736-4bfe-a55f-69a76ff25248 None None] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:39.736930 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff67e5e7-a736-4bfe-a55f-69a76ff25248 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:39.753428 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c09c2e-870b-4747-a59e-77a981aa3f9c None admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:39.758408 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.auth.core [None req-ff67e5e7-a736-4bfe-a55f-69a76ff25248 None None] MFA Rules not processed for user `fd836cf590a145df9acf19cf3ae2ac56`. Rule list: `[]` (Enabled: `True`). {{(pid=88113) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:34:39.759655 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.fernet_utils [None req-ff67e5e7-a736-4bfe-a55f-69a76ff25248 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:34:39.793598 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14aa1c47-785e-437e-917f-7beaf75df60c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:39.794544 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14aa1c47-785e-437e-917f-7beaf75df60c None admin] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:39.794993 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14aa1c47-785e-437e-917f-7beaf75df60c None admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:39.795436 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14aa1c47-785e-437e-917f-7beaf75df60c None admin] PATH_INFO: `/v3/domains` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:39.796414 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14aa1c47-785e-437e-917f-7beaf75df60c None admin] RBAC: Adding query filter params (name=Default) {{(pid=88114) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:34:39.796796 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14aa1c47-785e-437e-917f-7beaf75df60c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:39.799057 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14aa1c47-785e-437e-917f-7beaf75df60c None admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:39.824958 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 343/688] 10.222.0.13 () {56 vars in 1109 bytes} [Wed Aug 31 09:34:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:34:39.824958 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14aa1c47-785e-437e-917f-7beaf75df60c None admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:39.885504 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 346/689] 10.222.0.13 () {58 vars in 1024 bytes} [Wed Aug 31 09:34:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 09:34:39.885504 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff67e5e7-a736-4bfe-a55f-69a76ff25248 None None] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:39.936949 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=100 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:39.974915 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=100 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:40.007500 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14aa1c47-785e-437e-917f-7beaf75df60c None admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:40.041912 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff67e5e7-a736-4bfe-a55f-69a76ff25248 None None] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:40.071306 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ed8036-418b-4356-85dc-462a0a3503b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:40.072346 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ed8036-418b-4356-85dc-462a0a3503b6 None admin] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:40.072823 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ed8036-418b-4356-85dc-462a0a3503b6 None admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:40.073247 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ed8036-418b-4356-85dc-462a0a3503b6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:40.087501 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a07ba3b3-c8c4-4dfe-ad23-ae2f092ff926 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:40.088293 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07ba3b3-c8c4-4dfe-ad23-ae2f092ff926 None admin] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:40.088715 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07ba3b3-c8c4-4dfe-ad23-ae2f092ff926 None admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:40.089429 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07ba3b3-c8c4-4dfe-ad23-ae2f092ff926 None admin] PATH_INFO: `/v3/projects` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:40.090407 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a07ba3b3-c8c4-4dfe-ad23-ae2f092ff926 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88113) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:34:40.090876 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a07ba3b3-c8c4-4dfe-ad23-ae2f092ff926 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:40.092570 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a07ba3b3-c8c4-4dfe-ad23-ae2f092ff926 None admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:40.108233 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 347/690] 10.222.0.13 () {56 vars in 1148 bytes} [Wed Aug 31 09:34:39 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:34:40.109491 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a07ba3b3-c8c4-4dfe-ad23-ae2f092ff926 None admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:40.195349 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57ed8036-418b-4356-85dc-462a0a3503b6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:40.197242 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57ed8036-418b-4356-85dc-462a0a3503b6 None admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:40.257530 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a07ba3b3-c8c4-4dfe-ad23-ae2f092ff926 None admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:40.276343 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=100 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:40.293772 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=100 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:40.310119 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-339a370d-f99b-427a-a410-4c3fec89fb45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:40.310955 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339a370d-f99b-427a-a410-4c3fec89fb45 None admin] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:40.311296 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339a370d-f99b-427a-a410-4c3fec89fb45 None admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:40.311826 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339a370d-f99b-427a-a410-4c3fec89fb45 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:40.312869 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-339a370d-f99b-427a-a410-4c3fec89fb45 None admin] RBAC: Adding query filter params (role.id=e24ba4fd91584883af350cc84bc96c1d, scope.project.id=49548eb44d62415582af820ab66decee, user.id=0d52b69c8e94441dacfcff944326f169) {{(pid=88113) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:34:40.313323 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-339a370d-f99b-427a-a410-4c3fec89fb45 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:40.315253 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-339a370d-f99b-427a-a410-4c3fec89fb45 None admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:40.323540 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 348/691] 10.222.0.13 () {56 vars in 1374 bytes} [Wed Aug 31 09:34:40 2022] GET /identity/v3/role_assignments?user.id=0d52b69c8e94441dacfcff944326f169&scope.project.id=49548eb44d62415582af820ab66decee&role.id=e24ba4fd91584883af350cc84bc96c1d => generated 608 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:34:40.324612 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-339a370d-f99b-427a-a410-4c3fec89fb45 None admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:40.328193 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 344/692] 10.222.0.13 () {58 vars in 1282 bytes} [Wed Aug 31 09:34:39 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 508 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 31 09:34:40.329736 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ed8036-418b-4356-85dc-462a0a3503b6 None admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:40.506191 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-339a370d-f99b-427a-a410-4c3fec89fb45 None admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:40.523067 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ed8036-418b-4356-85dc-462a0a3503b6 None admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:40.578259 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca5bf3cc-34d0-4f4e-8673-1d724d4917c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:40.579487 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5bf3cc-34d0-4f4e-8673-1d724d4917c2 None admin] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:40.579983 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5bf3cc-34d0-4f4e-8673-1d724d4917c2 None admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:40.580466 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5bf3cc-34d0-4f4e-8673-1d724d4917c2 None admin] PATH_INFO: `/v3/users/swift` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:40.589425 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-563127f3-2b8d-48a4-a408-9e39b0817b2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:40.590422 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563127f3-2b8d-48a4-a408-9e39b0817b2f None admin] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:40.590877 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563127f3-2b8d-48a4-a408-9e39b0817b2f None admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:40.591296 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563127f3-2b8d-48a4-a408-9e39b0817b2f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:40.592195 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-563127f3-2b8d-48a4-a408-9e39b0817b2f None admin] RBAC: Adding query filter params (role.id=e24ba4fd91584883af350cc84bc96c1d, scope.project.id=49548eb44d62415582af820ab66decee, user.id=a9bdaacd597442c2bcde2c555f2cc140) {{(pid=88114) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:34:40.593222 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-563127f3-2b8d-48a4-a408-9e39b0817b2f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:40.596249 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca5bf3cc-34d0-4f4e-8673-1d724d4917c2 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:40.596506 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-563127f3-2b8d-48a4-a408-9e39b0817b2f None admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:40.599318 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca5bf3cc-34d0-4f4e-8673-1d724d4917c2 None admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:40.606580 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 345/693] 10.222.0.13 () {56 vars in 1374 bytes} [Wed Aug 31 09:34:40 2022] GET /identity/v3/role_assignments?user.id=a9bdaacd597442c2bcde2c555f2cc140&scope.project.id=49548eb44d62415582af820ab66decee&role.id=e24ba4fd91584883af350cc84bc96c1d => generated 608 bytes in 278 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:34:40.607493 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-563127f3-2b8d-48a4-a408-9e39b0817b2f None admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:40.623158 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application [None req-ca5bf3cc-34d0-4f4e-8673-1d724d4917c2 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 31 09:34:40.623158 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:34:40.623158 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:34:40.623158 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:34:40.623158 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:34:40.623158 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:34:40.623158 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:34:40.623158 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:34:40.623158 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:34:40.623158 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:34:40.623158 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:34:40.623158 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:34:40.623158 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 31 09:34:40.623158 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 31 09:34:40.623158 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 31 09:34:40.623158 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 31 09:34:40.626011 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:34:40.626011 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:34:40.626011 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 31 09:34:40.626011 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 09:34:40.626011 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 31 09:34:40.626011 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 09:34:40.626011 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:34:40.626011 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:34:40.626011 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:34:40.626011 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:34:40.626011 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:34:40.626011 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application with Lock( Aug 31 09:34:40.626011 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:34:40.626011 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:34:40.626011 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:34:40.626011 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:34:40.626011 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:34:40.626011 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:34:40.628152 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:34:40.628152 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:34:40.628152 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 31 09:34:40.628152 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 31 09:34:40.628152 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 31 09:34:40.628152 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 31 09:34:40.628152 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 31 09:34:40.628152 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 31 09:34:40.628152 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 31 09:34:40.628152 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application Aug 31 09:34:40.628152 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 349/694] 10.222.0.13 () {56 vars in 1096 bytes} [Wed Aug 31 09:34:40 2022] GET /identity/v3/users/swift => generated 83 bytes in 302 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:34:40.628152 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca5bf3cc-34d0-4f4e-8673-1d724d4917c2 None admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:40.659485 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=100 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:40.735826 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=100 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:40.756359 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-563127f3-2b8d-48a4-a408-9e39b0817b2f None admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:40.814323 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-028ed3f6-06c3-48c4-b35c-7071578c5767 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:40.815351 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028ed3f6-06c3-48c4-b35c-7071578c5767 None admin] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:40.815768 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028ed3f6-06c3-48c4-b35c-7071578c5767 None admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:40.816309 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028ed3f6-06c3-48c4-b35c-7071578c5767 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:40.830356 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-028ed3f6-06c3-48c4-b35c-7071578c5767 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:40.832083 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-028ed3f6-06c3-48c4-b35c-7071578c5767 None admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:40.835314 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca5bf3cc-34d0-4f4e-8673-1d724d4917c2 None admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:40.849995 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application [None req-028ed3f6-06c3-48c4-b35c-7071578c5767 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 09:34:40.849995 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:34:40.849995 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:34:40.849995 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:34:40.849995 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:34:40.849995 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:34:40.849995 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:34:40.849995 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:34:40.849995 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:34:40.849995 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:34:40.849995 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:34:40.849995 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:34:40.849995 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 09:34:40.849995 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 09:34:40.849995 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 09:34:40.849995 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 09:34:40.851028 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:34:40.851028 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:34:40.851028 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:34:40.851028 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:34:40.851028 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:34:40.851028 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:34:40.851028 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:34:40.851028 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application with Lock( Aug 31 09:34:40.851028 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:34:40.851028 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:34:40.851028 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:34:40.851028 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:34:40.851028 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:34:40.851028 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:34:40.851028 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:34:40.851028 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:34:40.851028 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 09:34:40.851932 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 09:34:40.851932 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 09:34:40.851932 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application Aug 31 09:34:40.851932 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 346/695] 10.222.0.13 () {56 vars in 1108 bytes} [Wed Aug 31 09:34:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:34:40.851932 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-028ed3f6-06c3-48c4-b35c-7071578c5767 None admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:40.888358 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb624e7-384c-4261-bbe7-eda3d894383b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:40.889450 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb624e7-384c-4261-bbe7-eda3d894383b None admin] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:40.889781 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb624e7-384c-4261-bbe7-eda3d894383b None admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:40.890060 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb624e7-384c-4261-bbe7-eda3d894383b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:40.903781 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eb624e7-384c-4261-bbe7-eda3d894383b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:40.905835 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eb624e7-384c-4261-bbe7-eda3d894383b None admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:40.925528 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application [None req-1eb624e7-384c-4261-bbe7-eda3d894383b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 09:34:40.925528 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:34:40.925528 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:34:40.925528 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:34:40.925528 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:34:40.925528 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:34:40.925528 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:34:40.925528 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:34:40.925528 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:34:40.925528 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:34:40.925528 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:34:40.925528 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:34:40.925528 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 09:34:40.925528 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 09:34:40.925528 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 09:34:40.925528 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 09:34:40.929143 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:34:40.929143 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:34:40.929143 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:34:40.929143 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:34:40.929143 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:34:40.929143 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:34:40.929143 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:34:40.929143 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application with Lock( Aug 31 09:34:40.929143 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:34:40.929143 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:34:40.929143 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:34:40.929143 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:34:40.929143 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:34:40.929143 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:34:40.929143 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:34:40.929143 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:34:40.929143 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 09:34:40.931486 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 09:34:40.931486 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 09:34:40.931486 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application Aug 31 09:34:40.931486 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 350/696] 10.222.0.13 () {56 vars in 1108 bytes} [Wed Aug 31 09:34:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 300 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:34:40.931486 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb624e7-384c-4261-bbe7-eda3d894383b None admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:41.028642 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=100 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:41.070869 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=100 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:41.076907 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-028ed3f6-06c3-48c4-b35c-7071578c5767 None admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:41.079331 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb624e7-384c-4261-bbe7-eda3d894383b None admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:41.120451 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0713fde6-d946-45f8-9333-1796eb7a5524 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:41.121073 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0713fde6-d946-45f8-9333-1796eb7a5524 None admin] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:41.121349 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0713fde6-d946-45f8-9333-1796eb7a5524 None admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:41.121649 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0713fde6-d946-45f8-9333-1796eb7a5524 None admin] PATH_INFO: `/v3/domains` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:41.122303 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0713fde6-d946-45f8-9333-1796eb7a5524 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88113) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:34:41.122966 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0713fde6-d946-45f8-9333-1796eb7a5524 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:41.124776 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0713fde6-d946-45f8-9333-1796eb7a5524 None admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:41.146395 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 351/697] 10.222.0.13 () {56 vars in 1109 bytes} [Wed Aug 31 09:34:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:34:41.148309 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0713fde6-d946-45f8-9333-1796eb7a5524 None admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:41.168151 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fa2dd0a-19a4-4465-b67a-9b18717b00f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:41.169575 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa2dd0a-19a4-4465-b67a-9b18717b00f0 None admin] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:41.170312 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa2dd0a-19a4-4465-b67a-9b18717b00f0 None admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:41.171030 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa2dd0a-19a4-4465-b67a-9b18717b00f0 None admin] PATH_INFO: `/v3/users` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:41.172678 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fa2dd0a-19a4-4465-b67a-9b18717b00f0 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=88114) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:34:41.173461 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fa2dd0a-19a4-4465-b67a-9b18717b00f0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:41.176346 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fa2dd0a-19a4-4465-b67a-9b18717b00f0 None admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:41.217552 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 347/698] 10.222.0.13 () {56 vars in 1135 bytes} [Wed Aug 31 09:34:40 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 361 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:34:41.219769 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fa2dd0a-19a4-4465-b67a-9b18717b00f0 None admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:41.272605 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0713fde6-d946-45f8-9333-1796eb7a5524 None admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:41.316059 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da35352c-c945-4a3c-958f-3814d1ea723d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:41.316747 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da35352c-c945-4a3c-958f-3814d1ea723d None admin] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:41.317031 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da35352c-c945-4a3c-958f-3814d1ea723d None admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:41.317305 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da35352c-c945-4a3c-958f-3814d1ea723d None admin] PATH_INFO: `/v3/domains` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:41.318024 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da35352c-c945-4a3c-958f-3814d1ea723d None admin] RBAC: Adding query filter params (name=Default) {{(pid=88113) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:34:41.318330 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da35352c-c945-4a3c-958f-3814d1ea723d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:41.319794 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da35352c-c945-4a3c-958f-3814d1ea723d None admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:41.339121 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 352/699] 10.222.0.13 () {56 vars in 1109 bytes} [Wed Aug 31 09:34:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:34:41.340131 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da35352c-c945-4a3c-958f-3814d1ea723d None admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:41.361809 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fa2dd0a-19a4-4465-b67a-9b18717b00f0 None admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:41.370480 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=100 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:41.410711 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a141bf-52d1-489f-a35e-08197f1acd25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:41.411349 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a141bf-52d1-489f-a35e-08197f1acd25 None admin] REQUEST_METHOD: `POST` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:41.411638 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a141bf-52d1-489f-a35e-08197f1acd25 None admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:41.411915 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a141bf-52d1-489f-a35e-08197f1acd25 None admin] PATH_INFO: `/v3/users` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:41.412647 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9a141bf-52d1-489f-a35e-08197f1acd25 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:41.414163 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9a141bf-52d1-489f-a35e-08197f1acd25 None admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:41.427378 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:41.457428 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 348/700] 10.222.0.13 () {60 vars in 1132 bytes} [Wed Aug 31 09:34:41 2022] POST /identity/v3/users => generated 254 bytes in 230 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 09:34:41.457428 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a141bf-52d1-489f-a35e-08197f1acd25 None admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:41.595539 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da35352c-c945-4a3c-958f-3814d1ea723d None admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:41.687852 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a141bf-52d1-489f-a35e-08197f1acd25 None admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:41.695791 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae75abad-73f0-4e01-94e1-ec52a09f4d42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:41.696724 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae75abad-73f0-4e01-94e1-ec52a09f4d42 None admin] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:41.697154 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae75abad-73f0-4e01-94e1-ec52a09f4d42 None admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:41.697605 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae75abad-73f0-4e01-94e1-ec52a09f4d42 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:41.773672 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21294ba0-8e0d-48dc-8877-4ce06dff07bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:41.775151 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21294ba0-8e0d-48dc-8877-4ce06dff07bd None admin] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:41.775866 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21294ba0-8e0d-48dc-8877-4ce06dff07bd None admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:41.776609 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21294ba0-8e0d-48dc-8877-4ce06dff07bd None admin] PATH_INFO: `/v3/projects/service` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:41.794193 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21294ba0-8e0d-48dc-8877-4ce06dff07bd None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:41.797752 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21294ba0-8e0d-48dc-8877-4ce06dff07bd None admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:41.822224 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application [None req-21294ba0-8e0d-48dc-8877-4ce06dff07bd None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 09:34:41.822224 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:34:41.822224 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:34:41.822224 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:34:41.822224 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:34:41.822224 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:34:41.822224 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:34:41.822224 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:34:41.822224 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:34:41.822224 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:34:41.822224 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:34:41.822224 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:34:41.822224 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 31 09:34:41.822224 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 31 09:34:41.822224 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 31 09:34:41.824005 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 31 09:34:41.824005 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:34:41.824005 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:34:41.824005 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:34:41.824005 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:34:41.824005 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:34:41.824005 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:34:41.824005 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:34:41.824005 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application with Lock( Aug 31 09:34:41.824005 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:34:41.824005 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:34:41.824005 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:34:41.824005 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:34:41.824005 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:34:41.824005 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:34:41.824005 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:34:41.824005 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:34:41.825973 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 31 09:34:41.825973 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 31 09:34:41.825973 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 31 09:34:41.825973 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 31 09:34:41.825973 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 31 09:34:41.825973 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 31 09:34:41.825973 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 09:34:41.825973 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application Aug 31 09:34:41.836882 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 349/701] 10.222.0.13 () {56 vars in 1111 bytes} [Wed Aug 31 09:34:41 2022] GET /identity/v3/projects/service => generated 88 bytes in 382 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:34:41.851639 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21294ba0-8e0d-48dc-8877-4ce06dff07bd None admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:41.855634 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae75abad-73f0-4e01-94e1-ec52a09f4d42 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:41.860543 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae75abad-73f0-4e01-94e1-ec52a09f4d42 None admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:41.907009 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=100 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:41.956379 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:42.050648 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 353/702] 10.222.0.13 () {58 vars in 1282 bytes} [Wed Aug 31 09:34:41 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 711 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 31 09:34:42.062477 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae75abad-73f0-4e01-94e1-ec52a09f4d42 None admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:42.081577 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21294ba0-8e0d-48dc-8877-4ce06dff07bd None admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:42.163866 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c62b0913-b9eb-4937-93f2-d6e0b5fb5daa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:42.164740 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62b0913-b9eb-4937-93f2-d6e0b5fb5daa None admin] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:42.165180 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62b0913-b9eb-4937-93f2-d6e0b5fb5daa None admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:42.165666 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62b0913-b9eb-4937-93f2-d6e0b5fb5daa None admin] PATH_INFO: `/v3/projects` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:42.166515 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c62b0913-b9eb-4937-93f2-d6e0b5fb5daa None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88114) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:34:42.167723 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c62b0913-b9eb-4937-93f2-d6e0b5fb5daa None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:42.170721 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c62b0913-b9eb-4937-93f2-d6e0b5fb5daa None admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:42.197349 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 350/703] 10.222.0.13 () {56 vars in 1148 bytes} [Wed Aug 31 09:34:41 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 347 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:34:42.206667 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c62b0913-b9eb-4937-93f2-d6e0b5fb5daa None admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:42.257306 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae75abad-73f0-4e01-94e1-ec52a09f4d42 None admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:42.309800 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=100 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:42.363871 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b942f0f4-930b-4ba1-bc8a-0b1b06ee581f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:42.364980 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b942f0f4-930b-4ba1-bc8a-0b1b06ee581f None admin] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:42.365478 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b942f0f4-930b-4ba1-bc8a-0b1b06ee581f None admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:42.365955 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b942f0f4-930b-4ba1-bc8a-0b1b06ee581f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:42.414928 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b942f0f4-930b-4ba1-bc8a-0b1b06ee581f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:42.417669 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b942f0f4-930b-4ba1-bc8a-0b1b06ee581f None admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:42.459278 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application [None req-b942f0f4-930b-4ba1-bc8a-0b1b06ee581f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 09:34:42.459278 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:34:42.459278 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:34:42.459278 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:34:42.459278 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:34:42.459278 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:34:42.459278 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:34:42.459278 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:34:42.459278 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:34:42.459278 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:34:42.459278 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:34:42.459278 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:34:42.459278 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 09:34:42.459278 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 09:34:42.459278 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 09:34:42.459278 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 09:34:42.460290 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:34:42.460290 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:34:42.460290 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:34:42.460290 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:34:42.460290 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:34:42.460290 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:34:42.460290 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:34:42.460290 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application with Lock( Aug 31 09:34:42.460290 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:34:42.460290 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:34:42.460290 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:34:42.460290 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:34:42.460290 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:34:42.460290 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:34:42.460290 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:34:42.460290 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:34:42.460290 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 09:34:42.461171 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 09:34:42.461171 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 09:34:42.461171 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application Aug 31 09:34:42.470724 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 354/704] 10.222.0.13 () {56 vars in 1108 bytes} [Wed Aug 31 09:34:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 409 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:34:42.476782 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b942f0f4-930b-4ba1-bc8a-0b1b06ee581f None admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:42.528247 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c62b0913-b9eb-4937-93f2-d6e0b5fb5daa None admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:42.570671 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=100 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:42.632479 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c7847b5-df04-4b44-9418-79f3c03e77aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:42.633109 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7847b5-df04-4b44-9418-79f3c03e77aa None admin] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:42.633416 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7847b5-df04-4b44-9418-79f3c03e77aa None admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:42.633686 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7847b5-df04-4b44-9418-79f3c03e77aa None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:42.634453 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c7847b5-df04-4b44-9418-79f3c03e77aa None admin] RBAC: Adding query filter params (role.id=5d4ee63a3add47fc9a6b8b16d39e0a1f, scope.project.id=49548eb44d62415582af820ab66decee, user.id=dc076493239d4ccdbdc37c0369c49529) {{(pid=88114) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:34:42.634945 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c7847b5-df04-4b44-9418-79f3c03e77aa None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:42.636452 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c7847b5-df04-4b44-9418-79f3c03e77aa None admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:42.652491 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 351/705] 10.222.0.13 () {56 vars in 1374 bytes} [Wed Aug 31 09:34:42 2022] GET /identity/v3/role_assignments?user.id=dc076493239d4ccdbdc37c0369c49529&scope.project.id=49548eb44d62415582af820ab66decee&role.id=5d4ee63a3add47fc9a6b8b16d39e0a1f => generated 259 bytes in 445 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:34:42.699346 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b942f0f4-930b-4ba1-bc8a-0b1b06ee581f None admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:42.753326 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7eb908a-1e4b-41af-9c16-f6e91bafdde5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:42.754031 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7eb908a-1e4b-41af-9c16-f6e91bafdde5 None admin] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:42.754314 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7eb908a-1e4b-41af-9c16-f6e91bafdde5 None admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:42.754588 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7eb908a-1e4b-41af-9c16-f6e91bafdde5 None admin] PATH_INFO: `/v3/domains` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:42.755255 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7eb908a-1e4b-41af-9c16-f6e91bafdde5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88113) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:34:42.755553 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7eb908a-1e4b-41af-9c16-f6e91bafdde5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:42.756971 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7eb908a-1e4b-41af-9c16-f6e91bafdde5 None admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:42.776516 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 355/706] 10.222.0.13 () {56 vars in 1109 bytes} [Wed Aug 31 09:34:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 300 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 09:34:42.782357 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c7847b5-df04-4b44-9418-79f3c03e77aa None admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:42.945899 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c7847b5-df04-4b44-9418-79f3c03e77aa None admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:42.995631 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b94ed84-41c3-4868-9c24-196c472a0f2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:42.996281 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b94ed84-41c3-4868-9c24-196c472a0f2c None admin] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:42.996554 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b94ed84-41c3-4868-9c24-196c472a0f2c None admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:42.996819 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b94ed84-41c3-4868-9c24-196c472a0f2c None admin] PATH_INFO: `/v3/projects/service` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:43.009178 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b94ed84-41c3-4868-9c24-196c472a0f2c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:43.015561 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b94ed84-41c3-4868-9c24-196c472a0f2c None admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:43.028368 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application [None req-4b94ed84-41c3-4868-9c24-196c472a0f2c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 09:34:43.028368 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:34:43.028368 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:34:43.028368 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:34:43.028368 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:34:43.028368 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:34:43.028368 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:34:43.028368 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:34:43.028368 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:34:43.028368 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:34:43.028368 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:34:43.028368 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:34:43.028368 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 31 09:34:43.028368 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 31 09:34:43.028368 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 31 09:34:43.030470 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 31 09:34:43.030470 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:34:43.030470 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:34:43.030470 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:34:43.030470 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:34:43.030470 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:34:43.030470 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:34:43.030470 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:34:43.030470 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application with Lock( Aug 31 09:34:43.030470 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:34:43.030470 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:34:43.030470 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:34:43.030470 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:34:43.030470 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:34:43.030470 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:34:43.030470 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:34:43.030470 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:34:43.033835 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 31 09:34:43.033835 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 31 09:34:43.033835 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 31 09:34:43.033835 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 31 09:34:43.033835 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 31 09:34:43.033835 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 31 09:34:43.033835 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 09:34:43.033835 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application Aug 31 09:34:43.033835 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 352/707] 10.222.0.13 () {56 vars in 1111 bytes} [Wed Aug 31 09:34:42 2022] GET /identity/v3/projects/service => generated 88 bytes in 252 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 09:34:43.038340 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7eb908a-1e4b-41af-9c16-f6e91bafdde5 None admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:43.120670 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=100 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:43.197069 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c89b512-dd30-43cc-b60f-3a0fb65a869f None None] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:43.197453 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c89b512-dd30-43cc-b60f-3a0fb65a869f None None] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:43.197626 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c89b512-dd30-43cc-b60f-3a0fb65a869f None None] PATH_INFO: `/` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:43.199112 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 353/708] 10.222.0.13 () {54 vars in 927 bytes} [Wed Aug 31 09:34:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 31 09:34:43.204865 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d153c0-3bca-46b0-83b3-6868c81fcf2d None None] REQUEST_METHOD: `POST` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:43.205403 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d153c0-3bca-46b0-83b3-6868c81fcf2d None None] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:43.205679 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d153c0-3bca-46b0-83b3-6868c81fcf2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:43.232458 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.auth.core [None req-d2d153c0-3bca-46b0-83b3-6868c81fcf2d None None] MFA Rules not processed for user `fd836cf590a145df9acf19cf3ae2ac56`. Rule list: `[]` (Enabled: `True`). {{(pid=88114) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:34:43.234429 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.fernet_utils [None req-d2d153c0-3bca-46b0-83b3-6868c81fcf2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88114) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:34:43.271673 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7eb908a-1e4b-41af-9c16-f6e91bafdde5 None admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:43.290026 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=100 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:43.313604 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 354/709] 10.222.0.13 () {58 vars in 1024 bytes} [Wed Aug 31 09:34:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 09:34:43.315219 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbbf440a-fb4d-4806-8ce0-9d01885fa501 None None] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:43.315563 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbbf440a-fb4d-4806-8ce0-9d01885fa501 None None] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:43.315903 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbbf440a-fb4d-4806-8ce0-9d01885fa501 None None] PATH_INFO: `/` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:43.321752 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 355/710] 10.222.0.13 () {54 vars in 927 bytes} [Wed Aug 31 09:34:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:34:43.323114 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ecb28c-adae-4e52-9bf8-ce52c8bb8ff1 None None] REQUEST_METHOD: `POST` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:43.323470 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ecb28c-adae-4e52-9bf8-ce52c8bb8ff1 None None] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:43.323899 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ecb28c-adae-4e52-9bf8-ce52c8bb8ff1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:43.343988 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.auth.core [None req-c1ecb28c-adae-4e52-9bf8-ce52c8bb8ff1 None None] MFA Rules not processed for user `fd836cf590a145df9acf19cf3ae2ac56`. Rule list: `[]` (Enabled: `True`). {{(pid=88114) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:34:43.345315 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.fernet_utils [None req-c1ecb28c-adae-4e52-9bf8-ce52c8bb8ff1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88114) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:34:43.346474 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-018fffec-4f58-48c8-842d-888ba0202427 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:43.347637 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018fffec-4f58-48c8-842d-888ba0202427 None admin] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:43.348110 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018fffec-4f58-48c8-842d-888ba0202427 None admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:43.348611 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018fffec-4f58-48c8-842d-888ba0202427 None admin] PATH_INFO: `/v3/projects` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:43.350023 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-018fffec-4f58-48c8-842d-888ba0202427 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88113) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:34:43.350544 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-018fffec-4f58-48c8-842d-888ba0202427 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:43.353422 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-018fffec-4f58-48c8-842d-888ba0202427 None admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:43.396811 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 356/711] 10.222.0.13 () {56 vars in 1148 bytes} [Wed Aug 31 09:34:43 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 361 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:34:43.404202 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-018fffec-4f58-48c8-842d-888ba0202427 None admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:43.428316 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 356/712] 10.222.0.13 () {58 vars in 1024 bytes} [Wed Aug 31 09:34:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 09:34:43.596429 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=100 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:43.611012 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-018fffec-4f58-48c8-842d-888ba0202427 None admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:43.672131 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d5e55d3-5e14-43f8-943a-64a81ac8cc93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:43.672765 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5e55d3-5e14-43f8-943a-64a81ac8cc93 None admin] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:43.673045 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5e55d3-5e14-43f8-943a-64a81ac8cc93 None admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:43.673324 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5e55d3-5e14-43f8-943a-64a81ac8cc93 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:43.674100 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d5e55d3-5e14-43f8-943a-64a81ac8cc93 None admin] RBAC: Adding query filter params (role.id=e24ba4fd91584883af350cc84bc96c1d, scope.project.id=49548eb44d62415582af820ab66decee, user.id=202bb9db7a8a4977ba788a290ed9a3fa) {{(pid=88113) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:34:43.674409 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d5e55d3-5e14-43f8-943a-64a81ac8cc93 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:43.675917 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d5e55d3-5e14-43f8-943a-64a81ac8cc93 None admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:43.686892 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 357/713] 10.222.0.13 () {56 vars in 1374 bytes} [Wed Aug 31 09:34:43 2022] GET /identity/v3/role_assignments?user.id=202bb9db7a8a4977ba788a290ed9a3fa&scope.project.id=49548eb44d62415582af820ab66decee&role.id=e24ba4fd91584883af350cc84bc96c1d => generated 608 bytes in 284 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 09:34:43.896975 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ecb28c-adae-4e52-9bf8-ce52c8bb8ff1 None None] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:43.945521 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d5e55d3-5e14-43f8-943a-64a81ac8cc93 None admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:44.058008 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=100 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:44.066412 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ecb28c-adae-4e52-9bf8-ce52c8bb8ff1 None None] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:44.111759 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d5e55d3-5e14-43f8-943a-64a81ac8cc93 None admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:44.127401 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf3ef7a-9564-4873-9c44-1480289bedaf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:44.128895 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf3ef7a-9564-4873-9c44-1480289bedaf None admin] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:44.129440 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf3ef7a-9564-4873-9c44-1480289bedaf None admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:44.129912 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf3ef7a-9564-4873-9c44-1480289bedaf None admin] PATH_INFO: `/v3/services/network` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:44.130738 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daf3ef7a-9564-4873-9c44-1480289bedaf None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:44.132498 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daf3ef7a-9564-4873-9c44-1480289bedaf None admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:44.146469 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application [None req-daf3ef7a-9564-4873-9c44-1480289bedaf None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Aug 31 09:34:44.146469 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:34:44.146469 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 31 09:34:44.146469 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 31 09:34:44.146469 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 31 09:34:44.146469 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 31 09:34:44.146469 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 31 09:34:44.146469 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 09:34:44.146469 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 31 09:34:44.146469 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application Aug 31 09:34:44.146469 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 31 09:34:44.146469 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application Aug 31 09:34:44.146469 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:34:44.146469 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:34:44.146469 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:34:44.146469 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:34:44.146469 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:34:44.147684 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:34:44.147684 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:34:44.147684 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:34:44.147684 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:34:44.147684 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:34:44.147684 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:34:44.147684 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 31 09:34:44.147684 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 31 09:34:44.147684 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 31 09:34:44.147684 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 31 09:34:44.147684 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:34:44.147684 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:34:44.147684 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:34:44.147684 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:34:44.147684 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:34:44.147684 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:34:44.148552 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:34:44.148552 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application with Lock( Aug 31 09:34:44.148552 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:34:44.148552 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:34:44.148552 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:34:44.148552 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:34:44.148552 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:34:44.148552 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:34:44.148552 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:34:44.148552 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:34:44.148552 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 31 09:34:44.148552 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 09:34:44.148552 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 31 09:34:44.148552 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application Aug 31 09:34:44.152172 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 357/714] 10.222.0.13 () {56 vars in 1111 bytes} [Wed Aug 31 09:34:43 2022] GET /identity/v3/services/network => generated 88 bytes in 256 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:34:44.157616 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ff447d-1242-4144-852e-627517d93fa4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:44.158277 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ff447d-1242-4144-852e-627517d93fa4 None admin] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:44.158563 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ff447d-1242-4144-852e-627517d93fa4 None admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:44.158829 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ff447d-1242-4144-852e-627517d93fa4 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:44.159434 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49ff447d-1242-4144-852e-627517d93fa4 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:44.160975 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49ff447d-1242-4144-852e-627517d93fa4 None admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:44.161891 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf3ef7a-9564-4873-9c44-1480289bedaf None admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:44.175028 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application [None req-49ff447d-1242-4144-852e-627517d93fa4 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 31 09:34:44.175028 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:34:44.175028 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 31 09:34:44.175028 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 31 09:34:44.175028 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 31 09:34:44.175028 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 31 09:34:44.175028 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 31 09:34:44.175028 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 09:34:44.175028 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 31 09:34:44.175028 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application Aug 31 09:34:44.175028 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 31 09:34:44.175028 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application Aug 31 09:34:44.175028 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:34:44.175028 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:34:44.175028 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:34:44.175028 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:34:44.175028 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:34:44.176098 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:34:44.176098 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:34:44.176098 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:34:44.176098 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:34:44.176098 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:34:44.176098 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:34:44.176098 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 31 09:34:44.176098 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 31 09:34:44.176098 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 31 09:34:44.176098 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 31 09:34:44.176098 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:34:44.176098 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:34:44.176098 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:34:44.176098 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:34:44.176098 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:34:44.176098 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:34:44.176956 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:34:44.176956 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application with Lock( Aug 31 09:34:44.176956 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:34:44.176956 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:34:44.176956 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:34:44.176956 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:34:44.176956 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:34:44.176956 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:34:44.176956 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:34:44.176956 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:34:44.176956 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 31 09:34:44.176956 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 09:34:44.176956 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 31 09:34:44.176956 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application Aug 31 09:34:44.179779 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 358/715] 10.222.0.13 () {56 vars in 1129 bytes} [Wed Aug 31 09:34:43 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:34:44.183653 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ff447d-1242-4144-852e-627517d93fa4 None admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:44.211242 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=100 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:44.317487 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ff447d-1242-4144-852e-627517d93fa4 None admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:44.359805 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7501570-2d46-4f56-9059-5832296b5a42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:44.361001 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7501570-2d46-4f56-9059-5832296b5a42 None admin] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:44.361284 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7501570-2d46-4f56-9059-5832296b5a42 None admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:44.361593 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7501570-2d46-4f56-9059-5832296b5a42 None admin] PATH_INFO: `/v3/services` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:44.362251 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7501570-2d46-4f56-9059-5832296b5a42 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=88113) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:34:44.362559 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7501570-2d46-4f56-9059-5832296b5a42 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:44.364027 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7501570-2d46-4f56-9059-5832296b5a42 None admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:44.373516 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 359/716] 10.222.0.13 () {56 vars in 1124 bytes} [Wed Aug 31 09:34:44 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 09:34:44.376438 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7501570-2d46-4f56-9059-5832296b5a42 None admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:44.436021 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf3ef7a-9564-4873-9c44-1480289bedaf None admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:44.485783 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=100 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:44.513746 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7501570-2d46-4f56-9059-5832296b5a42 None admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:44.542543 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad35ad76-cdab-4193-9239-c4862994bbec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:44.543577 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad35ad76-cdab-4193-9239-c4862994bbec None admin] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:44.544062 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad35ad76-cdab-4193-9239-c4862994bbec None admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:44.544543 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad35ad76-cdab-4193-9239-c4862994bbec None admin] PATH_INFO: `/v3/services` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:44.546274 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad35ad76-cdab-4193-9239-c4862994bbec None admin] RBAC: Adding query filter params (name=network) {{(pid=88114) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:34:44.546809 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad35ad76-cdab-4193-9239-c4862994bbec None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:44.549343 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad35ad76-cdab-4193-9239-c4862994bbec None admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:44.553821 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb3d5e9e-626e-4589-948e-3334fe0af254 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:44.554453 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3d5e9e-626e-4589-948e-3334fe0af254 None admin] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:44.554731 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3d5e9e-626e-4589-948e-3334fe0af254 None admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:44.555001 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3d5e9e-626e-4589-948e-3334fe0af254 None admin] PATH_INFO: `/v3/services` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:44.555642 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb3d5e9e-626e-4589-948e-3334fe0af254 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=88113) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:34:44.555944 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb3d5e9e-626e-4589-948e-3334fe0af254 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:44.557336 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb3d5e9e-626e-4589-948e-3334fe0af254 None admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:44.564926 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 360/717] 10.222.0.13 () {56 vars in 1124 bytes} [Wed Aug 31 09:34:44 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 09:34:44.566818 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 358/718] 10.222.0.13 () {56 vars in 1112 bytes} [Wed Aug 31 09:34:44 2022] GET /identity/v3/services?name=network => generated 124 bytes in 406 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:34:44.578695 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb3d5e9e-626e-4589-948e-3334fe0af254 None admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:44.698196 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb3d5e9e-626e-4589-948e-3334fe0af254 None admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:44.737295 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf7daeff-ee4b-4f6a-9f58-114f8062de7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:44.737974 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7daeff-ee4b-4f6a-9f58-114f8062de7a None admin] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:44.738256 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7daeff-ee4b-4f6a-9f58-114f8062de7a None admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:44.738531 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7daeff-ee4b-4f6a-9f58-114f8062de7a None admin] PATH_INFO: `/v3/services` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:44.739193 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf7daeff-ee4b-4f6a-9f58-114f8062de7a None admin] RBAC: Adding query filter params (type=network) {{(pid=88113) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:34:44.739500 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf7daeff-ee4b-4f6a-9f58-114f8062de7a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:44.740918 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf7daeff-ee4b-4f6a-9f58-114f8062de7a None admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:44.746658 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 361/719] 10.222.0.13 () {56 vars in 1112 bytes} [Wed Aug 31 09:34:44 2022] GET /identity/v3/services?type=network => generated 124 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:34:45.211187 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea1daa1-9e87-4ca4-8d60-6136244eea3e None None] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:45.212229 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea1daa1-9e87-4ca4-8d60-6136244eea3e None None] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:45.212706 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea1daa1-9e87-4ca4-8d60-6136244eea3e None None] PATH_INFO: `/` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:45.214581 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 359/720] 10.222.0.13 () {54 vars in 927 bytes} [Wed Aug 31 09:34:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:34:45.229471 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965339f8-9ca9-438b-9b53-1ed0a68e9c6e None None] REQUEST_METHOD: `POST` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:45.230412 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965339f8-9ca9-438b-9b53-1ed0a68e9c6e None None] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:45.230887 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965339f8-9ca9-438b-9b53-1ed0a68e9c6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:45.255001 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.auth.core [None req-965339f8-9ca9-438b-9b53-1ed0a68e9c6e None None] MFA Rules not processed for user `fd836cf590a145df9acf19cf3ae2ac56`. Rule list: `[]` (Enabled: `True`). {{(pid=88113) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:34:45.256612 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.fernet_utils [None req-965339f8-9ca9-438b-9b53-1ed0a68e9c6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:34:45.262484 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=100 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:45.365706 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 362/721] 10.222.0.13 () {58 vars in 1024 bytes} [Wed Aug 31 09:34:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 09:34:45.936811 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea1daa1-9e87-4ca4-8d60-6136244eea3e None None] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:45.976710 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=100 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:46.072763 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea1daa1-9e87-4ca4-8d60-6136244eea3e None None] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:46.122730 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d086ddfa-5192-4107-88de-b3db53467296 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:46.123380 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d086ddfa-5192-4107-88de-b3db53467296 None admin] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:46.123657 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d086ddfa-5192-4107-88de-b3db53467296 None admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:46.124293 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d086ddfa-5192-4107-88de-b3db53467296 None admin] PATH_INFO: `/v3/roles/service` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:46.143556 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b13c8f2-d924-4e22-965b-5d230a3aa783 None None] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:46.144233 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b13c8f2-d924-4e22-965b-5d230a3aa783 None None] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:46.144403 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b13c8f2-d924-4e22-965b-5d230a3aa783 None None] PATH_INFO: `/` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:46.146044 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 363/722] 10.222.0.13 () {54 vars in 927 bytes} [Wed Aug 31 09:34:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 31 09:34:46.147347 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d086ddfa-5192-4107-88de-b3db53467296 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 09:34:46.151603 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e96682-f649-4ee8-8c13-612c9929ffda None None] REQUEST_METHOD: `POST` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:46.151944 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e96682-f649-4ee8-8c13-612c9929ffda None None] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:46.152222 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e96682-f649-4ee8-8c13-612c9929ffda None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:46.166931 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application [None req-d086ddfa-5192-4107-88de-b3db53467296 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 31 09:34:46.166931 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:34:46.166931 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:34:46.166931 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:34:46.166931 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:34:46.166931 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:34:46.166931 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:34:46.166931 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:34:46.166931 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:34:46.166931 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:34:46.166931 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:34:46.166931 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:34:46.166931 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 09:34:46.166931 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 09:34:46.166931 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 09:34:46.166931 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 09:34:46.168794 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 09:34:46.168794 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 09:34:46.168794 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 09:34:46.168794 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 09:34:46.168794 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:34:46.168794 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:34:46.168794 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:34:46.168794 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:34:46.168794 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:34:46.168794 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:34:46.168794 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:34:46.168794 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application with Lock( Aug 31 09:34:46.168794 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:34:46.168794 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:34:46.168794 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:34:46.168794 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:34:46.170493 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:34:46.170493 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:34:46.170493 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:34:46.170493 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:34:46.170493 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 09:34:46.170493 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 09:34:46.170493 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 09:34:46.170493 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 09:34:46.170493 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 09:34:46.170493 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 09:34:46.170493 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 31 09:34:46.170493 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application Aug 31 09:34:46.175861 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.auth.core [None req-20e96682-f649-4ee8-8c13-612c9929ffda None None] MFA Rules not processed for user `fd836cf590a145df9acf19cf3ae2ac56`. Rule list: `[]` (Enabled: `True`). {{(pid=88113) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:34:46.176236 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 360/723] 10.222.0.13 () {56 vars in 1102 bytes} [Wed Aug 31 09:34:45 2022] GET /identity/v3/roles/service => generated 85 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:34:46.176951 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.fernet_utils [None req-20e96682-f649-4ee8-8c13-612c9929ffda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:34:46.185836 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d086ddfa-5192-4107-88de-b3db53467296 None admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:46.261650 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 364/724] 10.222.0.13 () {58 vars in 1024 bytes} [Wed Aug 31 09:34:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 31 09:34:46.354261 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=100 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:46.373997 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d086ddfa-5192-4107-88de-b3db53467296 None admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:46.427508 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fbcc77d-b294-4057-a0e6-fdf0170e48b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:46.428132 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fbcc77d-b294-4057-a0e6-fdf0170e48b5 None admin] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:46.428411 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fbcc77d-b294-4057-a0e6-fdf0170e48b5 None admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:46.428860 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fbcc77d-b294-4057-a0e6-fdf0170e48b5 None admin] PATH_INFO: `/v3/roles` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:46.429573 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fbcc77d-b294-4057-a0e6-fdf0170e48b5 None admin] RBAC: Adding query filter params (name=service) {{(pid=88114) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:34:46.429879 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fbcc77d-b294-4057-a0e6-fdf0170e48b5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:46.431343 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fbcc77d-b294-4057-a0e6-fdf0170e48b5 None admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:46.445889 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 361/725] 10.222.0.13 () {56 vars in 1103 bytes} [Wed Aug 31 09:34:46 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:34:46.451764 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e96682-f649-4ee8-8c13-612c9929ffda None None] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:46.542973 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=100 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:46.585294 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e96682-f649-4ee8-8c13-612c9929ffda None None] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:46.634675 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8313da75-ee77-4c8e-9d86-e9a890384885 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:46.635791 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8313da75-ee77-4c8e-9d86-e9a890384885 None admin] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:46.636286 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8313da75-ee77-4c8e-9d86-e9a890384885 None admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:46.636761 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8313da75-ee77-4c8e-9d86-e9a890384885 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:46.641887 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fbcc77d-b294-4057-a0e6-fdf0170e48b5 None admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:46.661004 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8313da75-ee77-4c8e-9d86-e9a890384885 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:46.661218 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8313da75-ee77-4c8e-9d86-e9a890384885 None admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:46.695180 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application [None req-8313da75-ee77-4c8e-9d86-e9a890384885 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 09:34:46.695180 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:34:46.695180 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:34:46.695180 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:34:46.695180 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:34:46.695180 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:34:46.695180 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:34:46.695180 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:34:46.695180 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:34:46.695180 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:34:46.695180 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:34:46.695180 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:34:46.695180 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 09:34:46.695180 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 09:34:46.695180 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 09:34:46.695180 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 09:34:46.696228 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:34:46.696228 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:34:46.696228 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:34:46.696228 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:34:46.696228 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:34:46.696228 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:34:46.696228 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:34:46.696228 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application with Lock( Aug 31 09:34:46.696228 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:34:46.696228 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:34:46.696228 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:34:46.696228 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:34:46.696228 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:34:46.696228 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:34:46.696228 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:34:46.696228 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:34:46.696228 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 09:34:46.697130 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 09:34:46.697130 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 09:34:46.697130 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application Aug 31 09:34:46.697767 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 365/726] 10.222.0.13 () {56 vars in 1108 bytes} [Wed Aug 31 09:34:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 246 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:34:46.704708 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8313da75-ee77-4c8e-9d86-e9a890384885 None admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:46.870768 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fbcc77d-b294-4057-a0e6-fdf0170e48b5 None admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:46.870987 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8313da75-ee77-4c8e-9d86-e9a890384885 None admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:46.898055 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=100 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:46.911859 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a93acb7a-974b-4263-b467-b792c4975935 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:46.912968 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93acb7a-974b-4263-b467-b792c4975935 None admin] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:46.913433 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7644b32a-df30-4c03-9172-fb56757ca3d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:46.913714 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93acb7a-974b-4263-b467-b792c4975935 None admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:46.914054 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7644b32a-df30-4c03-9172-fb56757ca3d6 None admin] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:46.914337 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7644b32a-df30-4c03-9172-fb56757ca3d6 None admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:46.914568 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93acb7a-974b-4263-b467-b792c4975935 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:46.914667 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7644b32a-df30-4c03-9172-fb56757ca3d6 None admin] PATH_INFO: `/v3/domains` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:46.915268 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7644b32a-df30-4c03-9172-fb56757ca3d6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88113) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:34:46.915687 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7644b32a-df30-4c03-9172-fb56757ca3d6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:46.917212 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7644b32a-df30-4c03-9172-fb56757ca3d6 None admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:46.927682 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a93acb7a-974b-4263-b467-b792c4975935 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 09:34:46.933833 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application [None req-a93acb7a-974b-4263-b467-b792c4975935 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 09:34:46.933833 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:34:46.933833 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:34:46.933833 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:34:46.933833 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:34:46.933833 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:34:46.933833 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:34:46.933833 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:34:46.933833 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:34:46.933833 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:34:46.933833 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:34:46.933833 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:34:46.933833 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 09:34:46.933833 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 09:34:46.933833 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 09:34:46.933833 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 09:34:46.934851 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 09:34:46.934851 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 09:34:46.934851 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 09:34:46.934851 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 09:34:46.934851 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:34:46.934851 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:34:46.934851 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:34:46.934851 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:34:46.934851 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:34:46.934851 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:34:46.934851 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:34:46.934851 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application with Lock( Aug 31 09:34:46.934851 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:34:46.934851 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:34:46.934851 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:34:46.934851 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:34:46.935692 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:34:46.935692 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:34:46.935692 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:34:46.935692 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:34:46.935692 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 09:34:46.935692 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 09:34:46.935692 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 09:34:46.935692 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 09:34:46.935692 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 09:34:46.935692 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 09:34:46.935692 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 09:34:46.935692 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application Aug 31 09:34:46.936359 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 366/727] 10.222.0.13 () {56 vars in 1109 bytes} [Wed Aug 31 09:34:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:34:46.937700 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206b572a-96a6-4ca1-af90-32550e2e4fad None None] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:46.937992 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206b572a-96a6-4ca1-af90-32550e2e4fad None None] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:46.938268 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206b572a-96a6-4ca1-af90-32550e2e4fad None None] PATH_INFO: `/` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:46.939347 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 362/728] 10.222.0.13 () {56 vars in 1096 bytes} [Wed Aug 31 09:34:46 2022] GET /identity/v3/roles/admin => generated 83 bytes in 298 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:34:46.942907 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a93acb7a-974b-4263-b467-b792c4975935 None admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:46.944744 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 367/729] 10.222.0.13 () {54 vars in 927 bytes} [Wed Aug 31 09:34:46 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:34:46.946245 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-206b572a-96a6-4ca1-af90-32550e2e4fad None None] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:46.957584 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=100 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:47.064445 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a93acb7a-974b-4263-b467-b792c4975935 None admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:47.071782 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-206b572a-96a6-4ca1-af90-32550e2e4fad None None] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:47.104427 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4857b257-91a6-43f1-b804-1e17980e05f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:47.105166 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4857b257-91a6-43f1-b804-1e17980e05f7 None admin] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:47.105688 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4857b257-91a6-43f1-b804-1e17980e05f7 None admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:47.106158 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4857b257-91a6-43f1-b804-1e17980e05f7 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:47.114416 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-389d6f8c-5ffb-4c65-bbbb-ecb5dc07371a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:47.115299 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389d6f8c-5ffb-4c65-bbbb-ecb5dc07371a None admin] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:47.115734 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389d6f8c-5ffb-4c65-bbbb-ecb5dc07371a None admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:47.116159 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389d6f8c-5ffb-4c65-bbbb-ecb5dc07371a None admin] PATH_INFO: `/v3/roles` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:47.116899 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4857b257-91a6-43f1-b804-1e17980e05f7 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:47.117215 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-389d6f8c-5ffb-4c65-bbbb-ecb5dc07371a None admin] RBAC: Adding query filter params (name=admin) {{(pid=88113) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:34:47.117700 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-389d6f8c-5ffb-4c65-bbbb-ecb5dc07371a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:47.118655 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4857b257-91a6-43f1-b804-1e17980e05f7 None admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:47.119473 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-389d6f8c-5ffb-4c65-bbbb-ecb5dc07371a None admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:47.132760 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application [None req-4857b257-91a6-43f1-b804-1e17980e05f7 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 31 09:34:47.132760 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:34:47.132760 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:34:47.132760 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:34:47.132760 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:34:47.132760 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:34:47.132760 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:34:47.132760 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:34:47.132760 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:34:47.132760 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:34:47.132760 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:34:47.132760 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:34:47.132760 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 31 09:34:47.132760 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 31 09:34:47.132760 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 31 09:34:47.132760 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 31 09:34:47.133776 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:34:47.133776 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:34:47.133776 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 31 09:34:47.133776 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 09:34:47.133776 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 31 09:34:47.133776 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 09:34:47.133776 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:34:47.133776 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:34:47.133776 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:34:47.133776 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:34:47.133776 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:34:47.133776 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application with Lock( Aug 31 09:34:47.133776 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:34:47.133776 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:34:47.133776 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:34:47.133776 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:34:47.133776 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:34:47.133776 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:34:47.134723 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:34:47.134723 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:34:47.134723 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 31 09:34:47.134723 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 31 09:34:47.134723 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 31 09:34:47.134723 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 31 09:34:47.134723 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 31 09:34:47.134723 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 31 09:34:47.134723 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 31 09:34:47.134723 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application Aug 31 09:34:47.135258 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 368/730] 10.222.0.13 () {56 vars in 1099 bytes} [Wed Aug 31 09:34:46 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:34:47.136418 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae79d69b-4e04-4cb7-8fd3-9179b4fe37a2 None None] REQUEST_METHOD: `POST` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:47.136715 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae79d69b-4e04-4cb7-8fd3-9179b4fe37a2 None None] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:47.136998 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae79d69b-4e04-4cb7-8fd3-9179b4fe37a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:47.143295 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 363/731] 10.222.0.13 () {56 vars in 1117 bytes} [Wed Aug 31 09:34:46 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:34:47.143295 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4857b257-91a6-43f1-b804-1e17980e05f7 None admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:47.167482 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.auth.core [None req-ae79d69b-4e04-4cb7-8fd3-9179b4fe37a2 None None] MFA Rules not processed for user `fd836cf590a145df9acf19cf3ae2ac56`. Rule list: `[]` (Enabled: `True`). {{(pid=88113) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:34:47.169911 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.fernet_utils [None req-ae79d69b-4e04-4cb7-8fd3-9179b4fe37a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:34:47.239632 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=100 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:47.261085 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=100 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:47.291141 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 369/732] 10.222.0.13 () {58 vars in 1024 bytes} [Wed Aug 31 09:34:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 09:34:47.293312 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae79d69b-4e04-4cb7-8fd3-9179b4fe37a2 None None] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:47.362057 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4857b257-91a6-43f1-b804-1e17980e05f7 None admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:47.444189 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b00c9cc-8a5a-432b-b46f-5ca862310918 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:47.444852 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b00c9cc-8a5a-432b-b46f-5ca862310918 None admin] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:47.445132 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b00c9cc-8a5a-432b-b46f-5ca862310918 None admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:47.445419 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b00c9cc-8a5a-432b-b46f-5ca862310918 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:47.466591 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b00c9cc-8a5a-432b-b46f-5ca862310918 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:47.470361 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b00c9cc-8a5a-432b-b46f-5ca862310918 None admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:47.487529 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application [None req-3b00c9cc-8a5a-432b-b46f-5ca862310918 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 09:34:47.487529 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:34:47.487529 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:34:47.487529 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:34:47.487529 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:34:47.487529 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:34:47.487529 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:34:47.487529 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:34:47.487529 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:34:47.487529 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:34:47.487529 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:34:47.487529 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:34:47.487529 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 09:34:47.487529 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 09:34:47.487529 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 09:34:47.487529 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 09:34:47.488594 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:34:47.488594 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:34:47.488594 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:34:47.488594 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:34:47.488594 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:34:47.488594 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:34:47.488594 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:34:47.488594 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application with Lock( Aug 31 09:34:47.488594 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:34:47.488594 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:34:47.488594 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:34:47.488594 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:34:47.488594 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:34:47.488594 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:34:47.488594 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:34:47.488594 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:34:47.488594 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 09:34:47.489590 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 09:34:47.489590 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 09:34:47.489590 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application Aug 31 09:34:47.491878 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 364/733] 10.222.0.13 () {56 vars in 1108 bytes} [Wed Aug 31 09:34:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 350 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:34:47.497442 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b00c9cc-8a5a-432b-b46f-5ca862310918 None admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:47.522683 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae79d69b-4e04-4cb7-8fd3-9179b4fe37a2 None None] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:47.606069 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3057a3db-d526-46ab-8103-c0eae0b5f722 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:47.607155 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3057a3db-d526-46ab-8103-c0eae0b5f722 None admin] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:47.607739 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3057a3db-d526-46ab-8103-c0eae0b5f722 None admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:47.608290 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3057a3db-d526-46ab-8103-c0eae0b5f722 None admin] PATH_INFO: `/v3/users` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:47.609833 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3057a3db-d526-46ab-8103-c0eae0b5f722 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=88113) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:34:47.610478 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3057a3db-d526-46ab-8103-c0eae0b5f722 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:47.612189 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3057a3db-d526-46ab-8103-c0eae0b5f722 None admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:47.638389 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 370/734] 10.222.0.13 () {56 vars in 1149 bytes} [Wed Aug 31 09:34:47 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 346 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:34:47.645814 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3057a3db-d526-46ab-8103-c0eae0b5f722 None admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:47.684835 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=100 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:47.709556 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b00c9cc-8a5a-432b-b46f-5ca862310918 None admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:47.734679 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=100 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:47.753091 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f6068b7-2069-4840-8edb-a31f42a0e011 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:47.753847 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6068b7-2069-4840-8edb-a31f42a0e011 None admin] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:47.754132 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6068b7-2069-4840-8edb-a31f42a0e011 None admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:47.754406 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6068b7-2069-4840-8edb-a31f42a0e011 None admin] PATH_INFO: `/v3/domains` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:47.755152 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f6068b7-2069-4840-8edb-a31f42a0e011 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88114) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:34:47.755462 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f6068b7-2069-4840-8edb-a31f42a0e011 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:47.756901 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f6068b7-2069-4840-8edb-a31f42a0e011 None admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:47.776681 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 365/735] 10.222.0.13 () {56 vars in 1109 bytes} [Wed Aug 31 09:34:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 275 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:34:47.777728 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f6068b7-2069-4840-8edb-a31f42a0e011 None admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:47.793822 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3057a3db-d526-46ab-8103-c0eae0b5f722 None admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:47.851775 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b23cf4-ef94-4981-836b-204bb57ca7e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:47.852888 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b23cf4-ef94-4981-836b-204bb57ca7e1 None admin] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:47.853352 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b23cf4-ef94-4981-836b-204bb57ca7e1 None admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:47.853834 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b23cf4-ef94-4981-836b-204bb57ca7e1 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:47.875098 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46b23cf4-ef94-4981-836b-204bb57ca7e1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 09:34:47.884804 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application [None req-46b23cf4-ef94-4981-836b-204bb57ca7e1 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 09:34:47.884804 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:34:47.884804 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:34:47.884804 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:34:47.884804 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:34:47.884804 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:34:47.884804 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:34:47.884804 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:34:47.884804 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:34:47.884804 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:34:47.884804 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:34:47.884804 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:34:47.884804 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 09:34:47.884804 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 09:34:47.884804 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 09:34:47.884804 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 09:34:47.886071 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 09:34:47.886071 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 09:34:47.886071 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 09:34:47.886071 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 09:34:47.886071 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:34:47.886071 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:34:47.886071 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:34:47.886071 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:34:47.886071 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:34:47.886071 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:34:47.886071 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:34:47.886071 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application with Lock( Aug 31 09:34:47.886071 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:34:47.886071 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:34:47.886071 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:34:47.886071 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:34:47.887017 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:34:47.887017 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:34:47.887017 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:34:47.887017 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:34:47.887017 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 09:34:47.887017 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 09:34:47.887017 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 09:34:47.887017 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 09:34:47.887017 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 09:34:47.887017 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 09:34:47.887017 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 09:34:47.887017 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application Aug 31 09:34:47.887665 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 371/736] 10.222.0.13 () {56 vars in 1096 bytes} [Wed Aug 31 09:34:47 2022] GET /identity/v3/roles/admin => generated 83 bytes in 248 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:34:47.888796 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b23cf4-ef94-4981-836b-204bb57ca7e1 None admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:47.930633 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f6068b7-2069-4840-8edb-a31f42a0e011 None admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:47.997214 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a1bc853-a28f-4c2d-a5f2-9799b1d54f43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:47.997885 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1bc853-a28f-4c2d-a5f2-9799b1d54f43 None admin] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:47.998164 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1bc853-a28f-4c2d-a5f2-9799b1d54f43 None admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:47.998434 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1bc853-a28f-4c2d-a5f2-9799b1d54f43 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:48.057737 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=100 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:48.067153 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=100 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:48.082782 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b23cf4-ef94-4981-836b-204bb57ca7e1 None admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:48.089365 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a1bc853-a28f-4c2d-a5f2-9799b1d54f43 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:48.090986 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a1bc853-a28f-4c2d-a5f2-9799b1d54f43 None admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:48.128988 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d5c3acd-f6fe-4043-8010-05b27dca490c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:48.130019 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5c3acd-f6fe-4043-8010-05b27dca490c None admin] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:48.130470 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5c3acd-f6fe-4043-8010-05b27dca490c None admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:48.130922 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5c3acd-f6fe-4043-8010-05b27dca490c None admin] PATH_INFO: `/v3/users/nova` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:48.151183 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d5c3acd-f6fe-4043-8010-05b27dca490c None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:48.153816 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d5c3acd-f6fe-4043-8010-05b27dca490c None admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:48.167774 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application [None req-8d5c3acd-f6fe-4043-8010-05b27dca490c None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 31 09:34:48.167774 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:34:48.167774 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:34:48.167774 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:34:48.167774 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:34:48.167774 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:34:48.167774 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:34:48.167774 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:34:48.167774 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:34:48.167774 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:34:48.167774 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:34:48.167774 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:34:48.167774 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 31 09:34:48.167774 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 31 09:34:48.167774 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 31 09:34:48.167774 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 31 09:34:48.168849 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:34:48.168849 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:34:48.168849 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 31 09:34:48.168849 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 09:34:48.168849 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 31 09:34:48.168849 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 09:34:48.168849 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:34:48.168849 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:34:48.168849 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:34:48.168849 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:34:48.168849 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:34:48.168849 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application with Lock( Aug 31 09:34:48.168849 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:34:48.168849 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:34:48.168849 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:34:48.168849 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:34:48.168849 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:34:48.168849 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:34:48.169877 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:34:48.169877 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:34:48.169877 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 31 09:34:48.169877 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 31 09:34:48.169877 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 31 09:34:48.169877 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 31 09:34:48.169877 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 31 09:34:48.169877 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 31 09:34:48.169877 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 31 09:34:48.169877 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application Aug 31 09:34:48.169877 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 372/737] 10.222.0.13 () {56 vars in 1093 bytes} [Wed Aug 31 09:34:47 2022] GET /identity/v3/users/nova => generated 82 bytes in 281 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:34:48.171127 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed55de6-6845-40b2-8aea-a8f02652be54 None None] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:48.171127 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed55de6-6845-40b2-8aea-a8f02652be54 None None] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:48.171127 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed55de6-6845-40b2-8aea-a8f02652be54 None None] PATH_INFO: `/` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:48.172043 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 373/738] 10.222.0.13 () {54 vars in 927 bytes} [Wed Aug 31 09:34:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:34:48.172778 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed55de6-6845-40b2-8aea-a8f02652be54 None None] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:48.236707 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 366/739] 10.222.0.13 () {58 vars in 1282 bytes} [Wed Aug 31 09:34:47 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 459 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 31 09:34:48.238546 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c3272b-4f9d-46c4-8c45-a4592c9e46a8 None None] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:48.238961 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c3272b-4f9d-46c4-8c45-a4592c9e46a8 None None] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:48.239353 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c3272b-4f9d-46c4-8c45-a4592c9e46a8 None None] PATH_INFO: `/` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:48.240794 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 367/740] 10.222.0.13 () {54 vars in 927 bytes} [Wed Aug 31 09:34:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:34:48.241563 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c3272b-4f9d-46c4-8c45-a4592c9e46a8 None None] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:48.320108 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed55de6-6845-40b2-8aea-a8f02652be54 None None] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:48.339398 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=100 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:48.358810 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ce3a40-f849-43b8-951c-a9c61e3fc33d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:48.359428 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ce3a40-f849-43b8-951c-a9c61e3fc33d None admin] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:48.359709 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ce3a40-f849-43b8-951c-a9c61e3fc33d None admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:48.359969 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ce3a40-f849-43b8-951c-a9c61e3fc33d None admin] PATH_INFO: `/v3/roles` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:48.360642 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8ce3a40-f849-43b8-951c-a9c61e3fc33d None admin] RBAC: Adding query filter params (name=admin) {{(pid=88113) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:34:48.360941 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8ce3a40-f849-43b8-951c-a9c61e3fc33d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:48.362381 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8ce3a40-f849-43b8-951c-a9c61e3fc33d None admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:48.372594 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 374/741] 10.222.0.13 () {56 vars in 1099 bytes} [Wed Aug 31 09:34:48 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:34:48.374787 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bdd6b06-c1af-4d81-8ed3-e3e3b28510d9 None None] REQUEST_METHOD: `POST` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:48.375261 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bdd6b06-c1af-4d81-8ed3-e3e3b28510d9 None None] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:48.375726 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bdd6b06-c1af-4d81-8ed3-e3e3b28510d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:48.378819 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c3272b-4f9d-46c4-8c45-a4592c9e46a8 None None] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:48.397749 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.auth.core [None req-0bdd6b06-c1af-4d81-8ed3-e3e3b28510d9 None None] MFA Rules not processed for user `fd836cf590a145df9acf19cf3ae2ac56`. Rule list: `[]` (Enabled: `True`). {{(pid=88113) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:34:48.398968 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.fernet_utils [None req-0bdd6b06-c1af-4d81-8ed3-e3e3b28510d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:34:48.411201 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=100 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:48.426488 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a635c55-ee69-43ff-b75b-c1853169c0a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:48.427452 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a635c55-ee69-43ff-b75b-c1853169c0a9 None admin] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:48.427909 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a635c55-ee69-43ff-b75b-c1853169c0a9 None admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:48.428347 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a635c55-ee69-43ff-b75b-c1853169c0a9 None admin] PATH_INFO: `/v3/users` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:48.429647 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a635c55-ee69-43ff-b75b-c1853169c0a9 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=88114) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:34:48.430071 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a635c55-ee69-43ff-b75b-c1853169c0a9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:48.432387 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a635c55-ee69-43ff-b75b-c1853169c0a9 None admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:48.472420 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 368/742] 10.222.0.13 () {56 vars in 1133 bytes} [Wed Aug 31 09:34:48 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:34:48.472420 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a635c55-ee69-43ff-b75b-c1853169c0a9 None admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:48.506680 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 375/743] 10.222.0.13 () {58 vars in 1024 bytes} [Wed Aug 31 09:34:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 09:34:48.508261 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171e58ab-2808-4807-a09f-a5fdfe4b1795 None None] REQUEST_METHOD: `POST` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:48.508687 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171e58ab-2808-4807-a09f-a5fdfe4b1795 None None] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:48.509090 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171e58ab-2808-4807-a09f-a5fdfe4b1795 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:48.545323 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.auth.core [None req-171e58ab-2808-4807-a09f-a5fdfe4b1795 None None] MFA Rules not processed for user `fd836cf590a145df9acf19cf3ae2ac56`. Rule list: `[]` (Enabled: `True`). {{(pid=88113) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:34:48.548119 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.fernet_utils [None req-171e58ab-2808-4807-a09f-a5fdfe4b1795 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:34:48.635605 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a635c55-ee69-43ff-b75b-c1853169c0a9 None admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:48.662046 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 376/744] 10.222.0.13 () {58 vars in 1024 bytes} [Wed Aug 31 09:34:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 09:34:48.664947 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-171e58ab-2808-4807-a09f-a5fdfe4b1795 None None] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:48.678997 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed147105-6932-4905-948a-002ce781ffd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:48.679646 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed147105-6932-4905-948a-002ce781ffd3 None admin] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:48.679918 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed147105-6932-4905-948a-002ce781ffd3 None admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:48.680186 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed147105-6932-4905-948a-002ce781ffd3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:48.689529 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=100 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:48.697843 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed147105-6932-4905-948a-002ce781ffd3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:48.702748 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed147105-6932-4905-948a-002ce781ffd3 None admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:48.725112 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application [None req-ed147105-6932-4905-948a-002ce781ffd3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 09:34:48.725112 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:34:48.725112 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:34:48.725112 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:34:48.725112 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:34:48.725112 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:34:48.725112 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:34:48.725112 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:34:48.725112 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:34:48.725112 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:34:48.725112 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:34:48.725112 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:34:48.725112 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 09:34:48.725112 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 09:34:48.725112 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 09:34:48.725112 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 09:34:48.726999 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:34:48.726999 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:34:48.726999 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:34:48.726999 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:34:48.726999 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:34:48.726999 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:34:48.726999 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:34:48.726999 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application with Lock( Aug 31 09:34:48.726999 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:34:48.726999 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:34:48.726999 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:34:48.726999 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:34:48.726999 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:34:48.726999 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:34:48.726999 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:34:48.726999 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:34:48.726999 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 09:34:48.729127 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 09:34:48.729127 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 09:34:48.729127 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application Aug 31 09:34:48.732207 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 369/745] 10.222.0.13 () {56 vars in 1108 bytes} [Wed Aug 31 09:34:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 265 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:34:48.734002 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed147105-6932-4905-948a-002ce781ffd3 None admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:48.818100 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=100 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:48.853111 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-171e58ab-2808-4807-a09f-a5fdfe4b1795 None None] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:48.897019 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0731dd2d-2678-46a1-bc0d-6a2dd4c6f05b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:48.897780 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0731dd2d-2678-46a1-bc0d-6a2dd4c6f05b None admin] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:48.898156 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0731dd2d-2678-46a1-bc0d-6a2dd4c6f05b None admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:48.898520 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0731dd2d-2678-46a1-bc0d-6a2dd4c6f05b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:48.910860 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0731dd2d-2678-46a1-bc0d-6a2dd4c6f05b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:48.912728 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0731dd2d-2678-46a1-bc0d-6a2dd4c6f05b None admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:48.931068 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application [None req-0731dd2d-2678-46a1-bc0d-6a2dd4c6f05b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 09:34:48.931068 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:34:48.931068 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:34:48.931068 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:34:48.931068 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:34:48.931068 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:34:48.931068 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:34:48.931068 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:34:48.931068 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:34:48.931068 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:34:48.931068 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:34:48.931068 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:34:48.931068 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 09:34:48.931068 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 09:34:48.931068 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 09:34:48.931068 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 09:34:48.933131 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:34:48.933131 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:34:48.933131 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:34:48.933131 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:34:48.933131 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:34:48.933131 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:34:48.933131 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:34:48.933131 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application with Lock( Aug 31 09:34:48.933131 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:34:48.933131 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:34:48.933131 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:34:48.933131 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:34:48.933131 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:34:48.933131 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:34:48.933131 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:34:48.933131 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:34:48.933131 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 09:34:48.935028 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 09:34:48.935028 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 09:34:48.935028 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application Aug 31 09:34:48.938372 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 377/746] 10.222.0.13 () {56 vars in 1108 bytes} [Wed Aug 31 09:34:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 275 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:34:48.939626 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0731dd2d-2678-46a1-bc0d-6a2dd4c6f05b None admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:48.950083 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed147105-6932-4905-948a-002ce781ffd3 None admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:49.017354 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15bc3b72-3b36-4ca6-bda3-242cabbb9939 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:49.018438 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15bc3b72-3b36-4ca6-bda3-242cabbb9939 None admin] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:49.019162 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15bc3b72-3b36-4ca6-bda3-242cabbb9939 None admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:49.019635 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15bc3b72-3b36-4ca6-bda3-242cabbb9939 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:49.039456 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15bc3b72-3b36-4ca6-bda3-242cabbb9939 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:49.041571 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15bc3b72-3b36-4ca6-bda3-242cabbb9939 None admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:49.058594 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application [None req-15bc3b72-3b36-4ca6-bda3-242cabbb9939 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 09:34:49.058594 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:34:49.058594 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:34:49.058594 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:34:49.058594 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:34:49.058594 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:34:49.058594 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:34:49.058594 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:34:49.058594 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:34:49.058594 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:34:49.058594 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:34:49.058594 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:34:49.058594 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 09:34:49.058594 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 09:34:49.058594 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 09:34:49.058594 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 09:34:49.060440 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:34:49.060440 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:34:49.060440 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:34:49.060440 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:34:49.060440 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:34:49.060440 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:34:49.060440 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:34:49.060440 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application with Lock( Aug 31 09:34:49.060440 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:34:49.060440 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:34:49.060440 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:34:49.060440 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:34:49.060440 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:34:49.060440 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:34:49.060440 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:34:49.060440 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:34:49.060440 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 09:34:49.062916 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 09:34:49.062916 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 09:34:49.062916 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application Aug 31 09:34:49.064967 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 370/747] 10.222.0.13 () {56 vars in 1108 bytes} [Wed Aug 31 09:34:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 332 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:34:49.070159 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15bc3b72-3b36-4ca6-bda3-242cabbb9939 None admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:49.107526 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0731dd2d-2678-46a1-bc0d-6a2dd4c6f05b None admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:49.138006 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=100 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:49.158630 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=100 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:49.175410 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f55c9bd6-18b0-49d3-a92d-d4542cf16b82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:49.176449 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55c9bd6-18b0-49d3-a92d-d4542cf16b82 None admin] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:49.176933 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55c9bd6-18b0-49d3-a92d-d4542cf16b82 None admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:49.177406 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55c9bd6-18b0-49d3-a92d-d4542cf16b82 None admin] PATH_INFO: `/v3/domains` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:49.178518 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f55c9bd6-18b0-49d3-a92d-d4542cf16b82 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88113) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:34:49.179054 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f55c9bd6-18b0-49d3-a92d-d4542cf16b82 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:49.181650 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f55c9bd6-18b0-49d3-a92d-d4542cf16b82 None admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:49.198400 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 378/748] 10.222.0.13 () {56 vars in 1109 bytes} [Wed Aug 31 09:34:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:34:49.199094 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f55c9bd6-18b0-49d3-a92d-d4542cf16b82 None admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:49.227091 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15bc3b72-3b36-4ca6-bda3-242cabbb9939 None admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:49.266446 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b69ca38-4fff-42d8-9b29-749d24422f60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:49.267070 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b69ca38-4fff-42d8-9b29-749d24422f60 None admin] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:49.267342 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b69ca38-4fff-42d8-9b29-749d24422f60 None admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:49.267608 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b69ca38-4fff-42d8-9b29-749d24422f60 None admin] PATH_INFO: `/v3/domains` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:49.268264 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b69ca38-4fff-42d8-9b29-749d24422f60 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88114) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:34:49.268572 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b69ca38-4fff-42d8-9b29-749d24422f60 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:49.270265 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b69ca38-4fff-42d8-9b29-749d24422f60 None admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:49.283835 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 371/749] 10.222.0.13 () {56 vars in 1109 bytes} [Wed Aug 31 09:34:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:34:49.284714 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b69ca38-4fff-42d8-9b29-749d24422f60 None admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:49.336212 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f55c9bd6-18b0-49d3-a92d-d4542cf16b82 None admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:49.378184 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a7030d5-a358-4180-87d6-ef00a655f66d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:49.378915 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7030d5-a358-4180-87d6-ef00a655f66d None admin] REQUEST_METHOD: `POST` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:49.379255 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7030d5-a358-4180-87d6-ef00a655f66d None admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:49.379620 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7030d5-a358-4180-87d6-ef00a655f66d None admin] PATH_INFO: `/v3/services` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:49.380350 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a7030d5-a358-4180-87d6-ef00a655f66d None admin] RBAC: Authorizing `identity:create_service()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:49.381988 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a7030d5-a358-4180-87d6-ef00a655f66d None admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:49.401048 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 379/750] 10.222.0.13 () {60 vars in 1141 bytes} [Wed Aug 31 09:34:49 2022] POST /identity/v3/services => generated 251 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 09:34:49.401932 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a7030d5-a358-4180-87d6-ef00a655f66d None admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:49.425044 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b69ca38-4fff-42d8-9b29-749d24422f60 None admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:49.428102 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=100 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:49.483022 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:49.541570 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f03477-0bdf-4777-b435-aa5c6d36df86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:49.542638 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f03477-0bdf-4777-b435-aa5c6d36df86 None admin] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:49.543131 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f03477-0bdf-4777-b435-aa5c6d36df86 None admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:49.543610 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f03477-0bdf-4777-b435-aa5c6d36df86 None admin] PATH_INFO: `/v3/domains` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:49.544833 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41f03477-0bdf-4777-b435-aa5c6d36df86 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88114) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:34:49.545409 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41f03477-0bdf-4777-b435-aa5c6d36df86 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:49.549264 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41f03477-0bdf-4777-b435-aa5c6d36df86 None admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:49.573641 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 372/751] 10.222.0.13 () {56 vars in 1109 bytes} [Wed Aug 31 09:34:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 289 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:34:49.573931 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a7030d5-a358-4180-87d6-ef00a655f66d None admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:49.574364 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f03477-0bdf-4777-b435-aa5c6d36df86 None admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:49.631379 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e29ce442-1b30-407d-8969-5e003b8dca5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:49.632276 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29ce442-1b30-407d-8969-5e003b8dca5a None admin] REQUEST_METHOD: `POST` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:49.632745 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29ce442-1b30-407d-8969-5e003b8dca5a None admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:49.633186 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29ce442-1b30-407d-8969-5e003b8dca5a None admin] PATH_INFO: `/v3/services` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:49.634347 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e29ce442-1b30-407d-8969-5e003b8dca5a None admin] RBAC: Authorizing `identity:create_service()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:49.636817 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e29ce442-1b30-407d-8969-5e003b8dca5a None admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:49.669427 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 380/752] 10.222.0.13 () {60 vars in 1141 bytes} [Wed Aug 31 09:34:49 2022] POST /identity/v3/services => generated 240 bytes in 262 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 09:34:49.669427 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e29ce442-1b30-407d-8969-5e003b8dca5a None admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:49.721442 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f03477-0bdf-4777-b435-aa5c6d36df86 None admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:49.771334 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-006bb903-ae6d-41a6-82b5-8a867da9df56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:49.772295 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006bb903-ae6d-41a6-82b5-8a867da9df56 None admin] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:49.772758 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006bb903-ae6d-41a6-82b5-8a867da9df56 None admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:49.773263 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006bb903-ae6d-41a6-82b5-8a867da9df56 None admin] PATH_INFO: `/v3/projects/service` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:49.782529 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-006bb903-ae6d-41a6-82b5-8a867da9df56 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:49.784285 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-006bb903-ae6d-41a6-82b5-8a867da9df56 None admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:49.798623 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application [None req-006bb903-ae6d-41a6-82b5-8a867da9df56 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 09:34:49.798623 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:34:49.798623 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:34:49.798623 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:34:49.798623 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:34:49.798623 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:34:49.798623 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:34:49.798623 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:34:49.798623 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:34:49.798623 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:34:49.798623 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:34:49.798623 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:34:49.798623 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 31 09:34:49.798623 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 31 09:34:49.798623 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 31 09:34:49.799689 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 31 09:34:49.799689 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:34:49.799689 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:34:49.799689 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:34:49.799689 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:34:49.799689 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:34:49.799689 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:34:49.799689 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:34:49.799689 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application with Lock( Aug 31 09:34:49.799689 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:34:49.799689 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:34:49.799689 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:34:49.799689 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:34:49.799689 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:34:49.799689 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:34:49.799689 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:34:49.799689 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:34:49.800953 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 31 09:34:49.800953 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 31 09:34:49.800953 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 31 09:34:49.800953 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 31 09:34:49.800953 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 31 09:34:49.800953 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 31 09:34:49.800953 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 09:34:49.800953 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application Aug 31 09:34:49.803096 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 373/753] 10.222.0.13 () {56 vars in 1111 bytes} [Wed Aug 31 09:34:49 2022] GET /identity/v3/projects/service => generated 88 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:34:49.804172 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-006bb903-ae6d-41a6-82b5-8a867da9df56 None admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:49.824180 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=100 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:49.893155 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e29ce442-1b30-407d-8969-5e003b8dca5a None admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:49.909773 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:49.935495 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-006bb903-ae6d-41a6-82b5-8a867da9df56 None admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:49.973800 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53296949-7944-4962-8dd9-77facc89de20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:49.973800 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53296949-7944-4962-8dd9-77facc89de20 None admin] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:49.973800 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53296949-7944-4962-8dd9-77facc89de20 None admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:49.973800 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53296949-7944-4962-8dd9-77facc89de20 None admin] PATH_INFO: `/v3/users/swift` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:49.986558 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53296949-7944-4962-8dd9-77facc89de20 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:49.989460 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53296949-7944-4962-8dd9-77facc89de20 None admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:49.997323 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a22a91-0b2e-4722-be43-c076bdf224e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:49.998016 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a22a91-0b2e-4722-be43-c076bdf224e5 None admin] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:49.998291 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a22a91-0b2e-4722-be43-c076bdf224e5 None admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:49.998592 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a22a91-0b2e-4722-be43-c076bdf224e5 None admin] PATH_INFO: `/v3/projects/service` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:50.007041 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5a22a91-0b2e-4722-be43-c076bdf224e5 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:50.008886 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5a22a91-0b2e-4722-be43-c076bdf224e5 None admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:50.019766 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application [None req-53296949-7944-4962-8dd9-77facc89de20 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 31 09:34:50.019766 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:34:50.019766 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:34:50.019766 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:34:50.019766 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:34:50.019766 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:34:50.019766 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:34:50.019766 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:34:50.019766 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:34:50.019766 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:34:50.019766 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:34:50.019766 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:34:50.019766 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 31 09:34:50.019766 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 31 09:34:50.019766 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 31 09:34:50.019766 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 31 09:34:50.022089 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:34:50.022089 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:34:50.022089 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 31 09:34:50.022089 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 09:34:50.022089 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 31 09:34:50.022089 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 09:34:50.022089 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:34:50.022089 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:34:50.022089 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:34:50.022089 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:34:50.022089 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:34:50.022089 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application with Lock( Aug 31 09:34:50.022089 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:34:50.022089 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:34:50.022089 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:34:50.022089 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:34:50.022089 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:34:50.022089 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:34:50.024417 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:34:50.024417 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:34:50.024417 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 31 09:34:50.024417 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 31 09:34:50.024417 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 31 09:34:50.024417 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 31 09:34:50.024417 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 31 09:34:50.024417 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 31 09:34:50.024417 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 31 09:34:50.024417 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application Aug 31 09:34:50.025940 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application [None req-a5a22a91-0b2e-4722-be43-c076bdf224e5 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 09:34:50.025940 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:34:50.025940 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:34:50.025940 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:34:50.025940 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:34:50.025940 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:34:50.025940 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:34:50.025940 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:34:50.025940 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:34:50.025940 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:34:50.025940 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:34:50.025940 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:34:50.025940 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 31 09:34:50.025940 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 31 09:34:50.025940 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 31 09:34:50.027770 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 31 09:34:50.027770 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:34:50.027770 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:34:50.027770 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:34:50.027770 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:34:50.027770 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:34:50.027770 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:34:50.027770 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:34:50.027770 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application with Lock( Aug 31 09:34:50.027770 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:34:50.027770 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:34:50.027770 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:34:50.027770 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:34:50.027770 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:34:50.027770 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:34:50.027770 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:34:50.027770 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:34:50.032455 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 31 09:34:50.032455 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 31 09:34:50.032455 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 31 09:34:50.032455 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 31 09:34:50.032455 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 31 09:34:50.032455 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 31 09:34:50.032455 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 09:34:50.032455 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application Aug 31 09:34:50.032455 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 374/754] 10.222.0.13 () {56 vars in 1111 bytes} [Wed Aug 31 09:34:49 2022] GET /identity/v3/projects/service => generated 88 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:34:50.032455 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a22a91-0b2e-4722-be43-c076bdf224e5 None admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:50.033680 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 381/755] 10.222.0.13 () {56 vars in 1096 bytes} [Wed Aug 31 09:34:49 2022] GET /identity/v3/users/swift => generated 83 bytes in 362 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:34:50.040124 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53296949-7944-4962-8dd9-77facc89de20 None admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:50.134120 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=100 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:50.165815 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a22a91-0b2e-4722-be43-c076bdf224e5 None admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:50.213924 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ac17e2-1389-490f-81c0-a3425b13ded5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:50.214689 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ac17e2-1389-490f-81c0-a3425b13ded5 None admin] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:50.215035 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ac17e2-1389-490f-81c0-a3425b13ded5 None admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:50.215473 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ac17e2-1389-490f-81c0-a3425b13ded5 None admin] PATH_INFO: `/v3/projects` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:50.216317 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92ac17e2-1389-490f-81c0-a3425b13ded5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88114) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:34:50.216624 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92ac17e2-1389-490f-81c0-a3425b13ded5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:50.218932 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92ac17e2-1389-490f-81c0-a3425b13ded5 None admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:50.231579 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53296949-7944-4962-8dd9-77facc89de20 None admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:50.236702 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 375/756] 10.222.0.13 () {56 vars in 1148 bytes} [Wed Aug 31 09:34:50 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:34:50.237632 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ac17e2-1389-490f-81c0-a3425b13ded5 None admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:50.304457 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5818cec9-1a02-42a0-80f3-a5dece775b17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:50.305629 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5818cec9-1a02-42a0-80f3-a5dece775b17 None admin] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:50.306108 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5818cec9-1a02-42a0-80f3-a5dece775b17 None admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:50.306576 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5818cec9-1a02-42a0-80f3-a5dece775b17 None admin] PATH_INFO: `/v3/users` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:50.308083 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5818cec9-1a02-42a0-80f3-a5dece775b17 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=88113) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:34:50.308720 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5818cec9-1a02-42a0-80f3-a5dece775b17 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:50.311743 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5818cec9-1a02-42a0-80f3-a5dece775b17 None admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:50.350508 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 382/757] 10.222.0.13 () {56 vars in 1135 bytes} [Wed Aug 31 09:34:50 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 313 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:34:50.351979 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5818cec9-1a02-42a0-80f3-a5dece775b17 None admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:50.356183 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ac17e2-1389-490f-81c0-a3425b13ded5 None admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:50.397932 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec134657-f996-4c62-87cb-c3e86581a2e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:50.398559 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec134657-f996-4c62-87cb-c3e86581a2e0 None admin] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:50.398831 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec134657-f996-4c62-87cb-c3e86581a2e0 None admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:50.399095 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec134657-f996-4c62-87cb-c3e86581a2e0 None admin] PATH_INFO: `/v3/projects` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:50.399830 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec134657-f996-4c62-87cb-c3e86581a2e0 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88114) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:34:50.400114 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec134657-f996-4c62-87cb-c3e86581a2e0 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:50.401696 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec134657-f996-4c62-87cb-c3e86581a2e0 None admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:50.403792 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=100 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:50.408047 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=100 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:50.416766 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 376/758] 10.222.0.13 () {56 vars in 1148 bytes} [Wed Aug 31 09:34:50 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:34:50.417643 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec134657-f996-4c62-87cb-c3e86581a2e0 None admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:50.540848 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec134657-f996-4c62-87cb-c3e86581a2e0 None admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:50.578379 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5818cec9-1a02-42a0-80f3-a5dece775b17 None admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:50.586611 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-484dbe8e-3920-4633-9aab-d193249bf526 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:50.587532 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484dbe8e-3920-4633-9aab-d193249bf526 None admin] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:50.587955 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484dbe8e-3920-4633-9aab-d193249bf526 None admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:50.588385 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484dbe8e-3920-4633-9aab-d193249bf526 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:50.651445 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e431b0a-0f17-4dd6-b0a8-54b2bdb6645a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:50.652938 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e431b0a-0f17-4dd6-b0a8-54b2bdb6645a None admin] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:50.653586 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e431b0a-0f17-4dd6-b0a8-54b2bdb6645a None admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:50.654191 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e431b0a-0f17-4dd6-b0a8-54b2bdb6645a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:50.656365 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e431b0a-0f17-4dd6-b0a8-54b2bdb6645a None admin] RBAC: Adding query filter params (role.id=e24ba4fd91584883af350cc84bc96c1d, scope.project.id=49548eb44d62415582af820ab66decee, user.id=768cb67c1ef24251aaa113c12d81a250) {{(pid=88113) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:34:50.657122 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e431b0a-0f17-4dd6-b0a8-54b2bdb6645a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:50.660451 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e431b0a-0f17-4dd6-b0a8-54b2bdb6645a None admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:50.665787 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=100 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:50.675238 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-484dbe8e-3920-4633-9aab-d193249bf526 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:50.675621 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 383/759] 10.222.0.13 () {56 vars in 1374 bytes} [Wed Aug 31 09:34:50 2022] GET /identity/v3/role_assignments?user.id=768cb67c1ef24251aaa113c12d81a250&scope.project.id=49548eb44d62415582af820ab66decee&role.id=e24ba4fd91584883af350cc84bc96c1d => generated 259 bytes in 324 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:34:50.677185 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e431b0a-0f17-4dd6-b0a8-54b2bdb6645a None admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:50.677642 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-484dbe8e-3920-4633-9aab-d193249bf526 None admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:50.803228 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 377/760] 10.222.0.13 () {58 vars in 1282 bytes} [Wed Aug 31 09:34:50 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 386 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 31 09:34:50.811388 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-484dbe8e-3920-4633-9aab-d193249bf526 None admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:50.862927 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=100 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:50.891312 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e431b0a-0f17-4dd6-b0a8-54b2bdb6645a None admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:50.965568 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6267834-0418-4f6a-aed7-e1be0fc6882b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:50.966967 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6267834-0418-4f6a-aed7-e1be0fc6882b None admin] REQUEST_METHOD: `PUT` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:50.968213 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6267834-0418-4f6a-aed7-e1be0fc6882b None admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:50.968950 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6267834-0418-4f6a-aed7-e1be0fc6882b None admin] PATH_INFO: `/v3/projects/49548eb44d62415582af820ab66decee/users/dc076493239d4ccdbdc37c0369c49529/roles/5d4ee63a3add47fc9a6b8b16d39e0a1f` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:50.975248 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6267834-0418-4f6a-aed7-e1be0fc6882b None admin] RBAC: Authorizing `identity:create_grant(project_id=49548eb44d62415582af820ab66decee, user_id=dc076493239d4ccdbdc37c0369c49529, role_id=5d4ee63a3add47fc9a6b8b16d39e0a1f)` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:50.978887 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6267834-0418-4f6a-aed7-e1be0fc6882b None admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:51.018014 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=100 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:51.022995 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 384/761] 10.222.0.13 () {58 vars in 1439 bytes} [Wed Aug 31 09:34:50 2022] PUT /identity/v3/projects/49548eb44d62415582af820ab66decee/users/dc076493239d4ccdbdc37c0369c49529/roles/5d4ee63a3add47fc9a6b8b16d39e0a1f => generated 0 bytes in 347 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:34:51.032348 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-484dbe8e-3920-4633-9aab-d193249bf526 None admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:51.090337 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-956e6ebd-57d7-4803-a40d-01ed382dde9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:51.090990 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956e6ebd-57d7-4803-a40d-01ed382dde9c None admin] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:51.091269 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956e6ebd-57d7-4803-a40d-01ed382dde9c None admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:51.091542 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956e6ebd-57d7-4803-a40d-01ed382dde9c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:51.107744 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-956e6ebd-57d7-4803-a40d-01ed382dde9c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:51.109323 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-956e6ebd-57d7-4803-a40d-01ed382dde9c None admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:51.128975 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application [None req-956e6ebd-57d7-4803-a40d-01ed382dde9c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 09:34:51.128975 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:34:51.128975 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:34:51.128975 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:34:51.128975 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:34:51.128975 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:34:51.128975 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:34:51.128975 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:34:51.128975 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:34:51.128975 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:34:51.128975 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:34:51.128975 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:34:51.128975 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 09:34:51.128975 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 09:34:51.128975 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 09:34:51.128975 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 09:34:51.130904 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:34:51.130904 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:34:51.130904 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:34:51.130904 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:34:51.130904 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:34:51.130904 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:34:51.130904 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:34:51.130904 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application with Lock( Aug 31 09:34:51.130904 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:34:51.130904 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:34:51.130904 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:34:51.130904 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:34:51.130904 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:34:51.130904 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:34:51.130904 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:34:51.130904 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:34:51.130904 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 09:34:51.132618 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 09:34:51.132618 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 09:34:51.132618 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application Aug 31 09:34:51.132618 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 378/762] 10.222.0.13 () {56 vars in 1108 bytes} [Wed Aug 31 09:34:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 320 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:34:51.138954 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6267834-0418-4f6a-aed7-e1be0fc6882b None admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:51.371107 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6267834-0418-4f6a-aed7-e1be0fc6882b None admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:51.428724 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e1ad565-d72c-4178-83d6-c83ed0528bb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:51.429361 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1ad565-d72c-4178-83d6-c83ed0528bb8 None admin] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:51.429670 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1ad565-d72c-4178-83d6-c83ed0528bb8 None admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:51.429941 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1ad565-d72c-4178-83d6-c83ed0528bb8 None admin] PATH_INFO: `/v3/domains` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:51.430609 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e1ad565-d72c-4178-83d6-c83ed0528bb8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88113) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:34:51.430912 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e1ad565-d72c-4178-83d6-c83ed0528bb8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:51.432331 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e1ad565-d72c-4178-83d6-c83ed0528bb8 None admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:51.450419 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 385/763] 10.222.0.13 () {56 vars in 1109 bytes} [Wed Aug 31 09:34:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 312 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:34:51.460001 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-956e6ebd-57d7-4803-a40d-01ed382dde9c None admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:51.653304 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-956e6ebd-57d7-4803-a40d-01ed382dde9c None admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:51.742213 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7ec52f9-2239-4b18-a911-4f0c982bce1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:51.743228 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ec52f9-2239-4b18-a911-4f0c982bce1e None admin] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:51.743694 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ec52f9-2239-4b18-a911-4f0c982bce1e None admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:51.744165 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ec52f9-2239-4b18-a911-4f0c982bce1e None admin] PATH_INFO: `/v3/projects/service` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:51.761892 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7ec52f9-2239-4b18-a911-4f0c982bce1e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:51.764562 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7ec52f9-2239-4b18-a911-4f0c982bce1e None admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:51.790852 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application [None req-c7ec52f9-2239-4b18-a911-4f0c982bce1e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 09:34:51.790852 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:34:51.790852 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:34:51.790852 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:34:51.790852 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:34:51.790852 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:34:51.790852 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:34:51.790852 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:34:51.790852 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:34:51.790852 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:34:51.790852 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:34:51.790852 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:34:51.790852 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 31 09:34:51.790852 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 31 09:34:51.790852 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 31 09:34:51.792422 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 31 09:34:51.792422 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:34:51.792422 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:34:51.792422 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:34:51.792422 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:34:51.792422 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:34:51.792422 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:34:51.792422 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:34:51.792422 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application with Lock( Aug 31 09:34:51.792422 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:34:51.792422 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:34:51.792422 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:34:51.792422 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:34:51.792422 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:34:51.792422 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:34:51.792422 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:34:51.792422 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:34:51.794196 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 31 09:34:51.794196 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 31 09:34:51.794196 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 31 09:34:51.794196 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 31 09:34:51.794196 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 31 09:34:51.794196 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 31 09:34:51.794196 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 09:34:51.794196 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application Aug 31 09:34:51.794196 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 379/764] 10.222.0.13 () {56 vars in 1111 bytes} [Wed Aug 31 09:34:51 2022] GET /identity/v3/projects/service => generated 88 bytes in 334 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:34:51.802638 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e1ad565-d72c-4178-83d6-c83ed0528bb8 None admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:51.824775 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:52.001121 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e1ad565-d72c-4178-83d6-c83ed0528bb8 None admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:52.045016 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bae4363-e861-49cc-aa47-ffe8e0dcf4c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:52.045909 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bae4363-e861-49cc-aa47-ffe8e0dcf4c5 None admin] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:52.046212 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bae4363-e861-49cc-aa47-ffe8e0dcf4c5 None admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:52.046353 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bae4363-e861-49cc-aa47-ffe8e0dcf4c5 None admin] PATH_INFO: `/v3/projects` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:52.046975 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bae4363-e861-49cc-aa47-ffe8e0dcf4c5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88113) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:34:52.047281 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bae4363-e861-49cc-aa47-ffe8e0dcf4c5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:52.048893 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bae4363-e861-49cc-aa47-ffe8e0dcf4c5 None admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:52.062610 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 386/765] 10.222.0.13 () {56 vars in 1148 bytes} [Wed Aug 31 09:34:51 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:34:52.074147 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7ec52f9-2239-4b18-a911-4f0c982bce1e None admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:52.096439 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=100 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:52.185569 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5baa76d-a4be-4015-9c5d-7caa35ea4eeb None None] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:52.185871 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5baa76d-a4be-4015-9c5d-7caa35ea4eeb None None] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:52.186739 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5baa76d-a4be-4015-9c5d-7caa35ea4eeb None None] PATH_INFO: `/` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:52.191480 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 387/766] 10.222.0.13 () {54 vars in 927 bytes} [Wed Aug 31 09:34:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:34:52.208061 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e87587-a0ae-43e0-953b-7503153a069e None None] REQUEST_METHOD: `POST` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:52.208323 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e87587-a0ae-43e0-953b-7503153a069e None None] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:52.208599 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e87587-a0ae-43e0-953b-7503153a069e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:52.231490 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.auth.core [None req-33e87587-a0ae-43e0-953b-7503153a069e None None] MFA Rules not processed for user `fd836cf590a145df9acf19cf3ae2ac56`. Rule list: `[]` (Enabled: `True`). {{(pid=88113) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:34:52.232648 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.fernet_utils [None req-33e87587-a0ae-43e0-953b-7503153a069e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:34:52.336315 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=100 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:52.346597 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 388/767] 10.222.0.13 () {58 vars in 1024 bytes} [Wed Aug 31 09:34:52 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 364 bytes (3 switches on core 0) Aug 31 09:34:52.360422 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7ec52f9-2239-4b18-a911-4f0c982bce1e None admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:52.447658 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c8a0f6-cdb8-4fa9-a608-30f78cfe0a5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:52.448722 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c8a0f6-cdb8-4fa9-a608-30f78cfe0a5f None admin] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:52.449200 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c8a0f6-cdb8-4fa9-a608-30f78cfe0a5f None admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:52.449707 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c8a0f6-cdb8-4fa9-a608-30f78cfe0a5f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:52.451047 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84c8a0f6-cdb8-4fa9-a608-30f78cfe0a5f None admin] RBAC: Adding query filter params (role.id=5d4ee63a3add47fc9a6b8b16d39e0a1f, scope.project.id=49548eb44d62415582af820ab66decee, user.id=202bb9db7a8a4977ba788a290ed9a3fa) {{(pid=88114) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:34:52.452006 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84c8a0f6-cdb8-4fa9-a608-30f78cfe0a5f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:52.454822 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84c8a0f6-cdb8-4fa9-a608-30f78cfe0a5f None admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:52.470664 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 380/768] 10.222.0.13 () {56 vars in 1374 bytes} [Wed Aug 31 09:34:52 2022] GET /identity/v3/role_assignments?user.id=202bb9db7a8a4977ba788a290ed9a3fa&scope.project.id=49548eb44d62415582af820ab66decee&role.id=5d4ee63a3add47fc9a6b8b16d39e0a1f => generated 259 bytes in 397 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:34:52.933426 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e87587-a0ae-43e0-953b-7503153a069e None None] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:53.085811 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e87587-a0ae-43e0-953b-7503153a069e None None] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:53.147136 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa23e742-a101-41a3-b49d-c931b12f86fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:53.147794 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa23e742-a101-41a3-b49d-c931b12f86fd None admin] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:53.148068 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa23e742-a101-41a3-b49d-c931b12f86fd None admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:53.148327 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa23e742-a101-41a3-b49d-c931b12f86fd None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:53.148922 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa23e742-a101-41a3-b49d-c931b12f86fd None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:53.150351 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa23e742-a101-41a3-b49d-c931b12f86fd None admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:53.163721 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application [None req-fa23e742-a101-41a3-b49d-c931b12f86fd None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 31 09:34:53.163721 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:34:53.163721 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 31 09:34:53.163721 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 31 09:34:53.163721 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 31 09:34:53.163721 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 31 09:34:53.163721 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 31 09:34:53.163721 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 09:34:53.163721 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 31 09:34:53.163721 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application Aug 31 09:34:53.163721 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 31 09:34:53.163721 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application Aug 31 09:34:53.163721 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:34:53.163721 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:34:53.163721 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:34:53.163721 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:34:53.163721 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:34:53.165850 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:34:53.165850 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:34:53.165850 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:34:53.165850 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:34:53.165850 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:34:53.165850 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:34:53.165850 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 31 09:34:53.165850 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 31 09:34:53.165850 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 31 09:34:53.165850 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 31 09:34:53.165850 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:34:53.165850 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:34:53.165850 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:34:53.165850 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:34:53.165850 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:34:53.165850 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:34:53.167579 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:34:53.167579 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application with Lock( Aug 31 09:34:53.167579 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:34:53.167579 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:34:53.167579 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:34:53.167579 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:34:53.167579 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:34:53.167579 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:34:53.167579 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:34:53.167579 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:34:53.167579 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 31 09:34:53.167579 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 09:34:53.167579 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 31 09:34:53.167579 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application Aug 31 09:34:53.167579 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 389/769] 10.222.0.13 () {56 vars in 1129 bytes} [Wed Aug 31 09:34:52 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:34:53.172868 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c8a0f6-cdb8-4fa9-a608-30f78cfe0a5f None admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:53.281555 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=100 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:53.312487 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c8a0f6-cdb8-4fa9-a608-30f78cfe0a5f None admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:53.355180 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-545edcba-3bfb-45ca-b493-891efd44cc29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:53.355806 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545edcba-3bfb-45ca-b493-891efd44cc29 None admin] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:53.356072 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545edcba-3bfb-45ca-b493-891efd44cc29 None admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:53.356324 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545edcba-3bfb-45ca-b493-891efd44cc29 None admin] PATH_INFO: `/v3/services` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:53.356967 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-545edcba-3bfb-45ca-b493-891efd44cc29 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=88114) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:34:53.357274 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-545edcba-3bfb-45ca-b493-891efd44cc29 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:53.359465 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-545edcba-3bfb-45ca-b493-891efd44cc29 None admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:53.364742 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 381/770] 10.222.0.13 () {56 vars in 1124 bytes} [Wed Aug 31 09:34:53 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 09:34:53.374646 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa23e742-a101-41a3-b49d-c931b12f86fd None admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:53.402431 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeadc92e-6307-432f-8ad8-f816926d69d7 None None] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:53.402873 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeadc92e-6307-432f-8ad8-f816926d69d7 None None] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:53.403127 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeadc92e-6307-432f-8ad8-f816926d69d7 None None] PATH_INFO: `/` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:53.405215 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 382/771] 10.222.0.13 () {54 vars in 927 bytes} [Wed Aug 31 09:34:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:34:53.418230 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141b4e56-0828-486e-9c6c-048a9c1bfa53 None None] REQUEST_METHOD: `POST` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:53.418675 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141b4e56-0828-486e-9c6c-048a9c1bfa53 None None] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:53.418930 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141b4e56-0828-486e-9c6c-048a9c1bfa53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:53.444602 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.auth.core [None req-141b4e56-0828-486e-9c6c-048a9c1bfa53 None None] MFA Rules not processed for user `fd836cf590a145df9acf19cf3ae2ac56`. Rule list: `[]` (Enabled: `True`). {{(pid=88114) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:34:53.449190 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.fernet_utils [None req-141b4e56-0828-486e-9c6c-048a9c1bfa53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88114) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:34:53.505957 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=100 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:53.527847 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 383/772] 10.222.0.13 () {58 vars in 1024 bytes} [Wed Aug 31 09:34:53 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 09:34:53.597038 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa23e742-a101-41a3-b49d-c931b12f86fd None admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:53.704085 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d69a64-6ed3-44ca-9a8c-36316583fb64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:53.705130 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d69a64-6ed3-44ca-9a8c-36316583fb64 None admin] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:53.705637 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d69a64-6ed3-44ca-9a8c-36316583fb64 None admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:53.706129 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d69a64-6ed3-44ca-9a8c-36316583fb64 None admin] PATH_INFO: `/v3/services` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:53.707266 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24d69a64-6ed3-44ca-9a8c-36316583fb64 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=88113) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:34:53.707805 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24d69a64-6ed3-44ca-9a8c-36316583fb64 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:53.710369 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24d69a64-6ed3-44ca-9a8c-36316583fb64 None admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:53.719944 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 390/773] 10.222.0.13 () {56 vars in 1124 bytes} [Wed Aug 31 09:34:53 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 346 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:34:53.729127 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-141b4e56-0828-486e-9c6c-048a9c1bfa53 None None] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:53.791190 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83e4c8e-6450-4f5b-a7f0-a4f736b055fe None None] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:53.791639 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83e4c8e-6450-4f5b-a7f0-a4f736b055fe None None] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:53.792096 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83e4c8e-6450-4f5b-a7f0-a4f736b055fe None None] PATH_INFO: `/` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:53.801417 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 391/774] 10.222.0.13 () {54 vars in 927 bytes} [Wed Aug 31 09:34:53 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:34:53.806133 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b9c0e4-88b6-40ce-b133-46e0387207b7 None None] REQUEST_METHOD: `POST` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:53.806594 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b9c0e4-88b6-40ce-b133-46e0387207b7 None None] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:53.807056 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b9c0e4-88b6-40ce-b133-46e0387207b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:53.853514 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.auth.core [None req-a1b9c0e4-88b6-40ce-b133-46e0387207b7 None None] MFA Rules not processed for user `fd836cf590a145df9acf19cf3ae2ac56`. Rule list: `[]` (Enabled: `True`). {{(pid=88113) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:34:53.853514 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.fernet_utils [None req-a1b9c0e4-88b6-40ce-b133-46e0387207b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:34:53.877652 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=100 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:53.950954 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-141b4e56-0828-486e-9c6c-048a9c1bfa53 None None] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:53.994358 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 392/775] 10.222.0.13 () {58 vars in 1024 bytes} [Wed Aug 31 09:34:53 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 31 09:34:53.994358 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b9c0e4-88b6-40ce-b133-46e0387207b7 None None] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:54.017175 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ef0bca1-d2bc-45eb-ae4f-56a9cf17d1c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:54.018305 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef0bca1-d2bc-45eb-ae4f-56a9cf17d1c2 None admin] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:54.019134 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef0bca1-d2bc-45eb-ae4f-56a9cf17d1c2 None admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:54.019748 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef0bca1-d2bc-45eb-ae4f-56a9cf17d1c2 None admin] PATH_INFO: `/v3/endpoints` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:54.021296 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ef0bca1-d2bc-45eb-ae4f-56a9cf17d1c2 None admin] RBAC: Adding query filter params (interface=public, service_id=0856264cf83244b69534e0eedf17c3ae, region_id=RegionOne) {{(pid=88114) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:34:54.022013 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ef0bca1-d2bc-45eb-ae4f-56a9cf17d1c2 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:54.024654 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ef0bca1-d2bc-45eb-ae4f-56a9cf17d1c2 None admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:54.039384 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 384/776] 10.222.0.13 () {56 vars in 1251 bytes} [Wed Aug 31 09:34:53 2022] GET /identity/v3/endpoints?service_id=0856264cf83244b69534e0eedf17c3ae&interface=public®ion_id=RegionOne => generated 194 bytes in 311 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:34:54.046515 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ef0bca1-d2bc-45eb-ae4f-56a9cf17d1c2 None admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:54.120818 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=100 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:54.265348 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ef0bca1-d2bc-45eb-ae4f-56a9cf17d1c2 None admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:54.313757 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b9c0e4-88b6-40ce-b133-46e0387207b7 None None] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:54.321841 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=100 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:54.337797 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-360926a3-4c65-4a05-8e4b-8f3f84d55cb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:54.339746 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360926a3-4c65-4a05-8e4b-8f3f84d55cb7 None admin] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:54.340225 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360926a3-4c65-4a05-8e4b-8f3f84d55cb7 None admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:54.340664 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360926a3-4c65-4a05-8e4b-8f3f84d55cb7 None admin] PATH_INFO: `/v3/services` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:54.341724 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-360926a3-4c65-4a05-8e4b-8f3f84d55cb7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:54.345017 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-360926a3-4c65-4a05-8e4b-8f3f84d55cb7 None admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:54.357014 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 385/777] 10.222.0.13 () {56 vars in 1087 bytes} [Wed Aug 31 09:34:54 2022] GET /identity/v3/services => generated 772 bytes in 312 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:34:54.423607 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9451da99-c08e-4a0c-baaf-f3c81ee8c586 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:54.424538 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9451da99-c08e-4a0c-baaf-f3c81ee8c586 None admin] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:54.425222 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9451da99-c08e-4a0c-baaf-f3c81ee8c586 None admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:54.425780 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9451da99-c08e-4a0c-baaf-f3c81ee8c586 None admin] PATH_INFO: `/v3/services/network` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:54.426788 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9451da99-c08e-4a0c-baaf-f3c81ee8c586 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:54.429250 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9451da99-c08e-4a0c-baaf-f3c81ee8c586 None admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:54.448618 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application [None req-9451da99-c08e-4a0c-baaf-f3c81ee8c586 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Aug 31 09:34:54.448618 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:34:54.448618 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 31 09:34:54.448618 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 31 09:34:54.448618 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 31 09:34:54.448618 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 31 09:34:54.448618 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 31 09:34:54.448618 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 09:34:54.448618 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 31 09:34:54.448618 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application Aug 31 09:34:54.448618 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 31 09:34:54.448618 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application Aug 31 09:34:54.448618 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:34:54.448618 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:34:54.448618 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:34:54.448618 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:34:54.448618 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:34:54.453047 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:34:54.453047 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:34:54.453047 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:34:54.453047 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:34:54.453047 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:34:54.453047 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:34:54.453047 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 31 09:34:54.453047 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 31 09:34:54.453047 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 31 09:34:54.453047 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 31 09:34:54.453047 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:34:54.453047 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:34:54.453047 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:34:54.453047 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:34:54.453047 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:34:54.453047 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:34:54.456462 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:34:54.456462 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application with Lock( Aug 31 09:34:54.456462 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:34:54.456462 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:34:54.456462 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:34:54.456462 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:34:54.456462 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:34:54.456462 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:34:54.456462 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:34:54.456462 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:34:54.456462 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 31 09:34:54.456462 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 09:34:54.456462 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 31 09:34:54.456462 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application Aug 31 09:34:54.459505 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 393/778] 10.222.0.13 () {56 vars in 1111 bytes} [Wed Aug 31 09:34:53 2022] GET /identity/v3/services/network => generated 88 bytes in 475 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:34:54.463153 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9451da99-c08e-4a0c-baaf-f3c81ee8c586 None admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:54.465435 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-360926a3-4c65-4a05-8e4b-8f3f84d55cb7 None admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:54.585962 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9451da99-c08e-4a0c-baaf-f3c81ee8c586 None admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:54.620671 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=100 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:54.650270 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-934f8dac-ae02-41c0-a4b6-1426006af254 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:54.651945 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934f8dac-ae02-41c0-a4b6-1426006af254 None admin] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:54.652628 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934f8dac-ae02-41c0-a4b6-1426006af254 None admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:54.653151 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934f8dac-ae02-41c0-a4b6-1426006af254 None admin] PATH_INFO: `/v3/roles/service` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:54.668106 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-934f8dac-ae02-41c0-a4b6-1426006af254 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 09:34:54.676629 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application [None req-934f8dac-ae02-41c0-a4b6-1426006af254 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 31 09:34:54.676629 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:34:54.676629 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:34:54.676629 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:34:54.676629 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:34:54.676629 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:34:54.676629 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:34:54.676629 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:34:54.676629 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:34:54.676629 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:34:54.676629 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:34:54.676629 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:34:54.676629 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 09:34:54.676629 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 09:34:54.676629 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 09:34:54.676629 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 09:34:54.678186 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 09:34:54.678186 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 09:34:54.678186 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 09:34:54.678186 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 09:34:54.678186 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:34:54.678186 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:34:54.678186 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:34:54.678186 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:34:54.678186 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:34:54.678186 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:34:54.678186 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:34:54.678186 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application with Lock( Aug 31 09:34:54.678186 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:34:54.678186 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:34:54.678186 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:34:54.678186 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:34:54.679050 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:34:54.679050 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:34:54.679050 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:34:54.679050 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:34:54.679050 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 09:34:54.679050 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 09:34:54.679050 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 09:34:54.679050 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 09:34:54.679050 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 09:34:54.679050 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 09:34:54.679050 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 31 09:34:54.679050 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application Aug 31 09:34:54.685560 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 394/779] 10.222.0.13 () {56 vars in 1102 bytes} [Wed Aug 31 09:34:54 2022] GET /identity/v3/roles/service => generated 85 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 09:34:54.689553 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-934f8dac-ae02-41c0-a4b6-1426006af254 None admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:54.739592 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-360926a3-4c65-4a05-8e4b-8f3f84d55cb7 None admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:54.806293 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f1ea4bb-454a-4a8a-a416-4d192f3d5cf0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:54.807396 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1ea4bb-454a-4a8a-a416-4d192f3d5cf0 None admin] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:54.808098 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1ea4bb-454a-4a8a-a416-4d192f3d5cf0 None admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:54.808329 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1ea4bb-454a-4a8a-a416-4d192f3d5cf0 None admin] PATH_INFO: `/v3/services` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:54.809509 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f1ea4bb-454a-4a8a-a416-4d192f3d5cf0 None admin] RBAC: Adding query filter params (name=network) {{(pid=88114) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:34:54.810058 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f1ea4bb-454a-4a8a-a416-4d192f3d5cf0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:54.813377 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f1ea4bb-454a-4a8a-a416-4d192f3d5cf0 None admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:54.823348 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 386/780] 10.222.0.13 () {56 vars in 1112 bytes} [Wed Aug 31 09:34:54 2022] GET /identity/v3/services?name=network => generated 124 bytes in 361 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:34:54.828032 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-934f8dac-ae02-41c0-a4b6-1426006af254 None admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:54.832874 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13eefaa7-ecd5-4594-b52e-98f43275bcbe None None] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:54.833344 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13eefaa7-ecd5-4594-b52e-98f43275bcbe None None] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:54.833852 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13eefaa7-ecd5-4594-b52e-98f43275bcbe None None] PATH_INFO: `/` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:54.836755 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 387/781] 10.222.0.13 () {54 vars in 927 bytes} [Wed Aug 31 09:34:54 2022] GET /identity => generated 270 bytes in 13 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:34:54.838157 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13eefaa7-ecd5-4594-b52e-98f43275bcbe None None] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:54.886254 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f976a000-0d26-4089-a5e5-e4a2d0fa89f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:54.887299 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f976a000-0d26-4089-a5e5-e4a2d0fa89f8 None admin] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:54.887687 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f976a000-0d26-4089-a5e5-e4a2d0fa89f8 None admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:54.888087 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f976a000-0d26-4089-a5e5-e4a2d0fa89f8 None admin] PATH_INFO: `/v3/roles` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:54.889890 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f976a000-0d26-4089-a5e5-e4a2d0fa89f8 None admin] RBAC: Adding query filter params (name=service) {{(pid=88113) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:34:54.890421 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f976a000-0d26-4089-a5e5-e4a2d0fa89f8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:54.893210 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f976a000-0d26-4089-a5e5-e4a2d0fa89f8 None admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:54.915158 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 395/782] 10.222.0.13 () {56 vars in 1103 bytes} [Wed Aug 31 09:34:54 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 09:34:54.915158 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57376dd-64d4-41ec-86f5-d7946513d1e3 None None] REQUEST_METHOD: `POST` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:54.915158 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57376dd-64d4-41ec-86f5-d7946513d1e3 None None] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:54.915158 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57376dd-64d4-41ec-86f5-d7946513d1e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:54.933988 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=100 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:54.947320 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.auth.core [None req-e57376dd-64d4-41ec-86f5-d7946513d1e3 None None] MFA Rules not processed for user `fd836cf590a145df9acf19cf3ae2ac56`. Rule list: `[]` (Enabled: `True`). {{(pid=88113) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:34:54.949950 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.fernet_utils [None req-e57376dd-64d4-41ec-86f5-d7946513d1e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:34:54.995500 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13eefaa7-ecd5-4594-b52e-98f43275bcbe None None] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:55.028233 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=100 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:55.044296 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa59b1fc-a2c4-49c1-a0ec-17d57c84e4c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:55.044921 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa59b1fc-a2c4-49c1-a0ec-17d57c84e4c8 None admin] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:55.045204 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa59b1fc-a2c4-49c1-a0ec-17d57c84e4c8 None admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:55.046066 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa59b1fc-a2c4-49c1-a0ec-17d57c84e4c8 None admin] PATH_INFO: `/v3/services` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:55.046725 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa59b1fc-a2c4-49c1-a0ec-17d57c84e4c8 None admin] RBAC: Adding query filter params (type=network) {{(pid=88114) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:34:55.047046 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa59b1fc-a2c4-49c1-a0ec-17d57c84e4c8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:55.048696 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa59b1fc-a2c4-49c1-a0ec-17d57c84e4c8 None admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:55.061419 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 388/783] 10.222.0.13 () {56 vars in 1112 bytes} [Wed Aug 31 09:34:54 2022] GET /identity/v3/services?type=network => generated 350 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:34:55.061419 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa59b1fc-a2c4-49c1-a0ec-17d57c84e4c8 None admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:55.117684 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 396/784] 10.222.0.13 () {58 vars in 1024 bytes} [Wed Aug 31 09:34:54 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 09:34:55.120600 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e57376dd-64d4-41ec-86f5-d7946513d1e3 None None] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:55.201418 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa59b1fc-a2c4-49c1-a0ec-17d57c84e4c8 None admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:55.237860 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=100 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:55.282722 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68afe0be-804f-491c-837f-d2a683c1b7e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:55.283737 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68afe0be-804f-491c-837f-d2a683c1b7e9 None admin] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:55.283737 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68afe0be-804f-491c-837f-d2a683c1b7e9 None admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:55.284780 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68afe0be-804f-491c-837f-d2a683c1b7e9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:55.312520 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68afe0be-804f-491c-837f-d2a683c1b7e9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:55.314041 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68afe0be-804f-491c-837f-d2a683c1b7e9 None admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:55.330221 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e57376dd-64d4-41ec-86f5-d7946513d1e3 None None] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:55.363374 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application [None req-68afe0be-804f-491c-837f-d2a683c1b7e9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 09:34:55.363374 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:34:55.363374 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:34:55.363374 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:34:55.363374 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:34:55.363374 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:34:55.363374 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:34:55.363374 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:34:55.363374 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:34:55.363374 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:34:55.363374 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:34:55.363374 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:34:55.363374 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 09:34:55.363374 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 09:34:55.363374 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 09:34:55.363374 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 09:34:55.365204 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:34:55.365204 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:34:55.365204 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:34:55.365204 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:34:55.365204 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:34:55.365204 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:34:55.365204 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:34:55.365204 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application with Lock( Aug 31 09:34:55.365204 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:34:55.365204 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:34:55.365204 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:34:55.365204 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:34:55.365204 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:34:55.365204 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:34:55.365204 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:34:55.365204 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:34:55.365204 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 09:34:55.366959 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 09:34:55.366959 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 09:34:55.366959 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application Aug 31 09:34:55.371983 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0afe4d44-004e-47c2-9d17-08ad78e6f546 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:55.377571 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0afe4d44-004e-47c2-9d17-08ad78e6f546 None admin] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:55.377571 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0afe4d44-004e-47c2-9d17-08ad78e6f546 None admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:55.377571 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0afe4d44-004e-47c2-9d17-08ad78e6f546 None admin] PATH_INFO: `/v3/endpoints` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:55.378469 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0afe4d44-004e-47c2-9d17-08ad78e6f546 None admin] RBAC: Adding query filter params (interface=public, service_id=5892a4c746df45f29f8e5c16f47d5b43, region_id=RegionOne) {{(pid=88113) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:34:55.378814 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0afe4d44-004e-47c2-9d17-08ad78e6f546 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:55.379365 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 389/785] 10.222.0.13 () {56 vars in 1108 bytes} [Wed Aug 31 09:34:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 322 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:34:55.380369 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0afe4d44-004e-47c2-9d17-08ad78e6f546 None admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:55.386764 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68afe0be-804f-491c-837f-d2a683c1b7e9 None admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:55.390455 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 397/786] 10.222.0.13 () {56 vars in 1251 bytes} [Wed Aug 31 09:34:55 2022] GET /identity/v3/endpoints?service_id=5892a4c746df45f29f8e5c16f47d5b43&interface=public®ion_id=RegionOne => generated 194 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:34:55.396435 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0afe4d44-004e-47c2-9d17-08ad78e6f546 None admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:55.437618 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=100 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:55.518116 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0afe4d44-004e-47c2-9d17-08ad78e6f546 None admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:55.559000 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d6ee61-9434-4c54-872b-d93dfd93fe27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:55.560144 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d6ee61-9434-4c54-872b-d93dfd93fe27 None admin] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:55.560745 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d6ee61-9434-4c54-872b-d93dfd93fe27 None admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:55.561807 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d6ee61-9434-4c54-872b-d93dfd93fe27 None admin] PATH_INFO: `/v3/services` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:55.562763 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27d6ee61-9434-4c54-872b-d93dfd93fe27 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:55.564424 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27d6ee61-9434-4c54-872b-d93dfd93fe27 None admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:55.576650 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 398/787] 10.222.0.13 () {56 vars in 1087 bytes} [Wed Aug 31 09:34:55 2022] GET /identity/v3/services => generated 772 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:34:55.677282 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68afe0be-804f-491c-837f-d2a683c1b7e9 None admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:55.744926 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e95c3a11-7656-49e8-ac6f-f19e30e3bbf7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:55.745955 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95c3a11-7656-49e8-ac6f-f19e30e3bbf7 None admin] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:55.746471 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95c3a11-7656-49e8-ac6f-f19e30e3bbf7 None admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:55.746976 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95c3a11-7656-49e8-ac6f-f19e30e3bbf7 None admin] PATH_INFO: `/v3/domains` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:55.747851 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e95c3a11-7656-49e8-ac6f-f19e30e3bbf7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88114) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:34:55.748921 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e95c3a11-7656-49e8-ac6f-f19e30e3bbf7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:55.750881 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e95c3a11-7656-49e8-ac6f-f19e30e3bbf7 None admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:55.769614 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 390/788] 10.222.0.13 () {56 vars in 1109 bytes} [Wed Aug 31 09:34:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 384 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:34:55.775412 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d6ee61-9434-4c54-872b-d93dfd93fe27 None admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:55.900512 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=100 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:55.906923 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d6ee61-9434-4c54-872b-d93dfd93fe27 None admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:55.918726 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e95c3a11-7656-49e8-ac6f-f19e30e3bbf7 None admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:55.972082 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27963700-d48d-467e-9d5c-f8d94acc312c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:55.974244 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27963700-d48d-467e-9d5c-f8d94acc312c None admin] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:55.974704 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27963700-d48d-467e-9d5c-f8d94acc312c None admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:55.975757 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27963700-d48d-467e-9d5c-f8d94acc312c None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:55.988967 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27963700-d48d-467e-9d5c-f8d94acc312c None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:55.994068 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27963700-d48d-467e-9d5c-f8d94acc312c None admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:56.001540 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=100 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:56.026391 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application [None req-27963700-d48d-467e-9d5c-f8d94acc312c None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 31 09:34:56.026391 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:34:56.026391 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:34:56.026391 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:34:56.026391 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:34:56.026391 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:34:56.026391 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:34:56.026391 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:34:56.026391 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:34:56.026391 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:34:56.026391 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:34:56.026391 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:34:56.026391 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 31 09:34:56.026391 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 31 09:34:56.026391 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 31 09:34:56.026391 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 31 09:34:56.028382 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:34:56.028382 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:34:56.028382 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 31 09:34:56.028382 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 09:34:56.028382 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 31 09:34:56.028382 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 09:34:56.028382 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:34:56.028382 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:34:56.028382 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:34:56.028382 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:34:56.028382 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:34:56.028382 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application with Lock( Aug 31 09:34:56.028382 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:34:56.028382 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:34:56.028382 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:34:56.028382 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:34:56.028382 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:34:56.028382 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:34:56.031894 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:34:56.031894 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:34:56.031894 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 31 09:34:56.031894 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 31 09:34:56.031894 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 31 09:34:56.031894 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 31 09:34:56.031894 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 31 09:34:56.031894 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 31 09:34:56.031894 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 31 09:34:56.031894 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application Aug 31 09:34:56.037430 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 399/789] 10.222.0.13 () {56 vars in 1117 bytes} [Wed Aug 31 09:34:55 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 253 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 09:34:56.037430 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27963700-d48d-467e-9d5c-f8d94acc312c None admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:56.242706 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e95c3a11-7656-49e8-ac6f-f19e30e3bbf7 None admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:56.278592 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27963700-d48d-467e-9d5c-f8d94acc312c None admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:56.319342 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-625d290a-52f4-4a70-9eeb-c74662de06ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:56.319969 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625d290a-52f4-4a70-9eeb-c74662de06ae None admin] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:56.320259 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625d290a-52f4-4a70-9eeb-c74662de06ae None admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:56.320518 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625d290a-52f4-4a70-9eeb-c74662de06ae None admin] PATH_INFO: `/v3/users` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:56.321248 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-625d290a-52f4-4a70-9eeb-c74662de06ae None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=88113) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:34:56.321579 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-625d290a-52f4-4a70-9eeb-c74662de06ae None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:56.323063 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-625d290a-52f4-4a70-9eeb-c74662de06ae None admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:56.343546 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6200f78-d692-478f-920d-198f394a86fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:56.344369 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6200f78-d692-478f-920d-198f394a86fd None admin] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:56.344498 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6200f78-d692-478f-920d-198f394a86fd None admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:56.344936 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6200f78-d692-478f-920d-198f394a86fd None admin] PATH_INFO: `/v3/roles/admin` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:56.348700 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 400/790] 10.222.0.13 () {56 vars in 1149 bytes} [Wed Aug 31 09:34:56 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 311 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:34:56.351183 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-625d290a-52f4-4a70-9eeb-c74662de06ae None admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:56.373225 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=100 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:56.374591 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6200f78-d692-478f-920d-198f394a86fd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 09:34:56.379666 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application [None req-e6200f78-d692-478f-920d-198f394a86fd None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 09:34:56.379666 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:34:56.379666 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:34:56.379666 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:34:56.379666 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:34:56.379666 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:34:56.379666 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:34:56.379666 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:34:56.379666 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:34:56.379666 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:34:56.379666 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:34:56.379666 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:34:56.379666 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 09:34:56.379666 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 09:34:56.379666 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 09:34:56.379666 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 09:34:56.381557 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 09:34:56.381557 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 09:34:56.381557 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 09:34:56.381557 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 09:34:56.381557 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:34:56.381557 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:34:56.381557 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:34:56.381557 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:34:56.381557 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:34:56.381557 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:34:56.381557 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:34:56.381557 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application with Lock( Aug 31 09:34:56.381557 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:34:56.381557 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:34:56.381557 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:34:56.381557 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:34:56.383191 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:34:56.383191 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:34:56.383191 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:34:56.383191 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:34:56.383191 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 09:34:56.383191 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 09:34:56.383191 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 09:34:56.383191 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 09:34:56.383191 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 09:34:56.383191 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 09:34:56.383191 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 09:34:56.383191 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application Aug 31 09:34:56.383191 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 391/791] 10.222.0.13 () {56 vars in 1096 bytes} [Wed Aug 31 09:34:55 2022] GET /identity/v3/roles/admin => generated 83 bytes in 463 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:34:56.399053 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6200f78-d692-478f-920d-198f394a86fd None admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:56.491648 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-625d290a-52f4-4a70-9eeb-c74662de06ae None admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:56.542755 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9534c593-f91e-4d80-9f24-758b6b0d1dbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:56.543319 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9534c593-f91e-4d80-9f24-758b6b0d1dbd None admin] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:56.543597 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9534c593-f91e-4d80-9f24-758b6b0d1dbd None admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:56.543950 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9534c593-f91e-4d80-9f24-758b6b0d1dbd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:56.558014 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9534c593-f91e-4d80-9f24-758b6b0d1dbd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:56.559538 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9534c593-f91e-4d80-9f24-758b6b0d1dbd None admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:56.577213 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application [None req-9534c593-f91e-4d80-9f24-758b6b0d1dbd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 09:34:56.577213 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:34:56.577213 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:34:56.577213 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:34:56.577213 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:34:56.577213 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:34:56.577213 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:34:56.577213 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:34:56.577213 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:34:56.577213 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:34:56.577213 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:34:56.577213 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:34:56.577213 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 09:34:56.577213 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 09:34:56.577213 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 09:34:56.577213 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 09:34:56.580439 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:34:56.580439 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:34:56.580439 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:34:56.580439 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:34:56.580439 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:34:56.580439 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:34:56.580439 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:34:56.580439 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application with Lock( Aug 31 09:34:56.580439 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:34:56.580439 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:34:56.580439 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:34:56.580439 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:34:56.580439 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:34:56.580439 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:34:56.580439 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:34:56.580439 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:34:56.580439 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 09:34:56.583545 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 09:34:56.583545 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 09:34:56.583545 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application Aug 31 09:34:56.583545 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 401/792] 10.222.0.13 () {56 vars in 1108 bytes} [Wed Aug 31 09:34:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 09:34:56.592237 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f05c992-2af9-4e68-afa1-1c0f090b5714 None None] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:56.592237 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f05c992-2af9-4e68-afa1-1c0f090b5714 None None] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:56.592237 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f05c992-2af9-4e68-afa1-1c0f090b5714 None None] PATH_INFO: `/` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:56.592237 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 402/793] 10.222.0.13 () {54 vars in 927 bytes} [Wed Aug 31 09:34:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:34:56.592237 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f05c992-2af9-4e68-afa1-1c0f090b5714 None None] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:56.686882 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=100 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:56.752846 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f05c992-2af9-4e68-afa1-1c0f090b5714 None None] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:56.785126 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=100 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:56.808552 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6200f78-d692-478f-920d-198f394a86fd None admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:56.818050 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16bb4cf3-9a93-428d-8f98-1c2988f3f15f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:56.818685 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16bb4cf3-9a93-428d-8f98-1c2988f3f15f None admin] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:56.818963 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16bb4cf3-9a93-428d-8f98-1c2988f3f15f None admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:56.819232 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16bb4cf3-9a93-428d-8f98-1c2988f3f15f None admin] PATH_INFO: `/v3/domains` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:56.819909 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16bb4cf3-9a93-428d-8f98-1c2988f3f15f None admin] RBAC: Adding query filter params (name=Default) {{(pid=88113) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:34:56.820210 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16bb4cf3-9a93-428d-8f98-1c2988f3f15f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:56.821729 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16bb4cf3-9a93-428d-8f98-1c2988f3f15f None admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:56.845544 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 403/794] 10.222.0.13 () {56 vars in 1109 bytes} [Wed Aug 31 09:34:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:34:56.845544 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf94599-b5f1-436e-86fc-410cb8933d3c None None] REQUEST_METHOD: `POST` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:56.845544 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf94599-b5f1-436e-86fc-410cb8933d3c None None] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:56.846137 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf94599-b5f1-436e-86fc-410cb8933d3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:56.868835 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.auth.core [None req-8cf94599-b5f1-436e-86fc-410cb8933d3c None None] MFA Rules not processed for user `fd836cf590a145df9acf19cf3ae2ac56`. Rule list: `[]` (Enabled: `True`). {{(pid=88113) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:34:56.870115 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.fernet_utils [None req-8cf94599-b5f1-436e-86fc-410cb8933d3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:34:56.896658 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f458044-baca-421b-b84b-82f5510b6641 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:56.897289 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f458044-baca-421b-b84b-82f5510b6641 None admin] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:56.897874 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f458044-baca-421b-b84b-82f5510b6641 None admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:56.898139 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f458044-baca-421b-b84b-82f5510b6641 None admin] PATH_INFO: `/v3/roles` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:56.898823 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f458044-baca-421b-b84b-82f5510b6641 None admin] RBAC: Adding query filter params (name=admin) {{(pid=88114) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:34:56.899302 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f458044-baca-421b-b84b-82f5510b6641 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:56.901093 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f458044-baca-421b-b84b-82f5510b6641 None admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:56.917416 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 392/795] 10.222.0.13 () {56 vars in 1099 bytes} [Wed Aug 31 09:34:56 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 513 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:34:56.917416 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f458044-baca-421b-b84b-82f5510b6641 None admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:56.960446 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 404/796] 10.222.0.13 () {58 vars in 1024 bytes} [Wed Aug 31 09:34:56 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 09:34:56.966083 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19bb9f8-b304-4ed7-99f5-85c574043c9f None None] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:56.966083 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19bb9f8-b304-4ed7-99f5-85c574043c9f None None] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:56.966083 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19bb9f8-b304-4ed7-99f5-85c574043c9f None None] PATH_INFO: `/` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:56.966083 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 405/797] 10.222.0.13 () {54 vars in 927 bytes} [Wed Aug 31 09:34:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:34:56.966083 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b19bb9f8-b304-4ed7-99f5-85c574043c9f None None] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:57.065513 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=100 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:57.090423 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f458044-baca-421b-b84b-82f5510b6641 None admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:57.130082 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc3590e-66fb-4ea6-8c15-766243a7633a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:57.130984 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc3590e-66fb-4ea6-8c15-766243a7633a None admin] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:57.131411 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc3590e-66fb-4ea6-8c15-766243a7633a None admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:57.131826 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc3590e-66fb-4ea6-8c15-766243a7633a None admin] PATH_INFO: `/v3/projects/service` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:57.140408 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccc3590e-66fb-4ea6-8c15-766243a7633a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:57.142317 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccc3590e-66fb-4ea6-8c15-766243a7633a None admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:57.156026 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application [None req-ccc3590e-66fb-4ea6-8c15-766243a7633a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 09:34:57.156026 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:34:57.156026 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:34:57.156026 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:34:57.156026 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:34:57.156026 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:34:57.156026 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:34:57.156026 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:34:57.156026 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:34:57.156026 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:34:57.156026 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:34:57.156026 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:34:57.156026 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 31 09:34:57.156026 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 31 09:34:57.156026 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 31 09:34:57.156990 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 31 09:34:57.156990 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:34:57.156990 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:34:57.156990 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:34:57.156990 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:34:57.156990 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:34:57.156990 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:34:57.156990 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:34:57.156990 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application with Lock( Aug 31 09:34:57.156990 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:34:57.156990 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:34:57.156990 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:34:57.156990 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:34:57.156990 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:34:57.156990 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:34:57.156990 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:34:57.156990 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:34:57.157949 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 31 09:34:57.157949 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 31 09:34:57.157949 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 31 09:34:57.157949 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 31 09:34:57.157949 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 31 09:34:57.157949 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 31 09:34:57.157949 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 09:34:57.157949 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application Aug 31 09:34:57.158389 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b19bb9f8-b304-4ed7-99f5-85c574043c9f None None] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:57.166126 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 393/798] 10.222.0.13 () {56 vars in 1111 bytes} [Wed Aug 31 09:34:56 2022] GET /identity/v3/projects/service => generated 88 bytes in 254 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:34:57.167908 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb5067f-40e6-40df-a8a8-b1ca5faad9c0 None None] REQUEST_METHOD: `POST` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:57.168425 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb5067f-40e6-40df-a8a8-b1ca5faad9c0 None None] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:57.168928 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb5067f-40e6-40df-a8a8-b1ca5faad9c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:57.189274 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.auth.core [None req-afb5067f-40e6-40df-a8a8-b1ca5faad9c0 None None] MFA Rules not processed for user `fd836cf590a145df9acf19cf3ae2ac56`. Rule list: `[]` (Enabled: `True`). {{(pid=88114) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:34:57.190921 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.fernet_utils [None req-afb5067f-40e6-40df-a8a8-b1ca5faad9c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88114) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:34:57.218507 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=100 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:57.221908 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4919e7c5-7c12-48e1-8699-84ef18a2effc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:57.222942 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4919e7c5-7c12-48e1-8699-84ef18a2effc None admin] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:57.223435 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4919e7c5-7c12-48e1-8699-84ef18a2effc None admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:57.223916 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4919e7c5-7c12-48e1-8699-84ef18a2effc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:57.237259 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4919e7c5-7c12-48e1-8699-84ef18a2effc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:57.239217 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4919e7c5-7c12-48e1-8699-84ef18a2effc None admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:57.256721 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application [None req-4919e7c5-7c12-48e1-8699-84ef18a2effc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 09:34:57.256721 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:34:57.256721 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:34:57.256721 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:34:57.256721 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:34:57.256721 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:34:57.256721 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:34:57.256721 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:34:57.256721 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:34:57.256721 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:34:57.256721 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:34:57.256721 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:34:57.256721 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 09:34:57.256721 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 09:34:57.256721 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 09:34:57.256721 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 09:34:57.257774 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:34:57.257774 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:34:57.257774 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:34:57.257774 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:34:57.257774 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:34:57.257774 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:34:57.257774 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:34:57.257774 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application with Lock( Aug 31 09:34:57.257774 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:34:57.257774 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:34:57.257774 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:34:57.257774 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:34:57.257774 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:34:57.257774 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:34:57.257774 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:34:57.257774 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:34:57.257774 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 09:34:57.258757 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 09:34:57.258757 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 09:34:57.258757 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application Aug 31 09:34:57.261876 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 406/799] 10.222.0.13 () {56 vars in 1108 bytes} [Wed Aug 31 09:34:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 297 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:34:57.263159 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4919e7c5-7c12-48e1-8699-84ef18a2effc None admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:57.330424 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 394/800] 10.222.0.13 () {58 vars in 1024 bytes} [Wed Aug 31 09:34:57 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 09:34:57.331754 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb5067f-40e6-40df-a8a8-b1ca5faad9c0 None None] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:57.465494 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4919e7c5-7c12-48e1-8699-84ef18a2effc None admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:57.486118 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb5067f-40e6-40df-a8a8-b1ca5faad9c0 None None] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:57.492224 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=100 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:57.525147 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae481c7d-0bf1-4a77-a8f8-36ed89182416 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:57.525810 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae481c7d-0bf1-4a77-a8f8-36ed89182416 None admin] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:57.526084 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae481c7d-0bf1-4a77-a8f8-36ed89182416 None admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:57.526348 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae481c7d-0bf1-4a77-a8f8-36ed89182416 None admin] PATH_INFO: `/v3/domains` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:57.527029 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae481c7d-0bf1-4a77-a8f8-36ed89182416 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88114) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:34:57.527326 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae481c7d-0bf1-4a77-a8f8-36ed89182416 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:57.528876 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae481c7d-0bf1-4a77-a8f8-36ed89182416 None admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:57.534121 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdac50ab-3d9e-413c-bd50-26519b778fd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:57.535043 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdac50ab-3d9e-413c-bd50-26519b778fd2 None admin] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:57.535502 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdac50ab-3d9e-413c-bd50-26519b778fd2 None admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:57.535978 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdac50ab-3d9e-413c-bd50-26519b778fd2 None admin] PATH_INFO: `/v3/projects` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:57.537499 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdac50ab-3d9e-413c-bd50-26519b778fd2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88113) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:34:57.538043 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdac50ab-3d9e-413c-bd50-26519b778fd2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:57.540984 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdac50ab-3d9e-413c-bd50-26519b778fd2 None admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:57.542467 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 395/801] 10.222.0.13 () {56 vars in 1109 bytes} [Wed Aug 31 09:34:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:34:57.543339 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae481c7d-0bf1-4a77-a8f8-36ed89182416 None admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:57.554478 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=100 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:57.577889 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 407/802] 10.222.0.13 () {56 vars in 1148 bytes} [Wed Aug 31 09:34:57 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 315 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:34:57.579463 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdac50ab-3d9e-413c-bd50-26519b778fd2 None admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:57.656538 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae481c7d-0bf1-4a77-a8f8-36ed89182416 None admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:57.694040 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3cbcc73-9a1b-4e72-a38c-6f1728caf62d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:57.694677 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3cbcc73-9a1b-4e72-a38c-6f1728caf62d None admin] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:57.694947 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3cbcc73-9a1b-4e72-a38c-6f1728caf62d None admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:57.695216 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3cbcc73-9a1b-4e72-a38c-6f1728caf62d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:57.706962 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3cbcc73-9a1b-4e72-a38c-6f1728caf62d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 09:34:57.712267 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application [None req-d3cbcc73-9a1b-4e72-a38c-6f1728caf62d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 09:34:57.712267 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:34:57.712267 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:34:57.712267 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:34:57.712267 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:34:57.712267 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:34:57.712267 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:34:57.712267 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:34:57.712267 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:34:57.712267 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:34:57.712267 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:34:57.712267 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:34:57.712267 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 09:34:57.712267 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 09:34:57.712267 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 09:34:57.712267 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 09:34:57.713280 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 09:34:57.713280 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 09:34:57.713280 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 09:34:57.713280 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 09:34:57.713280 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:34:57.713280 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:34:57.713280 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:34:57.713280 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:34:57.713280 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:34:57.713280 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:34:57.713280 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:34:57.713280 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application with Lock( Aug 31 09:34:57.713280 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:34:57.713280 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:34:57.713280 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:34:57.713280 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:34:57.715587 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:34:57.715587 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:34:57.715587 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:34:57.715587 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:34:57.715587 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 09:34:57.715587 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 09:34:57.715587 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 09:34:57.715587 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 09:34:57.715587 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 09:34:57.715587 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 09:34:57.715587 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 09:34:57.715587 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application Aug 31 09:34:57.715587 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 396/803] 10.222.0.13 () {56 vars in 1096 bytes} [Wed Aug 31 09:34:57 2022] GET /identity/v3/roles/admin => generated 83 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:34:57.715587 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3cbcc73-9a1b-4e72-a38c-6f1728caf62d None admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:57.771348 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdac50ab-3d9e-413c-bd50-26519b778fd2 None admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:57.817969 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=100 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:57.827724 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36bc345e-7286-4f93-abe0-dff8cf20839d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:57.828530 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36bc345e-7286-4f93-abe0-dff8cf20839d None admin] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:57.828965 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36bc345e-7286-4f93-abe0-dff8cf20839d None admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:57.829425 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36bc345e-7286-4f93-abe0-dff8cf20839d None admin] PATH_INFO: `/v3/users/nova` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:57.845205 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36bc345e-7286-4f93-abe0-dff8cf20839d None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:57.848001 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36bc345e-7286-4f93-abe0-dff8cf20839d None admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:57.869541 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application [None req-36bc345e-7286-4f93-abe0-dff8cf20839d None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 31 09:34:57.869541 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:34:57.869541 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:34:57.869541 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:34:57.869541 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:34:57.869541 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:34:57.869541 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:34:57.869541 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:34:57.869541 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:34:57.869541 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:34:57.869541 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:34:57.869541 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:34:57.869541 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 31 09:34:57.869541 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 31 09:34:57.869541 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 31 09:34:57.869541 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 31 09:34:57.870569 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:34:57.870569 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:34:57.870569 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 31 09:34:57.870569 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 09:34:57.870569 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 31 09:34:57.870569 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 09:34:57.870569 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:34:57.870569 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:34:57.870569 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:34:57.870569 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:34:57.870569 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:34:57.870569 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application with Lock( Aug 31 09:34:57.870569 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:34:57.870569 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:34:57.870569 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:34:57.870569 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:34:57.870569 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:34:57.870569 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:34:57.871546 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:34:57.871546 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:34:57.871546 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 31 09:34:57.871546 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 31 09:34:57.871546 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 31 09:34:57.871546 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 31 09:34:57.871546 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 31 09:34:57.871546 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 31 09:34:57.871546 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 31 09:34:57.871546 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application Aug 31 09:34:57.871546 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 408/804] 10.222.0.13 () {56 vars in 1093 bytes} [Wed Aug 31 09:34:57 2022] GET /identity/v3/users/nova => generated 82 bytes in 292 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:34:57.874311 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36bc345e-7286-4f93-abe0-dff8cf20839d None admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:57.894919 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3cbcc73-9a1b-4e72-a38c-6f1728caf62d None admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:57.913604 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=100 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:57.949953 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d9d16a-49f5-4bc1-869e-3548926b3891 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:57.951119 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d9d16a-49f5-4bc1-869e-3548926b3891 None admin] REQUEST_METHOD: `PUT` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:57.952183 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d9d16a-49f5-4bc1-869e-3548926b3891 None admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:57.952759 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d9d16a-49f5-4bc1-869e-3548926b3891 None admin] PATH_INFO: `/v3/projects/49548eb44d62415582af820ab66decee/users/768cb67c1ef24251aaa113c12d81a250/roles/e24ba4fd91584883af350cc84bc96c1d` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:57.978324 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8d9d16a-49f5-4bc1-869e-3548926b3891 None admin] RBAC: Authorizing `identity:create_grant(project_id=49548eb44d62415582af820ab66decee, user_id=768cb67c1ef24251aaa113c12d81a250, role_id=e24ba4fd91584883af350cc84bc96c1d)` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:57.979773 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8d9d16a-49f5-4bc1-869e-3548926b3891 None admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:57.997045 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36bc345e-7286-4f93-abe0-dff8cf20839d None admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:57.998957 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 397/805] 10.222.0.13 () {58 vars in 1439 bytes} [Wed Aug 31 09:34:57 2022] PUT /identity/v3/projects/49548eb44d62415582af820ab66decee/users/768cb67c1ef24251aaa113c12d81a250/roles/e24ba4fd91584883af350cc84bc96c1d => generated 0 bytes in 284 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:34:57.999540 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d9d16a-49f5-4bc1-869e-3548926b3891 None admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:58.115791 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b06d64-44bf-4f55-9de4-4ddc7e7402b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:58.118385 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b06d64-44bf-4f55-9de4-4ddc7e7402b9 None admin] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:58.119508 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b06d64-44bf-4f55-9de4-4ddc7e7402b9 None admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:58.121081 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b06d64-44bf-4f55-9de4-4ddc7e7402b9 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:58.122770 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41b06d64-44bf-4f55-9de4-4ddc7e7402b9 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:58.127892 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41b06d64-44bf-4f55-9de4-4ddc7e7402b9 None admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:58.146060 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d9d16a-49f5-4bc1-869e-3548926b3891 None admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:58.151886 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application [None req-41b06d64-44bf-4f55-9de4-4ddc7e7402b9 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 31 09:34:58.151886 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:34:58.151886 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 31 09:34:58.151886 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 31 09:34:58.151886 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 31 09:34:58.151886 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 31 09:34:58.151886 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 31 09:34:58.151886 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 09:34:58.151886 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 31 09:34:58.151886 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application Aug 31 09:34:58.151886 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 31 09:34:58.151886 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application Aug 31 09:34:58.151886 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:34:58.151886 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:34:58.151886 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:34:58.151886 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:34:58.151886 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:34:58.153939 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:34:58.153939 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:34:58.153939 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:34:58.153939 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:34:58.153939 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:34:58.153939 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:34:58.153939 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 31 09:34:58.153939 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 31 09:34:58.153939 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 31 09:34:58.153939 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 31 09:34:58.153939 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:34:58.153939 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:34:58.153939 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:34:58.153939 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:34:58.153939 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:34:58.153939 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:34:58.156875 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:34:58.156875 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application with Lock( Aug 31 09:34:58.156875 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:34:58.156875 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:34:58.156875 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:34:58.156875 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:34:58.156875 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:34:58.156875 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:34:58.156875 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:34:58.156875 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:34:58.156875 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 31 09:34:58.156875 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 09:34:58.156875 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 31 09:34:58.156875 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application Aug 31 09:34:58.160211 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 409/806] 10.222.0.13 () {56 vars in 1129 bytes} [Wed Aug 31 09:34:57 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 282 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:34:58.160211 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b06d64-44bf-4f55-9de4-4ddc7e7402b9 None admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:58.179112 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:58.209967 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8edd1e57-2924-456b-94e1-aaaab614d58b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:58.210588 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8edd1e57-2924-456b-94e1-aaaab614d58b None admin] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:58.210867 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8edd1e57-2924-456b-94e1-aaaab614d58b None admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:58.211136 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8edd1e57-2924-456b-94e1-aaaab614d58b None admin] PATH_INFO: `/v3/roles` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:58.212436 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8edd1e57-2924-456b-94e1-aaaab614d58b None admin] RBAC: Adding query filter params (name=admin) {{(pid=88114) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:34:58.212436 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8edd1e57-2924-456b-94e1-aaaab614d58b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:58.213496 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8edd1e57-2924-456b-94e1-aaaab614d58b None admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:58.228713 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 398/807] 10.222.0.13 () {56 vars in 1099 bytes} [Wed Aug 31 09:34:57 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:34:58.228713 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8edd1e57-2924-456b-94e1-aaaab614d58b None admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:58.299286 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=100 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:58.306533 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b06d64-44bf-4f55-9de4-4ddc7e7402b9 None admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:58.352409 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1221856-35bd-4870-9018-36c40f78e5b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:58.353339 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1221856-35bd-4870-9018-36c40f78e5b1 None admin] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:58.353843 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1221856-35bd-4870-9018-36c40f78e5b1 None admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:58.354284 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1221856-35bd-4870-9018-36c40f78e5b1 None admin] PATH_INFO: `/v3/users` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:58.355306 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1221856-35bd-4870-9018-36c40f78e5b1 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=88113) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:34:58.355821 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1221856-35bd-4870-9018-36c40f78e5b1 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:58.357605 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1221856-35bd-4870-9018-36c40f78e5b1 None admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:58.383703 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 410/808] 10.222.0.13 () {56 vars in 1133 bytes} [Wed Aug 31 09:34:58 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:34:58.383703 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1221856-35bd-4870-9018-36c40f78e5b1 None admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:58.403376 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8edd1e57-2924-456b-94e1-aaaab614d58b None admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:58.452382 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc259fb0-fca2-48d6-bd41-bb2cb06195bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:58.453897 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc259fb0-fca2-48d6-bd41-bb2cb06195bb None admin] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:58.454393 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc259fb0-fca2-48d6-bd41-bb2cb06195bb None admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:58.454864 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc259fb0-fca2-48d6-bd41-bb2cb06195bb None admin] PATH_INFO: `/v3/services` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:58.455769 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc259fb0-fca2-48d6-bd41-bb2cb06195bb None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=88114) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:34:58.456735 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc259fb0-fca2-48d6-bd41-bb2cb06195bb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:58.459034 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc259fb0-fca2-48d6-bd41-bb2cb06195bb None admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:58.471296 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 399/809] 10.222.0.13 () {56 vars in 1124 bytes} [Wed Aug 31 09:34:58 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:34:58.471296 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc259fb0-fca2-48d6-bd41-bb2cb06195bb None admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:58.535927 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=100 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:58.568121 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1221856-35bd-4870-9018-36c40f78e5b1 None admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:58.609765 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a8cc7a-fb77-4316-a015-38705f3e602a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:58.610820 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a8cc7a-fb77-4316-a015-38705f3e602a None admin] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:58.611374 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a8cc7a-fb77-4316-a015-38705f3e602a None admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:58.611843 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a8cc7a-fb77-4316-a015-38705f3e602a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:58.624142 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81a8cc7a-fb77-4316-a015-38705f3e602a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:58.626459 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81a8cc7a-fb77-4316-a015-38705f3e602a None admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:58.645365 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application [None req-81a8cc7a-fb77-4316-a015-38705f3e602a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 09:34:58.645365 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:34:58.645365 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:34:58.645365 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:34:58.645365 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:34:58.645365 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:34:58.645365 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:34:58.645365 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:34:58.645365 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:34:58.645365 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:34:58.645365 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:34:58.645365 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:34:58.645365 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 09:34:58.645365 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 09:34:58.645365 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 09:34:58.645365 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 09:34:58.646395 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:34:58.646395 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:34:58.646395 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:34:58.646395 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:34:58.646395 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:34:58.646395 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:34:58.646395 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:34:58.646395 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application with Lock( Aug 31 09:34:58.646395 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:34:58.646395 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:34:58.646395 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:34:58.646395 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:34:58.646395 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:34:58.646395 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:34:58.646395 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:34:58.646395 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:34:58.646395 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 09:34:58.647313 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 09:34:58.647313 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 09:34:58.647313 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application Aug 31 09:34:58.649277 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc259fb0-fca2-48d6-bd41-bb2cb06195bb None admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:58.655352 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 411/810] 10.222.0.13 () {56 vars in 1108 bytes} [Wed Aug 31 09:34:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 275 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:34:58.656551 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a8cc7a-fb77-4316-a015-38705f3e602a None admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:58.671992 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=100 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:58.711914 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba4b5e87-7443-4563-9ad3-d5e506e94d32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:58.713011 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4b5e87-7443-4563-9ad3-d5e506e94d32 None admin] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:58.713592 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4b5e87-7443-4563-9ad3-d5e506e94d32 None admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:58.714103 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4b5e87-7443-4563-9ad3-d5e506e94d32 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:58.812050 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba4b5e87-7443-4563-9ad3-d5e506e94d32 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:58.815476 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba4b5e87-7443-4563-9ad3-d5e506e94d32 None admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:58.841568 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a8cc7a-fb77-4316-a015-38705f3e602a None admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:58.889893 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=100 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:58.900885 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c743eab-a428-4d5f-9791-345b1ed277e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:58.902096 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c743eab-a428-4d5f-9791-345b1ed277e8 None admin] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:58.903043 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c743eab-a428-4d5f-9791-345b1ed277e8 None admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:58.903516 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c743eab-a428-4d5f-9791-345b1ed277e8 None admin] PATH_INFO: `/v3/services` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:58.909611 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c743eab-a428-4d5f-9791-345b1ed277e8 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=88113) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:34:58.910264 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c743eab-a428-4d5f-9791-345b1ed277e8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:58.912863 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c743eab-a428-4d5f-9791-345b1ed277e8 None admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:58.927739 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 412/811] 10.222.0.13 () {56 vars in 1124 bytes} [Wed Aug 31 09:34:58 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:34:58.927739 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c743eab-a428-4d5f-9791-345b1ed277e8 None admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:58.995079 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 400/812] 10.222.0.13 () {58 vars in 1282 bytes} [Wed Aug 31 09:34:58 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 521 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 31 09:34:58.995079 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19cf4ded-1f4b-4e94-932a-51d1dd26a6ba None None] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:58.995079 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19cf4ded-1f4b-4e94-932a-51d1dd26a6ba None None] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:58.995079 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19cf4ded-1f4b-4e94-932a-51d1dd26a6ba None None] PATH_INFO: `/` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:58.995079 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 401/813] 10.222.0.13 () {54 vars in 927 bytes} [Wed Aug 31 09:34:58 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:34:58.995079 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19cf4ded-1f4b-4e94-932a-51d1dd26a6ba None None] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:59.066979 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=100 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:59.126155 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c743eab-a428-4d5f-9791-345b1ed277e8 None admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:59.165550 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19cf4ded-1f4b-4e94-932a-51d1dd26a6ba None None] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:59.189150 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-724b1747-6b4c-4540-9a01-bfdcd2bc87df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:59.190233 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724b1747-6b4c-4540-9a01-bfdcd2bc87df None admin] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:59.190703 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724b1747-6b4c-4540-9a01-bfdcd2bc87df None admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:59.191191 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724b1747-6b4c-4540-9a01-bfdcd2bc87df None admin] PATH_INFO: `/v3/domains` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:59.192375 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-724b1747-6b4c-4540-9a01-bfdcd2bc87df None admin] RBAC: Adding query filter params (name=Default) {{(pid=88113) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:34:59.192918 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-724b1747-6b4c-4540-9a01-bfdcd2bc87df None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:59.195138 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-724b1747-6b4c-4540-9a01-bfdcd2bc87df None admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:59.209342 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83fea86b-e691-45a3-ab92-3b3d9a18c311 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:59.210119 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83fea86b-e691-45a3-ab92-3b3d9a18c311 None admin] REQUEST_METHOD: `POST` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:59.210491 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83fea86b-e691-45a3-ab92-3b3d9a18c311 None admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:59.210857 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83fea86b-e691-45a3-ab92-3b3d9a18c311 None admin] PATH_INFO: `/v3/endpoints` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:59.211637 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83fea86b-e691-45a3-ab92-3b3d9a18c311 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:59.213133 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83fea86b-e691-45a3-ab92-3b3d9a18c311 None admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:59.224544 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 413/814] 10.222.0.13 () {56 vars in 1109 bytes} [Wed Aug 31 09:34:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 295 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:34:59.224544 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-724b1747-6b4c-4540-9a01-bfdcd2bc87df None admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:59.233737 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:59.250714 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 402/815] 10.222.0.13 () {60 vars in 1144 bytes} [Wed Aug 31 09:34:58 2022] POST /identity/v3/endpoints => generated 346 bytes in 257 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 09:34:59.253142 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5aab03-63bb-46a6-bac3-4c3385408c38 None None] REQUEST_METHOD: `POST` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:59.253865 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5aab03-63bb-46a6-bac3-4c3385408c38 None None] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:59.254485 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5aab03-63bb-46a6-bac3-4c3385408c38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:59.288001 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.auth.core [None req-bd5aab03-63bb-46a6-bac3-4c3385408c38 None None] MFA Rules not processed for user `fd836cf590a145df9acf19cf3ae2ac56`. Rule list: `[]` (Enabled: `True`). {{(pid=88114) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:34:59.290761 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.fernet_utils [None req-bd5aab03-63bb-46a6-bac3-4c3385408c38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88114) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:34:59.440642 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-724b1747-6b4c-4540-9a01-bfdcd2bc87df None admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:59.461975 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 403/816] 10.222.0.13 () {58 vars in 1024 bytes} [Wed Aug 31 09:34:59 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 09:34:59.463886 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd5aab03-63bb-46a6-bac3-4c3385408c38 None None] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:59.506159 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=100 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:59.533152 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc4c9c76-1724-4b7f-89f7-3da7fbd67fb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:59.534320 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4c9c76-1724-4b7f-89f7-3da7fbd67fb3 None admin] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:59.534815 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4c9c76-1724-4b7f-89f7-3da7fbd67fb3 None admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:59.535305 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4c9c76-1724-4b7f-89f7-3da7fbd67fb3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:59.574552 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc4c9c76-1724-4b7f-89f7-3da7fbd67fb3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:59.578865 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc4c9c76-1724-4b7f-89f7-3da7fbd67fb3 None admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:59.618602 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application [None req-fc4c9c76-1724-4b7f-89f7-3da7fbd67fb3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 09:34:59.618602 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:34:59.618602 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:34:59.618602 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:34:59.618602 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:34:59.618602 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:34:59.618602 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:34:59.618602 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:34:59.618602 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:34:59.618602 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:34:59.618602 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:34:59.618602 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:34:59.618602 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 09:34:59.618602 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 09:34:59.618602 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 09:34:59.618602 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 09:34:59.620417 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:34:59.620417 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:34:59.620417 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:34:59.620417 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:34:59.620417 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:34:59.620417 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:34:59.620417 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:34:59.620417 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application with Lock( Aug 31 09:34:59.620417 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:34:59.620417 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:34:59.620417 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:34:59.620417 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:34:59.620417 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:34:59.620417 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:34:59.620417 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:34:59.620417 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:34:59.620417 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 09:34:59.623946 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 09:34:59.623946 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 09:34:59.623946 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application Aug 31 09:34:59.623946 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 414/817] 10.222.0.13 () {56 vars in 1108 bytes} [Wed Aug 31 09:34:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 402 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:34:59.629966 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc4c9c76-1724-4b7f-89f7-3da7fbd67fb3 None admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:59.688802 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd5aab03-63bb-46a6-bac3-4c3385408c38 None None] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:59.716907 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=100 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:34:59.731765 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f67d8af-9a91-4502-b81e-c03189bf9dd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:59.732427 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f67d8af-9a91-4502-b81e-c03189bf9dd8 None admin] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:59.732748 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f67d8af-9a91-4502-b81e-c03189bf9dd8 None admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:59.733017 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f67d8af-9a91-4502-b81e-c03189bf9dd8 None admin] PATH_INFO: `/v3/users/swift` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:59.741637 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f67d8af-9a91-4502-b81e-c03189bf9dd8 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:59.743118 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f67d8af-9a91-4502-b81e-c03189bf9dd8 None admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:59.756587 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application [None req-3f67d8af-9a91-4502-b81e-c03189bf9dd8 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 31 09:34:59.756587 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:34:59.756587 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:34:59.756587 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:34:59.756587 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:34:59.756587 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:34:59.756587 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:34:59.756587 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:34:59.756587 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:34:59.756587 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:34:59.756587 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:34:59.756587 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:34:59.756587 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 31 09:34:59.756587 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 31 09:34:59.756587 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 31 09:34:59.756587 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 31 09:34:59.760371 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:34:59.760371 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:34:59.760371 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 31 09:34:59.760371 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 09:34:59.760371 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 31 09:34:59.760371 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 09:34:59.760371 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:34:59.760371 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:34:59.760371 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:34:59.760371 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:34:59.760371 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:34:59.760371 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application with Lock( Aug 31 09:34:59.760371 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:34:59.760371 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:34:59.760371 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:34:59.760371 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:34:59.760371 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:34:59.760371 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:34:59.767097 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:34:59.767097 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:34:59.767097 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 31 09:34:59.767097 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 31 09:34:59.767097 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 31 09:34:59.767097 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 31 09:34:59.767097 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 31 09:34:59.767097 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 31 09:34:59.767097 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 31 09:34:59.767097 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application Aug 31 09:34:59.767097 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 404/818] 10.222.0.13 () {56 vars in 1096 bytes} [Wed Aug 31 09:34:59 2022] GET /identity/v3/users/swift => generated 83 bytes in 295 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:34:59.767097 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f67d8af-9a91-4502-b81e-c03189bf9dd8 None admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:34:59.917415 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f67d8af-9a91-4502-b81e-c03189bf9dd8 None admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:59.954218 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb0f842-e8ca-4e51-b716-8f22a9b418ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:34:59.955037 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb0f842-e8ca-4e51-b716-8f22a9b418ee None admin] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:34:59.955424 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb0f842-e8ca-4e51-b716-8f22a9b418ee None admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:34:59.955804 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb0f842-e8ca-4e51-b716-8f22a9b418ee None admin] PATH_INFO: `/v3/users` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:34:59.956770 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bb0f842-e8ca-4e51-b716-8f22a9b418ee None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=88114) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:34:59.957191 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bb0f842-e8ca-4e51-b716-8f22a9b418ee None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:34:59.961413 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bb0f842-e8ca-4e51-b716-8f22a9b418ee None admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:34:59.965349 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc4c9c76-1724-4b7f-89f7-3da7fbd67fb3 None admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:34:59.985562 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 405/819] 10.222.0.13 () {56 vars in 1135 bytes} [Wed Aug 31 09:34:59 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:34:59.992647 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb0f842-e8ca-4e51-b716-8f22a9b418ee None admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:35:00.052971 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=100 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:35:00.083323 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b094e8e-88ab-447c-a028-4d3004942fef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:35:00.084287 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b094e8e-88ab-447c-a028-4d3004942fef None admin] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:00.084755 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b094e8e-88ab-447c-a028-4d3004942fef None admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:00.085199 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b094e8e-88ab-447c-a028-4d3004942fef None admin] PATH_INFO: `/v3/domains` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:00.086063 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b094e8e-88ab-447c-a028-4d3004942fef None admin] RBAC: Adding query filter params (name=Default) {{(pid=88113) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:35:00.086546 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b094e8e-88ab-447c-a028-4d3004942fef None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:35:00.088213 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b094e8e-88ab-447c-a028-4d3004942fef None admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:35:00.102816 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 415/820] 10.222.0.13 () {56 vars in 1109 bytes} [Wed Aug 31 09:34:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 474 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:35:00.104046 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b094e8e-88ab-447c-a028-4d3004942fef None admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:35:00.161674 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb0f842-e8ca-4e51-b716-8f22a9b418ee None admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:35:00.178803 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=100 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:35:00.227188 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e2f72b-906c-417a-92e4-69d70f28729e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:35:00.228265 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e2f72b-906c-417a-92e4-69d70f28729e None admin] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:00.228788 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e2f72b-906c-417a-92e4-69d70f28729e None admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:00.229348 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e2f72b-906c-417a-92e4-69d70f28729e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:00.246263 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b094e8e-88ab-447c-a028-4d3004942fef None admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:35:00.250571 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17e2f72b-906c-417a-92e4-69d70f28729e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:35:00.253037 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17e2f72b-906c-417a-92e4-69d70f28729e None admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:35:00.282704 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application [None req-17e2f72b-906c-417a-92e4-69d70f28729e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 09:35:00.282704 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:35:00.282704 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:35:00.282704 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:35:00.282704 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:35:00.282704 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:35:00.282704 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:35:00.282704 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:35:00.282704 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:35:00.282704 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:35:00.282704 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:35:00.282704 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:35:00.282704 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 09:35:00.282704 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 09:35:00.282704 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 09:35:00.282704 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 09:35:00.283746 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:35:00.283746 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:35:00.283746 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:35:00.283746 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:35:00.283746 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:35:00.283746 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:35:00.283746 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:35:00.283746 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application with Lock( Aug 31 09:35:00.283746 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:35:00.283746 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:35:00.283746 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:35:00.283746 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:35:00.283746 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:35:00.283746 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:35:00.283746 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:35:00.283746 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:35:00.283746 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 09:35:00.284632 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 09:35:00.284632 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 09:35:00.284632 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application Aug 31 09:35:00.285836 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 406/821] 10.222.0.13 () {56 vars in 1108 bytes} [Wed Aug 31 09:34:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 298 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 09:35:00.286513 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f75a15dd-09dd-4bdf-9531-5000c6c1094e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:35:00.287155 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75a15dd-09dd-4bdf-9531-5000c6c1094e None admin] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:00.287436 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75a15dd-09dd-4bdf-9531-5000c6c1094e None admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:00.287710 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75a15dd-09dd-4bdf-9531-5000c6c1094e None admin] PATH_INFO: `/v3/services/network` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:00.288306 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f75a15dd-09dd-4bdf-9531-5000c6c1094e None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:35:00.289784 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f75a15dd-09dd-4bdf-9531-5000c6c1094e None admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:35:00.294503 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e2f72b-906c-417a-92e4-69d70f28729e None admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:35:00.301165 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application [None req-f75a15dd-09dd-4bdf-9531-5000c6c1094e None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Aug 31 09:35:00.301165 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:35:00.301165 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 31 09:35:00.301165 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 31 09:35:00.301165 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 31 09:35:00.301165 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 31 09:35:00.301165 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 31 09:35:00.301165 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 09:35:00.301165 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 31 09:35:00.301165 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application Aug 31 09:35:00.301165 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 31 09:35:00.301165 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application Aug 31 09:35:00.301165 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:35:00.301165 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:35:00.301165 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:35:00.301165 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:35:00.301165 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:35:00.303290 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:35:00.303290 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:35:00.303290 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:35:00.303290 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:35:00.303290 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:35:00.303290 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:35:00.303290 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 31 09:35:00.303290 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 31 09:35:00.303290 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 31 09:35:00.303290 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 31 09:35:00.303290 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:35:00.303290 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:35:00.303290 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:35:00.303290 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:35:00.303290 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:35:00.303290 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:35:00.304402 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:35:00.304402 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application with Lock( Aug 31 09:35:00.304402 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:35:00.304402 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:35:00.304402 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:35:00.304402 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:35:00.304402 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:35:00.304402 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:35:00.304402 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:35:00.304402 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:35:00.304402 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 31 09:35:00.304402 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 09:35:00.304402 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 31 09:35:00.304402 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application Aug 31 09:35:00.304402 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 416/822] 10.222.0.13 () {56 vars in 1111 bytes} [Wed Aug 31 09:35:00 2022] GET /identity/v3/services/network => generated 88 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:35:00.307260 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f75a15dd-09dd-4bdf-9531-5000c6c1094e None admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:35:00.419451 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=100 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:35:00.430624 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e2f72b-906c-417a-92e4-69d70f28729e None admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:35:00.431911 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f75a15dd-09dd-4bdf-9531-5000c6c1094e None admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:35:00.442743 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=100 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:35:00.468160 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c697ecf-a218-4eeb-959a-44b91fc54a99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:35:00.468805 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c697ecf-a218-4eeb-959a-44b91fc54a99 None admin] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:00.469290 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c697ecf-a218-4eeb-959a-44b91fc54a99 None admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:00.469615 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c697ecf-a218-4eeb-959a-44b91fc54a99 None admin] PATH_INFO: `/v3/projects/service` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:00.477243 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c697ecf-a218-4eeb-959a-44b91fc54a99 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:35:00.477929 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f8c387-878f-4a98-9ad1-8e0b5e8396ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:35:00.478715 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f8c387-878f-4a98-9ad1-8e0b5e8396ff None admin] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:00.478859 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c697ecf-a218-4eeb-959a-44b91fc54a99 None admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:35:00.479267 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f8c387-878f-4a98-9ad1-8e0b5e8396ff None admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:00.479687 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f8c387-878f-4a98-9ad1-8e0b5e8396ff None admin] PATH_INFO: `/v3/domains` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:00.480595 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65f8c387-878f-4a98-9ad1-8e0b5e8396ff None admin] RBAC: Adding query filter params (name=Default) {{(pid=88113) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:35:00.481047 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65f8c387-878f-4a98-9ad1-8e0b5e8396ff None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:35:00.483230 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65f8c387-878f-4a98-9ad1-8e0b5e8396ff None admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:35:00.491742 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application [None req-4c697ecf-a218-4eeb-959a-44b91fc54a99 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 09:35:00.491742 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:35:00.491742 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:35:00.491742 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:35:00.491742 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:35:00.491742 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:35:00.491742 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:35:00.491742 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:35:00.491742 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:35:00.491742 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:35:00.491742 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:35:00.491742 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:35:00.491742 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 31 09:35:00.491742 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 31 09:35:00.491742 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 31 09:35:00.492742 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 31 09:35:00.492742 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:35:00.492742 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:35:00.492742 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:35:00.492742 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:35:00.492742 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:35:00.492742 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:35:00.492742 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:35:00.492742 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application with Lock( Aug 31 09:35:00.492742 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:35:00.492742 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:35:00.492742 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:35:00.492742 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:35:00.492742 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:35:00.492742 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:35:00.492742 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:35:00.492742 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:35:00.493694 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 31 09:35:00.493694 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 31 09:35:00.493694 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 31 09:35:00.493694 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 31 09:35:00.493694 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 31 09:35:00.493694 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 31 09:35:00.493694 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 09:35:00.493694 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application Aug 31 09:35:00.493694 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 407/823] 10.222.0.13 () {56 vars in 1111 bytes} [Wed Aug 31 09:35:00 2022] GET /identity/v3/projects/service => generated 88 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:35:00.494184 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c697ecf-a218-4eeb-959a-44b91fc54a99 None admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:35:00.508145 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 417/824] 10.222.0.13 () {56 vars in 1109 bytes} [Wed Aug 31 09:35:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:35:00.508145 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f8c387-878f-4a98-9ad1-8e0b5e8396ff None admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:35:00.667775 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f8c387-878f-4a98-9ad1-8e0b5e8396ff None admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:35:00.681304 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c697ecf-a218-4eeb-959a-44b91fc54a99 None admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:35:00.721248 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19cff182-e421-41cd-9336-048d02e5591e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:35:00.722389 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19cff182-e421-41cd-9336-048d02e5591e None admin] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:00.722861 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19cff182-e421-41cd-9336-048d02e5591e None admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:00.723370 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19cff182-e421-41cd-9336-048d02e5591e None admin] PATH_INFO: `/v3/projects` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:00.723738 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0575854b-e27d-4458-ac61-f3943179e84f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:35:00.724357 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0575854b-e27d-4458-ac61-f3943179e84f None admin] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:00.724636 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0575854b-e27d-4458-ac61-f3943179e84f None admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:00.724732 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19cff182-e421-41cd-9336-048d02e5591e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88113) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:35:00.724907 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0575854b-e27d-4458-ac61-f3943179e84f None admin] PATH_INFO: `/v3/services` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:00.725242 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19cff182-e421-41cd-9336-048d02e5591e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:35:00.725579 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0575854b-e27d-4458-ac61-f3943179e84f None admin] RBAC: Adding query filter params (name=network) {{(pid=88114) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:35:00.725885 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0575854b-e27d-4458-ac61-f3943179e84f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:35:00.727553 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0575854b-e27d-4458-ac61-f3943179e84f None admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:35:00.727969 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19cff182-e421-41cd-9336-048d02e5591e None admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:35:00.738988 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 408/825] 10.222.0.13 () {56 vars in 1112 bytes} [Wed Aug 31 09:35:00 2022] GET /identity/v3/services?name=network => generated 124 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:35:00.738988 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0575854b-e27d-4458-ac61-f3943179e84f None admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:35:00.749576 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 418/826] 10.222.0.13 () {56 vars in 1148 bytes} [Wed Aug 31 09:35:00 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:35:00.752805 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f34fe83-7a97-4077-b01d-1350f5353491 None None] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:00.753778 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f34fe83-7a97-4077-b01d-1350f5353491 None None] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:00.754108 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f34fe83-7a97-4077-b01d-1350f5353491 None None] PATH_INFO: `/` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:00.761286 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 419/827] 10.222.0.13 () {54 vars in 927 bytes} [Wed Aug 31 09:35:00 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:35:00.761286 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f34fe83-7a97-4077-b01d-1350f5353491 None None] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:35:00.765024 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=100 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:35:00.798627 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=100 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:35:00.895481 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0575854b-e27d-4458-ac61-f3943179e84f None admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:35:00.909422 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f34fe83-7a97-4077-b01d-1350f5353491 None None] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:35:00.933584 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd0c8b01-8702-4136-a7a8-32b8f238d183 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:35:00.934232 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0c8b01-8702-4136-a7a8-32b8f238d183 None admin] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:00.934514 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0c8b01-8702-4136-a7a8-32b8f238d183 None admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:00.934791 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0c8b01-8702-4136-a7a8-32b8f238d183 None admin] PATH_INFO: `/v3/projects/service` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:00.943104 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd0c8b01-8702-4136-a7a8-32b8f238d183 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:35:00.944604 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd0c8b01-8702-4136-a7a8-32b8f238d183 None admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:35:00.952534 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d7dfc8c-12b6-4b37-a5fa-847140f22740 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:35:00.953465 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7dfc8c-12b6-4b37-a5fa-847140f22740 None admin] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:00.953953 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7dfc8c-12b6-4b37-a5fa-847140f22740 None admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:00.954395 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7dfc8c-12b6-4b37-a5fa-847140f22740 None admin] PATH_INFO: `/v3/services` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:00.955203 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d7dfc8c-12b6-4b37-a5fa-847140f22740 None admin] RBAC: Adding query filter params (type=network) {{(pid=88113) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:35:00.955674 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d7dfc8c-12b6-4b37-a5fa-847140f22740 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:35:00.957229 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d7dfc8c-12b6-4b37-a5fa-847140f22740 None admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:35:00.957641 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application [None req-bd0c8b01-8702-4136-a7a8-32b8f238d183 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 09:35:00.957641 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:35:00.957641 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:35:00.957641 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:35:00.957641 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:35:00.957641 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:35:00.957641 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:35:00.957641 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:35:00.957641 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:35:00.957641 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:35:00.957641 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:35:00.957641 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:35:00.957641 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 31 09:35:00.957641 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 31 09:35:00.957641 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 31 09:35:00.958491 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 31 09:35:00.958491 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:35:00.958491 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:35:00.958491 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:35:00.958491 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:35:00.958491 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:35:00.958491 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:35:00.958491 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:35:00.958491 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application with Lock( Aug 31 09:35:00.958491 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:35:00.958491 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:35:00.958491 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:35:00.958491 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:35:00.958491 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:35:00.958491 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:35:00.958491 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:35:00.958491 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:35:00.959460 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 31 09:35:00.959460 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 31 09:35:00.959460 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 31 09:35:00.959460 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 31 09:35:00.959460 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 31 09:35:00.959460 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 31 09:35:00.959460 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 09:35:00.959460 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application Aug 31 09:35:00.959460 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 409/828] 10.222.0.13 () {56 vars in 1111 bytes} [Wed Aug 31 09:35:00 2022] GET /identity/v3/projects/service => generated 88 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:35:00.959941 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd0c8b01-8702-4136-a7a8-32b8f238d183 None admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:35:00.978395 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 420/829] 10.222.0.13 () {56 vars in 1112 bytes} [Wed Aug 31 09:35:00 2022] GET /identity/v3/services?type=network => generated 350 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:35:00.984967 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ec2483-6bac-4e7d-9641-e44a7e3367ec None None] REQUEST_METHOD: `POST` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:00.985489 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ec2483-6bac-4e7d-9641-e44a7e3367ec None None] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:00.985860 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ec2483-6bac-4e7d-9641-e44a7e3367ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:01.017779 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.auth.core [None req-34ec2483-6bac-4e7d-9641-e44a7e3367ec None None] MFA Rules not processed for user `fd836cf590a145df9acf19cf3ae2ac56`. Rule list: `[]` (Enabled: `True`). {{(pid=88113) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:35:01.019266 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.fernet_utils [None req-34ec2483-6bac-4e7d-9641-e44a7e3367ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:35:01.094787 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=100 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:35:01.100710 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 421/830] 10.222.0.13 () {58 vars in 1024 bytes} [Wed Aug 31 09:35:00 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 09:35:01.101969 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ec2483-6bac-4e7d-9641-e44a7e3367ec None None] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:35:01.125125 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd0c8b01-8702-4136-a7a8-32b8f238d183 None admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:35:01.129611 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=100 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:35:01.169128 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6672cde9-7aa5-47c2-a067-f20411d21da3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:35:01.170195 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6672cde9-7aa5-47c2-a067-f20411d21da3 None admin] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:01.170746 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6672cde9-7aa5-47c2-a067-f20411d21da3 None admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:01.171223 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6672cde9-7aa5-47c2-a067-f20411d21da3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:01.172286 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6672cde9-7aa5-47c2-a067-f20411d21da3 None admin] RBAC: Adding query filter params (role.id=5d4ee63a3add47fc9a6b8b16d39e0a1f, scope.project.id=49548eb44d62415582af820ab66decee, user.id=dc076493239d4ccdbdc37c0369c49529) {{(pid=88114) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:35:01.172825 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6672cde9-7aa5-47c2-a067-f20411d21da3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:35:01.175401 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6672cde9-7aa5-47c2-a067-f20411d21da3 None admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:35:01.190076 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 410/831] 10.222.0.13 () {56 vars in 1374 bytes} [Wed Aug 31 09:35:00 2022] GET /identity/v3/role_assignments?user.id=dc076493239d4ccdbdc37c0369c49529&scope.project.id=49548eb44d62415582af820ab66decee&role.id=5d4ee63a3add47fc9a6b8b16d39e0a1f => generated 608 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:35:01.191527 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6672cde9-7aa5-47c2-a067-f20411d21da3 None admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:35:01.253223 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ec2483-6bac-4e7d-9641-e44a7e3367ec None None] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:35:01.315599 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d857ee-3f89-4c98-b1f8-3f45b90d535e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:35:01.316417 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d857ee-3f89-4c98-b1f8-3f45b90d535e None admin] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:01.316861 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d857ee-3f89-4c98-b1f8-3f45b90d535e None admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:01.317260 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d857ee-3f89-4c98-b1f8-3f45b90d535e None admin] PATH_INFO: `/v3/projects` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:01.318192 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5d857ee-3f89-4c98-b1f8-3f45b90d535e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88113) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:35:01.318541 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5d857ee-3f89-4c98-b1f8-3f45b90d535e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:35:01.320933 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5d857ee-3f89-4c98-b1f8-3f45b90d535e None admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:35:01.340420 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 422/832] 10.222.0.13 () {56 vars in 1148 bytes} [Wed Aug 31 09:35:01 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:35:01.349891 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d857ee-3f89-4c98-b1f8-3f45b90d535e None admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:35:01.361967 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6672cde9-7aa5-47c2-a067-f20411d21da3 None admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:35:01.408314 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e7d5d2-7bd0-4f37-a536-1265760b0083 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:35:01.409512 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e7d5d2-7bd0-4f37-a536-1265760b0083 None admin] REQUEST_METHOD: `POST` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:01.410055 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e7d5d2-7bd0-4f37-a536-1265760b0083 None admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:01.410552 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e7d5d2-7bd0-4f37-a536-1265760b0083 None admin] PATH_INFO: `/v3/endpoints` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:01.411498 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40e7d5d2-7bd0-4f37-a536-1265760b0083 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:35:01.413339 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40e7d5d2-7bd0-4f37-a536-1265760b0083 None admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:35:01.425249 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:35:01.443253 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 411/833] 10.222.0.13 () {60 vars in 1144 bytes} [Wed Aug 31 09:35:01 2022] POST /identity/v3/endpoints => generated 337 bytes in 252 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 09:35:01.478132 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e7d5d2-7bd0-4f37-a536-1265760b0083 None admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:35:01.510449 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=100 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:35:01.548495 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d857ee-3f89-4c98-b1f8-3f45b90d535e None admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:35:01.625372 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d83a5f-ee44-483d-a5a9-c8a5b3768b01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:35:01.626153 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d83a5f-ee44-483d-a5a9-c8a5b3768b01 None admin] REQUEST_METHOD: `PUT` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:01.626501 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d83a5f-ee44-483d-a5a9-c8a5b3768b01 None admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:01.626850 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d83a5f-ee44-483d-a5a9-c8a5b3768b01 None admin] PATH_INFO: `/v3/projects/49548eb44d62415582af820ab66decee/users/202bb9db7a8a4977ba788a290ed9a3fa/roles/5d4ee63a3add47fc9a6b8b16d39e0a1f` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:01.628964 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06d83a5f-ee44-483d-a5a9-c8a5b3768b01 None admin] RBAC: Authorizing `identity:create_grant(project_id=49548eb44d62415582af820ab66decee, user_id=202bb9db7a8a4977ba788a290ed9a3fa, role_id=5d4ee63a3add47fc9a6b8b16d39e0a1f)` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:35:01.630984 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06d83a5f-ee44-483d-a5a9-c8a5b3768b01 None admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:35:01.634423 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e7d5d2-7bd0-4f37-a536-1265760b0083 None admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:35:01.660930 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 423/834] 10.222.0.13 () {58 vars in 1439 bytes} [Wed Aug 31 09:35:01 2022] PUT /identity/v3/projects/49548eb44d62415582af820ab66decee/users/202bb9db7a8a4977ba788a290ed9a3fa/roles/5d4ee63a3add47fc9a6b8b16d39e0a1f => generated 0 bytes in 312 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:35:01.697063 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d197a04-95a8-4061-99e7-6e30f7fa48d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:35:01.698187 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d197a04-95a8-4061-99e7-6e30f7fa48d2 None admin] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:01.698711 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d197a04-95a8-4061-99e7-6e30f7fa48d2 None admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:01.699235 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d197a04-95a8-4061-99e7-6e30f7fa48d2 None admin] PATH_INFO: `/v3/roles/service` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:01.713361 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d197a04-95a8-4061-99e7-6e30f7fa48d2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 09:35:01.720192 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application [None req-0d197a04-95a8-4061-99e7-6e30f7fa48d2 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 31 09:35:01.720192 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:35:01.720192 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:35:01.720192 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:35:01.720192 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:35:01.720192 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:35:01.720192 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:35:01.720192 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:35:01.720192 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:35:01.720192 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:35:01.720192 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:35:01.720192 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:35:01.720192 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 09:35:01.720192 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 09:35:01.720192 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 09:35:01.720192 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 09:35:01.721701 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 09:35:01.721701 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 09:35:01.721701 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 09:35:01.721701 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 09:35:01.721701 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:35:01.721701 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:35:01.721701 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:35:01.721701 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:35:01.721701 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:35:01.721701 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:35:01.721701 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:35:01.721701 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application with Lock( Aug 31 09:35:01.721701 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:35:01.721701 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:35:01.721701 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:35:01.721701 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:35:01.722641 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:35:01.722641 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:35:01.722641 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:35:01.722641 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:35:01.722641 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 09:35:01.722641 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 09:35:01.722641 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 09:35:01.722641 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 09:35:01.722641 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 09:35:01.722641 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 09:35:01.722641 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 31 09:35:01.722641 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application Aug 31 09:35:01.722641 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 412/835] 10.222.0.13 () {56 vars in 1102 bytes} [Wed Aug 31 09:35:01 2022] GET /identity/v3/roles/service => generated 85 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 09:35:01.728296 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d83a5f-ee44-483d-a5a9-c8a5b3768b01 None admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:35:01.814452 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9c589e-feeb-4088-8e71-b76f245451b1 None None] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:01.814971 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9c589e-feeb-4088-8e71-b76f245451b1 None None] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:01.814971 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9c589e-feeb-4088-8e71-b76f245451b1 None None] PATH_INFO: `/` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:01.816685 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 413/836] 10.222.0.13 () {54 vars in 927 bytes} [Wed Aug 31 09:35:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:35:01.822669 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f607c641-452d-4e49-b0e3-55cd0e076157 None None] REQUEST_METHOD: `POST` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:01.823007 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f607c641-452d-4e49-b0e3-55cd0e076157 None None] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:01.823212 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f607c641-452d-4e49-b0e3-55cd0e076157 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:01.856236 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.auth.core [None req-f607c641-452d-4e49-b0e3-55cd0e076157 None None] MFA Rules not processed for user `fd836cf590a145df9acf19cf3ae2ac56`. Rule list: `[]` (Enabled: `True`). {{(pid=88114) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:35:01.860868 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.fernet_utils [None req-f607c641-452d-4e49-b0e3-55cd0e076157 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88114) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:35:01.896371 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d83a5f-ee44-483d-a5a9-c8a5b3768b01 None admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:35:01.940767 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=100 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:35:01.959782 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f207348-5e99-4f73-ad03-bdf14b4aab00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:35:01.961058 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f207348-5e99-4f73-ad03-bdf14b4aab00 None admin] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:01.961505 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f207348-5e99-4f73-ad03-bdf14b4aab00 None admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:01.962302 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f207348-5e99-4f73-ad03-bdf14b4aab00 None admin] PATH_INFO: `/v3/roles` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:01.964001 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f207348-5e99-4f73-ad03-bdf14b4aab00 None admin] RBAC: Adding query filter params (name=service) {{(pid=88113) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:35:01.964507 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f207348-5e99-4f73-ad03-bdf14b4aab00 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:35:01.967597 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f207348-5e99-4f73-ad03-bdf14b4aab00 None admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:35:01.968900 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 414/837] 10.222.0.13 () {58 vars in 1024 bytes} [Wed Aug 31 09:35:01 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 31 09:35:01.971282 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:35:01.981779 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 424/838] 10.222.0.13 () {56 vars in 1103 bytes} [Wed Aug 31 09:35:01 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:35:01.991311 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f607c641-452d-4e49-b0e3-55cd0e076157 None None] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:35:02.114665 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f607c641-452d-4e49-b0e3-55cd0e076157 None None] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:35:02.157735 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c526c2-5f84-473e-9c33-1b68c33d69e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:35:02.158399 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c526c2-5f84-473e-9c33-1b68c33d69e4 None admin] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:02.158779 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c526c2-5f84-473e-9c33-1b68c33d69e4 None admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:02.159062 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c526c2-5f84-473e-9c33-1b68c33d69e4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:02.171469 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5c526c2-5f84-473e-9c33-1b68c33d69e4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:35:02.173128 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5c526c2-5f84-473e-9c33-1b68c33d69e4 None admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:35:02.190119 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application [None req-e5c526c2-5f84-473e-9c33-1b68c33d69e4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 09:35:02.190119 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:35:02.190119 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:35:02.190119 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:35:02.190119 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:35:02.190119 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:35:02.190119 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:35:02.190119 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:35:02.190119 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:35:02.190119 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:35:02.190119 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:35:02.190119 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:35:02.190119 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 09:35:02.190119 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 09:35:02.190119 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 09:35:02.190119 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 09:35:02.191154 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:35:02.191154 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:35:02.191154 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:35:02.191154 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:35:02.191154 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:35:02.191154 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:35:02.191154 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:35:02.191154 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application with Lock( Aug 31 09:35:02.191154 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:35:02.191154 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:35:02.191154 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:35:02.191154 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:35:02.191154 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:35:02.191154 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:35:02.191154 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:35:02.191154 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:35:02.191154 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 09:35:02.192458 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 09:35:02.192458 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 09:35:02.192458 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application Aug 31 09:35:02.192458 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 415/839] 10.222.0.13 () {56 vars in 1108 bytes} [Wed Aug 31 09:35:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:35:02.197255 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f207348-5e99-4f73-ad03-bdf14b4aab00 None admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:35:02.273286 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c526c2-5f84-473e-9c33-1b68c33d69e4 None admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:35:02.318185 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=100 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:35:02.346716 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f207348-5e99-4f73-ad03-bdf14b4aab00 None admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:35:02.395773 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c84d0d0a-548e-4a4d-9a3f-1c5f4670275e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:35:02.396776 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84d0d0a-548e-4a4d-9a3f-1c5f4670275e None admin] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:02.397578 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84d0d0a-548e-4a4d-9a3f-1c5f4670275e None admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:02.398302 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84d0d0a-548e-4a4d-9a3f-1c5f4670275e None admin] PATH_INFO: `/v3/domains` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:02.399125 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c84d0d0a-548e-4a4d-9a3f-1c5f4670275e None admin] RBAC: Adding query filter params (name=Default) {{(pid=88113) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:35:02.399717 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c84d0d0a-548e-4a4d-9a3f-1c5f4670275e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:35:02.401908 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c84d0d0a-548e-4a4d-9a3f-1c5f4670275e None admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:35:02.403701 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c526c2-5f84-473e-9c33-1b68c33d69e4 None admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:35:02.418485 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 425/840] 10.222.0.13 () {56 vars in 1109 bytes} [Wed Aug 31 09:35:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:35:02.425093 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c84d0d0a-548e-4a4d-9a3f-1c5f4670275e None admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:35:02.447790 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb43aa84-9755-4e78-88a0-0c632be01a59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:35:02.448413 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb43aa84-9755-4e78-88a0-0c632be01a59 None admin] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:02.448692 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb43aa84-9755-4e78-88a0-0c632be01a59 None admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:02.449136 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb43aa84-9755-4e78-88a0-0c632be01a59 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:02.450080 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb43aa84-9755-4e78-88a0-0c632be01a59 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:35:02.451138 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb43aa84-9755-4e78-88a0-0c632be01a59 None admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:35:02.461609 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application [None req-cb43aa84-9755-4e78-88a0-0c632be01a59 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 31 09:35:02.461609 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:35:02.461609 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 31 09:35:02.461609 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 31 09:35:02.461609 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 31 09:35:02.461609 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 31 09:35:02.461609 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 31 09:35:02.461609 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 09:35:02.461609 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 31 09:35:02.461609 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application Aug 31 09:35:02.461609 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 31 09:35:02.461609 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application Aug 31 09:35:02.461609 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:35:02.461609 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:35:02.461609 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:35:02.461609 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:35:02.461609 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:35:02.462743 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:35:02.462743 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:35:02.462743 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:35:02.462743 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:35:02.462743 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:35:02.462743 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:35:02.462743 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 31 09:35:02.462743 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 31 09:35:02.462743 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 31 09:35:02.462743 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 31 09:35:02.462743 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:35:02.462743 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:35:02.462743 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:35:02.462743 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:35:02.462743 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:35:02.462743 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:35:02.463601 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:35:02.463601 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application with Lock( Aug 31 09:35:02.463601 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:35:02.463601 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:35:02.463601 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:35:02.463601 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:35:02.463601 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:35:02.463601 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:35:02.463601 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:35:02.463601 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:35:02.463601 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 31 09:35:02.463601 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 09:35:02.463601 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 31 09:35:02.463601 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application Aug 31 09:35:02.463601 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 416/841] 10.222.0.13 () {56 vars in 1114 bytes} [Wed Aug 31 09:35:02 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 09:35:02.470142 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb43aa84-9755-4e78-88a0-0c632be01a59 None admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:35:02.503840 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=100 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:35:02.552264 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c84d0d0a-548e-4a4d-9a3f-1c5f4670275e None admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:35:02.579445 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=100 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:35:02.584831 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb43aa84-9755-4e78-88a0-0c632be01a59 None admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:35:02.596169 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb7ca786-c630-4af4-ae6e-d455403f2019 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:35:02.596817 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7ca786-c630-4af4-ae6e-d455403f2019 None admin] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:02.597098 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7ca786-c630-4af4-ae6e-d455403f2019 None admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:02.597369 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7ca786-c630-4af4-ae6e-d455403f2019 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:02.609447 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb7ca786-c630-4af4-ae6e-d455403f2019 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:35:02.610963 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb7ca786-c630-4af4-ae6e-d455403f2019 None admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:35:02.627948 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application [None req-fb7ca786-c630-4af4-ae6e-d455403f2019 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 31 09:35:02.627948 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:35:02.627948 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:35:02.627948 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:35:02.627948 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:35:02.627948 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:35:02.627948 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:35:02.627948 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:35:02.627948 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:35:02.627948 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:35:02.627948 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:35:02.627948 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:35:02.627948 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 31 09:35:02.627948 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 31 09:35:02.627948 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 31 09:35:02.627948 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 31 09:35:02.628977 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:35:02.628977 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:35:02.628977 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 31 09:35:02.628977 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 09:35:02.628977 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 31 09:35:02.628977 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 09:35:02.628977 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:35:02.628977 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:35:02.628977 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:35:02.628977 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:35:02.628977 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:35:02.628977 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application with Lock( Aug 31 09:35:02.628977 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:35:02.628977 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:35:02.628977 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:35:02.628977 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:35:02.628977 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:35:02.628977 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:35:02.630371 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:35:02.630371 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:35:02.630371 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 31 09:35:02.630371 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 31 09:35:02.630371 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 31 09:35:02.630371 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 31 09:35:02.630371 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 31 09:35:02.630371 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 31 09:35:02.630371 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 31 09:35:02.630371 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application Aug 31 09:35:02.633526 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 426/842] 10.222.0.13 () {56 vars in 1117 bytes} [Wed Aug 31 09:35:02 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:35:02.635100 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb7ca786-c630-4af4-ae6e-d455403f2019 None admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:35:02.637364 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5693aa4d-1355-45c8-8b03-290ff7651c34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:35:02.638026 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5693aa4d-1355-45c8-8b03-290ff7651c34 None admin] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:02.638304 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5693aa4d-1355-45c8-8b03-290ff7651c34 None admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:02.638574 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5693aa4d-1355-45c8-8b03-290ff7651c34 None admin] PATH_INFO: `/v3/services` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:02.640244 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5693aa4d-1355-45c8-8b03-290ff7651c34 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=88114) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:35:02.640508 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5693aa4d-1355-45c8-8b03-290ff7651c34 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:35:02.643057 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5693aa4d-1355-45c8-8b03-290ff7651c34 None admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:35:02.649648 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 417/843] 10.222.0.13 () {56 vars in 1114 bytes} [Wed Aug 31 09:35:02 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 09:35:02.655053 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5693aa4d-1355-45c8-8b03-290ff7651c34 None admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:35:02.772679 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5693aa4d-1355-45c8-8b03-290ff7651c34 None admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:35:02.774607 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb7ca786-c630-4af4-ae6e-d455403f2019 None admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:35:02.794747 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=100 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:35:02.816998 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1daebaa-b550-43e3-8934-019144a71cb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:35:02.817721 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1daebaa-b550-43e3-8934-019144a71cb4 None admin] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:02.817984 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f6736a-a0dd-4c21-9f02-e40b7bc488ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:35:02.818435 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1daebaa-b550-43e3-8934-019144a71cb4 None admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:02.818435 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1daebaa-b550-43e3-8934-019144a71cb4 None admin] PATH_INFO: `/v3/users` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:02.819297 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f6736a-a0dd-4c21-9f02-e40b7bc488ac None admin] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:02.819417 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1daebaa-b550-43e3-8934-019144a71cb4 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=88113) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:35:02.819670 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1daebaa-b550-43e3-8934-019144a71cb4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:35:02.820036 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f6736a-a0dd-4c21-9f02-e40b7bc488ac None admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:02.820480 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f6736a-a0dd-4c21-9f02-e40b7bc488ac None admin] PATH_INFO: `/v3/services` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:02.821304 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5f6736a-a0dd-4c21-9f02-e40b7bc488ac None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=88114) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:35:02.821449 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1daebaa-b550-43e3-8934-019144a71cb4 None admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:35:02.821936 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5f6736a-a0dd-4c21-9f02-e40b7bc488ac None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:35:02.823619 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5f6736a-a0dd-4c21-9f02-e40b7bc488ac None admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:35:02.830257 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 418/844] 10.222.0.13 () {56 vars in 1114 bytes} [Wed Aug 31 09:35:02 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:35:02.843768 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 427/845] 10.222.0.13 () {56 vars in 1149 bytes} [Wed Aug 31 09:35:02 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:35:02.849789 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f6736a-a0dd-4c21-9f02-e40b7bc488ac None admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:35:02.886309 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=100 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:35:02.993915 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f6736a-a0dd-4c21-9f02-e40b7bc488ac None admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:35:03.035444 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f6fbc7-2f78-484f-b5bc-7059ed87f2e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:35:03.036070 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f6fbc7-2f78-484f-b5bc-7059ed87f2e1 None admin] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:03.036372 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f6fbc7-2f78-484f-b5bc-7059ed87f2e1 None admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:03.036633 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f6fbc7-2f78-484f-b5bc-7059ed87f2e1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:03.118878 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8f6fbc7-2f78-484f-b5bc-7059ed87f2e1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:35:03.120336 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8f6fbc7-2f78-484f-b5bc-7059ed87f2e1 None admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:35:03.153098 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=100 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:35:03.241880 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 419/846] 10.222.0.13 () {58 vars in 1282 bytes} [Wed Aug 31 09:35:02 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 393 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 31 09:35:03.250673 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1daebaa-b550-43e3-8934-019144a71cb4 None admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:35:03.435204 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1daebaa-b550-43e3-8934-019144a71cb4 None admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:35:03.504067 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d24e53-d718-4495-bb77-40602af52948 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:35:03.505119 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d24e53-d718-4495-bb77-40602af52948 None admin] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:03.505621 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d24e53-d718-4495-bb77-40602af52948 None admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:03.506097 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d24e53-d718-4495-bb77-40602af52948 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:03.526328 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52d24e53-d718-4495-bb77-40602af52948 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:35:03.528840 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52d24e53-d718-4495-bb77-40602af52948 None admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:35:03.557982 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application [None req-52d24e53-d718-4495-bb77-40602af52948 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 09:35:03.557982 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:35:03.557982 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:35:03.557982 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:35:03.557982 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:35:03.557982 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:35:03.557982 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:35:03.557982 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:35:03.557982 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:35:03.557982 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:35:03.557982 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:35:03.557982 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:35:03.557982 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 09:35:03.557982 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 09:35:03.557982 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 09:35:03.557982 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 09:35:03.559797 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:35:03.559797 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:35:03.559797 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:35:03.559797 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:35:03.559797 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:35:03.559797 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:35:03.559797 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:35:03.559797 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application with Lock( Aug 31 09:35:03.559797 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:35:03.559797 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:35:03.559797 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:35:03.559797 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:35:03.559797 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:35:03.559797 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:35:03.559797 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:35:03.559797 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:35:03.559797 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 09:35:03.564204 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 09:35:03.564204 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 09:35:03.564204 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application Aug 31 09:35:03.564204 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 428/847] 10.222.0.13 () {56 vars in 1108 bytes} [Wed Aug 31 09:35:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 310 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:35:03.568863 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f6fbc7-2f78-484f-b5bc-7059ed87f2e1 None admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:35:03.749783 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f6fbc7-2f78-484f-b5bc-7059ed87f2e1 None admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:35:03.798161 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=100 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:35:03.815843 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5e24940-5dbe-4201-b57d-282310304283 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:35:03.817003 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e24940-5dbe-4201-b57d-282310304283 None admin] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:03.817560 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e24940-5dbe-4201-b57d-282310304283 None admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:03.818030 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e24940-5dbe-4201-b57d-282310304283 None admin] PATH_INFO: `/v3/domains` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:03.819147 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5e24940-5dbe-4201-b57d-282310304283 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88114) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:35:03.819639 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5e24940-5dbe-4201-b57d-282310304283 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:35:03.822112 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5e24940-5dbe-4201-b57d-282310304283 None admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:35:03.846392 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 420/848] 10.222.0.13 () {56 vars in 1109 bytes} [Wed Aug 31 09:35:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 277 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:35:03.857887 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d24e53-d718-4495-bb77-40602af52948 None admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:35:03.910312 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=100 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:35:04.046296 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d24e53-d718-4495-bb77-40602af52948 None admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:35:04.059725 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82376e37-5003-4f73-aa73-ab5e9ea126e7 None None] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:04.060144 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82376e37-5003-4f73-aa73-ab5e9ea126e7 None None] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:04.060579 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82376e37-5003-4f73-aa73-ab5e9ea126e7 None None] PATH_INFO: `/` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:04.063378 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 421/849] 10.222.0.13 () {54 vars in 927 bytes} [Wed Aug 31 09:35:04 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 31 09:35:04.072132 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fcff8c2-2f01-4ae4-b172-81003f57c45e None None] REQUEST_METHOD: `POST` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:04.072604 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fcff8c2-2f01-4ae4-b172-81003f57c45e None None] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:04.073088 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fcff8c2-2f01-4ae4-b172-81003f57c45e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:04.108576 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.auth.core [None req-9fcff8c2-2f01-4ae4-b172-81003f57c45e None None] MFA Rules not processed for user `fd836cf590a145df9acf19cf3ae2ac56`. Rule list: `[]` (Enabled: `True`). {{(pid=88114) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:35:04.111255 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.fernet_utils [None req-9fcff8c2-2f01-4ae4-b172-81003f57c45e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88114) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:35:04.119685 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec6ecb64-4f96-401b-8af4-8abb2ded4c61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:35:04.120720 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6ecb64-4f96-401b-8af4-8abb2ded4c61 None admin] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:04.121200 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6ecb64-4f96-401b-8af4-8abb2ded4c61 None admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:04.121714 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6ecb64-4f96-401b-8af4-8abb2ded4c61 None admin] PATH_INFO: `/v3/projects/service` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:04.132025 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec6ecb64-4f96-401b-8af4-8abb2ded4c61 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:35:04.134078 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec6ecb64-4f96-401b-8af4-8abb2ded4c61 None admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:35:04.149660 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application [None req-ec6ecb64-4f96-401b-8af4-8abb2ded4c61 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 09:35:04.149660 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:35:04.149660 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:35:04.149660 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:35:04.149660 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:35:04.149660 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:35:04.149660 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:35:04.149660 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:35:04.149660 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:35:04.149660 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:35:04.149660 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:35:04.149660 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:35:04.149660 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 31 09:35:04.149660 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 31 09:35:04.149660 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 31 09:35:04.151394 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 31 09:35:04.151394 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:35:04.151394 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:35:04.151394 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:35:04.151394 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:35:04.151394 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:35:04.151394 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:35:04.151394 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:35:04.151394 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application with Lock( Aug 31 09:35:04.151394 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:35:04.151394 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:35:04.151394 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:35:04.151394 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:35:04.151394 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:35:04.151394 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:35:04.151394 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:35:04.151394 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:35:04.153302 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 31 09:35:04.153302 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 31 09:35:04.153302 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 31 09:35:04.153302 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 31 09:35:04.153302 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 31 09:35:04.153302 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 31 09:35:04.153302 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 09:35:04.153302 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application Aug 31 09:35:04.153302 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 429/850] 10.222.0.13 () {56 vars in 1111 bytes} [Wed Aug 31 09:35:03 2022] GET /identity/v3/projects/service => generated 88 bytes in 299 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:35:04.154689 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a15e0c-e12e-4665-814a-557a86da418d None None] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:04.154689 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a15e0c-e12e-4665-814a-557a86da418d None None] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:04.155896 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a15e0c-e12e-4665-814a-557a86da418d None None] PATH_INFO: `/` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:04.157525 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 430/851] 10.222.0.13 () {54 vars in 927 bytes} [Wed Aug 31 09:35:04 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:35:04.163837 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f3d149-b4b0-49cf-83f7-f80606e76619 None None] REQUEST_METHOD: `POST` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:04.164294 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f3d149-b4b0-49cf-83f7-f80606e76619 None None] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:04.164655 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f3d149-b4b0-49cf-83f7-f80606e76619 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:04.188348 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.auth.core [None req-a5f3d149-b4b0-49cf-83f7-f80606e76619 None None] MFA Rules not processed for user `fd836cf590a145df9acf19cf3ae2ac56`. Rule list: `[]` (Enabled: `True`). {{(pid=88113) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:35:04.189662 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.fernet_utils [None req-a5f3d149-b4b0-49cf-83f7-f80606e76619 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:35:04.254465 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 422/852] 10.222.0.13 () {58 vars in 1024 bytes} [Wed Aug 31 09:35:04 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 31 09:35:04.255907 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fcff8c2-2f01-4ae4-b172-81003f57c45e None None] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:35:04.322010 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 431/853] 10.222.0.13 () {58 vars in 1024 bytes} [Wed Aug 31 09:35:04 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 09:35:04.460684 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fcff8c2-2f01-4ae4-b172-81003f57c45e None None] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:35:04.493669 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=100 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:35:04.517174 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fec8baa-e39e-418e-b9e4-02f6f7361688 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:35:04.517824 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fec8baa-e39e-418e-b9e4-02f6f7361688 None admin] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:04.518107 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fec8baa-e39e-418e-b9e4-02f6f7361688 None admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:04.518372 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fec8baa-e39e-418e-b9e4-02f6f7361688 None admin] PATH_INFO: `/v3/projects` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:04.519313 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fec8baa-e39e-418e-b9e4-02f6f7361688 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88114) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:35:04.519616 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fec8baa-e39e-418e-b9e4-02f6f7361688 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:35:04.521042 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fec8baa-e39e-418e-b9e4-02f6f7361688 None admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:35:04.535078 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 423/854] 10.222.0.13 () {56 vars in 1148 bytes} [Wed Aug 31 09:35:04 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 280 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:35:04.544186 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f3d149-b4b0-49cf-83f7-f80606e76619 None None] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:35:04.576269 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=100 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:35:04.659158 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fec8baa-e39e-418e-b9e4-02f6f7361688 None admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:35:04.683772 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f3d149-b4b0-49cf-83f7-f80606e76619 None None] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:35:04.720462 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6767ee1-a6fd-4790-b5c3-335708d8653d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:35:04.721081 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6767ee1-a6fd-4790-b5c3-335708d8653d None admin] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:04.721359 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6767ee1-a6fd-4790-b5c3-335708d8653d None admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:04.721653 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6767ee1-a6fd-4790-b5c3-335708d8653d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:04.722406 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6767ee1-a6fd-4790-b5c3-335708d8653d None admin] RBAC: Adding query filter params (role.id=e24ba4fd91584883af350cc84bc96c1d, scope.project.id=49548eb44d62415582af820ab66decee, user.id=768cb67c1ef24251aaa113c12d81a250) {{(pid=88113) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:35:04.722711 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6767ee1-a6fd-4790-b5c3-335708d8653d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:35:04.724140 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6767ee1-a6fd-4790-b5c3-335708d8653d None admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:35:04.730148 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 432/855] 10.222.0.13 () {56 vars in 1374 bytes} [Wed Aug 31 09:35:04 2022] GET /identity/v3/role_assignments?user.id=768cb67c1ef24251aaa113c12d81a250&scope.project.id=49548eb44d62415582af820ab66decee&role.id=e24ba4fd91584883af350cc84bc96c1d => generated 608 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:35:04.730952 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6767ee1-a6fd-4790-b5c3-335708d8653d None admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:35:04.871028 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fec8baa-e39e-418e-b9e4-02f6f7361688 None admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:35:04.901092 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=100 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:35:04.936450 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf2899c-b6ff-43f9-9535-15d031bae68d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:35:04.936901 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf2899c-b6ff-43f9-9535-15d031bae68d None admin] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:04.937142 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf2899c-b6ff-43f9-9535-15d031bae68d None admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:04.937469 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf2899c-b6ff-43f9-9535-15d031bae68d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:04.939581 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6767ee1-a6fd-4790-b5c3-335708d8653d None admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:35:04.951667 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faf2899c-b6ff-43f9-9535-15d031bae68d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 09:35:04.957572 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application [None req-faf2899c-b6ff-43f9-9535-15d031bae68d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 09:35:04.957572 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:35:04.957572 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:35:04.957572 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:35:04.957572 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:35:04.957572 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:35:04.957572 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:35:04.957572 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:35:04.957572 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:35:04.957572 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:35:04.957572 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:35:04.957572 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:35:04.957572 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 09:35:04.957572 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 09:35:04.957572 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 09:35:04.957572 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 09:35:04.959329 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 09:35:04.959329 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 09:35:04.959329 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 09:35:04.959329 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 09:35:04.959329 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:35:04.959329 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:35:04.959329 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:35:04.959329 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:35:04.959329 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:35:04.959329 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:35:04.959329 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:35:04.959329 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application with Lock( Aug 31 09:35:04.959329 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:35:04.959329 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:35:04.959329 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:35:04.959329 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:35:04.960992 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:35:04.960992 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:35:04.960992 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:35:04.960992 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:35:04.960992 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 09:35:04.960992 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 09:35:04.960992 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 09:35:04.960992 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 09:35:04.960992 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 09:35:04.960992 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 09:35:04.960992 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 09:35:04.960992 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application Aug 31 09:35:04.960992 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 424/856] 10.222.0.13 () {56 vars in 1096 bytes} [Wed Aug 31 09:35:04 2022] GET /identity/v3/roles/admin => generated 83 bytes in 302 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:35:04.965872 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf2899c-b6ff-43f9-9535-15d031bae68d None admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:35:05.033326 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73e216c1-bf2a-499b-9636-7109203d623c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:35:05.034366 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e216c1-bf2a-499b-9636-7109203d623c None admin] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:05.034879 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e216c1-bf2a-499b-9636-7109203d623c None admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:05.035303 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e216c1-bf2a-499b-9636-7109203d623c None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:05.036277 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73e216c1-bf2a-499b-9636-7109203d623c None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:35:05.038509 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73e216c1-bf2a-499b-9636-7109203d623c None admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:35:05.052095 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=100 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:35:05.056048 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application [None req-73e216c1-bf2a-499b-9636-7109203d623c None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 31 09:35:05.056048 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:35:05.056048 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 31 09:35:05.056048 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 31 09:35:05.056048 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 31 09:35:05.056048 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 31 09:35:05.056048 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 31 09:35:05.056048 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 09:35:05.056048 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 31 09:35:05.056048 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application Aug 31 09:35:05.056048 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 31 09:35:05.056048 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application Aug 31 09:35:05.056048 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:35:05.056048 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:35:05.056048 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:35:05.056048 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:35:05.058363 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:35:05.058363 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:35:05.058363 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:35:05.058363 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:35:05.058363 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:35:05.058363 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:35:05.058363 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:35:05.058363 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 31 09:35:05.058363 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 31 09:35:05.058363 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 31 09:35:05.058363 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 31 09:35:05.058363 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:35:05.058363 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:35:05.058363 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:35:05.058363 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:35:05.058363 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:35:05.058363 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:35:05.060074 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:35:05.060074 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application with Lock( Aug 31 09:35:05.060074 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:35:05.060074 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:35:05.060074 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:35:05.060074 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:35:05.060074 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:35:05.060074 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:35:05.060074 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:35:05.060074 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:35:05.060074 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 31 09:35:05.060074 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 09:35:05.060074 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 31 09:35:05.060074 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application Aug 31 09:35:05.060074 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 433/857] 10.222.0.13 () {56 vars in 1132 bytes} [Wed Aug 31 09:35:04 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 327 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:35:05.067854 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73e216c1-bf2a-499b-9636-7109203d623c None admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:35:05.195146 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf2899c-b6ff-43f9-9535-15d031bae68d None admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:35:05.264854 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ea31e7-e00f-4e5e-a85b-24b36e4599b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:35:05.265870 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ea31e7-e00f-4e5e-a85b-24b36e4599b9 None admin] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:05.266340 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ea31e7-e00f-4e5e-a85b-24b36e4599b9 None admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:05.266812 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ea31e7-e00f-4e5e-a85b-24b36e4599b9 None admin] PATH_INFO: `/v3/roles` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:05.267185 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73e216c1-bf2a-499b-9636-7109203d623c None admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:35:05.267996 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78ea31e7-e00f-4e5e-a85b-24b36e4599b9 None admin] RBAC: Adding query filter params (name=admin) {{(pid=88114) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:35:05.268773 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78ea31e7-e00f-4e5e-a85b-24b36e4599b9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:35:05.271719 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78ea31e7-e00f-4e5e-a85b-24b36e4599b9 None admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:35:05.293749 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 425/858] 10.222.0.13 () {56 vars in 1099 bytes} [Wed Aug 31 09:35:04 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 323 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 09:35:05.297274 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ea31e7-e00f-4e5e-a85b-24b36e4599b9 None admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:35:05.335983 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-087bceea-faf2-4a4a-9dab-dcb3a4e0a656 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:35:05.337000 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087bceea-faf2-4a4a-9dab-dcb3a4e0a656 None admin] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:05.337527 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087bceea-faf2-4a4a-9dab-dcb3a4e0a656 None admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:05.337995 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087bceea-faf2-4a4a-9dab-dcb3a4e0a656 None admin] PATH_INFO: `/v3/services` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:05.339106 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-087bceea-faf2-4a4a-9dab-dcb3a4e0a656 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=88113) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:35:05.339771 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-087bceea-faf2-4a4a-9dab-dcb3a4e0a656 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:35:05.343424 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-087bceea-faf2-4a4a-9dab-dcb3a4e0a656 None admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:35:05.352938 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 434/859] 10.222.0.13 () {56 vars in 1126 bytes} [Wed Aug 31 09:35:05 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 286 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:35:05.363842 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-087bceea-faf2-4a4a-9dab-dcb3a4e0a656 None admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:35:05.397657 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=100 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:35:05.527894 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=100 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:35:05.538519 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ea31e7-e00f-4e5e-a85b-24b36e4599b9 None admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:35:05.605198 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-087bceea-faf2-4a4a-9dab-dcb3a4e0a656 None admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:35:05.605958 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f70e0f87-8635-48da-9572-8324c4bb7f79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:35:05.607032 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70e0f87-8635-48da-9572-8324c4bb7f79 None admin] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:05.607560 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70e0f87-8635-48da-9572-8324c4bb7f79 None admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:05.608048 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70e0f87-8635-48da-9572-8324c4bb7f79 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:05.627528 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f70e0f87-8635-48da-9572-8324c4bb7f79 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:35:05.630301 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f70e0f87-8635-48da-9572-8324c4bb7f79 None admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:35:05.658038 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application [None req-f70e0f87-8635-48da-9572-8324c4bb7f79 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 09:35:05.658038 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:35:05.658038 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:35:05.658038 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:35:05.658038 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:35:05.658038 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:35:05.658038 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:35:05.658038 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:35:05.658038 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:35:05.658038 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:35:05.658038 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:35:05.658038 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:35:05.658038 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 09:35:05.658038 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 09:35:05.658038 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 09:35:05.658038 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 09:35:05.660101 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:35:05.660101 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:35:05.660101 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:35:05.660101 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:35:05.660101 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:35:05.660101 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:35:05.660101 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:35:05.660101 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application with Lock( Aug 31 09:35:05.660101 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:35:05.660101 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:35:05.660101 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:35:05.660101 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:35:05.660101 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:35:05.660101 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:35:05.660101 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:35:05.660101 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:35:05.660101 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 09:35:05.662245 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 09:35:05.662245 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 09:35:05.662245 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application Aug 31 09:35:05.667594 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 426/860] 10.222.0.13 () {56 vars in 1108 bytes} [Wed Aug 31 09:35:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 372 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:35:05.669986 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8b3390-aeca-4b6a-aef4-55553634c3e0 None None] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:05.671243 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8b3390-aeca-4b6a-aef4-55553634c3e0 None None] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:05.671736 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8b3390-aeca-4b6a-aef4-55553634c3e0 None None] PATH_INFO: `/` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:05.674180 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 427/861] 10.222.0.13 () {54 vars in 927 bytes} [Wed Aug 31 09:35:05 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:35:05.675463 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb8b3390-aeca-4b6a-aef4-55553634c3e0 None None] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:35:05.681352 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c55d38d-f3a9-4235-b0af-ac08d0f9424c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:35:05.682435 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c55d38d-f3a9-4235-b0af-ac08d0f9424c None admin] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:05.682925 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c55d38d-f3a9-4235-b0af-ac08d0f9424c None admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:05.683416 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c55d38d-f3a9-4235-b0af-ac08d0f9424c None admin] PATH_INFO: `/v3/services` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:05.684584 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c55d38d-f3a9-4235-b0af-ac08d0f9424c None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=88113) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:35:05.685128 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c55d38d-f3a9-4235-b0af-ac08d0f9424c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:35:05.687857 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c55d38d-f3a9-4235-b0af-ac08d0f9424c None admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:35:05.695699 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 435/862] 10.222.0.13 () {56 vars in 1126 bytes} [Wed Aug 31 09:35:05 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 334 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 09:35:05.698738 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b61a7b9-de7e-4f64-aae3-a0f96aa6083c None None] REQUEST_METHOD: `POST` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:05.699202 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b61a7b9-de7e-4f64-aae3-a0f96aa6083c None None] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:05.699582 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b61a7b9-de7e-4f64-aae3-a0f96aa6083c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:05.725291 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.auth.core [None req-8b61a7b9-de7e-4f64-aae3-a0f96aa6083c None None] MFA Rules not processed for user `fd836cf590a145df9acf19cf3ae2ac56`. Rule list: `[]` (Enabled: `True`). {{(pid=88113) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:35:05.727549 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.fernet_utils [None req-8b61a7b9-de7e-4f64-aae3-a0f96aa6083c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:35:05.831520 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 436/863] 10.222.0.13 () {58 vars in 1024 bytes} [Wed Aug 31 09:35:05 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 09:35:05.905250 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb8b3390-aeca-4b6a-aef4-55553634c3e0 None None] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:35:05.972693 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e0d69e8-97d3-4995-8531-225fbe035171 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:35:05.974102 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0d69e8-97d3-4995-8531-225fbe035171 None admin] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:05.977016 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0d69e8-97d3-4995-8531-225fbe035171 None admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:05.977016 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0d69e8-97d3-4995-8531-225fbe035171 None admin] PATH_INFO: `/v3/domains` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:05.977016 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e0d69e8-97d3-4995-8531-225fbe035171 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88114) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:35:05.977016 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e0d69e8-97d3-4995-8531-225fbe035171 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:35:05.979595 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e0d69e8-97d3-4995-8531-225fbe035171 None admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:35:06.006222 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 428/864] 10.222.0.13 () {56 vars in 1109 bytes} [Wed Aug 31 09:35:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 331 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:35:06.016592 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b61a7b9-de7e-4f64-aae3-a0f96aa6083c None None] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:35:06.022744 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=100 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:35:06.147630 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e0d69e8-97d3-4995-8531-225fbe035171 None admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:35:06.166252 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=100 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:35:06.202317 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b61a7b9-de7e-4f64-aae3-a0f96aa6083c None None] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:35:06.270650 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce192be-060a-4c42-978e-cac532f421e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:35:06.271914 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce192be-060a-4c42-978e-cac532f421e4 None admin] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:06.272160 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce192be-060a-4c42-978e-cac532f421e4 None admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:06.273037 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce192be-060a-4c42-978e-cac532f421e4 None admin] PATH_INFO: `/v3/users/swift` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:06.283979 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ce192be-060a-4c42-978e-cac532f421e4 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:35:06.286440 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ce192be-060a-4c42-978e-cac532f421e4 None admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:35:06.308021 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application [None req-3ce192be-060a-4c42-978e-cac532f421e4 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 31 09:35:06.308021 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:35:06.308021 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:35:06.308021 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:35:06.308021 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:35:06.308021 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:35:06.308021 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:35:06.308021 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:35:06.308021 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:35:06.308021 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:35:06.308021 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:35:06.308021 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:35:06.308021 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 31 09:35:06.308021 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 31 09:35:06.308021 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 31 09:35:06.308021 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 31 09:35:06.310001 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:35:06.310001 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:35:06.310001 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 31 09:35:06.310001 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 09:35:06.310001 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 31 09:35:06.310001 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 09:35:06.310001 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:35:06.310001 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:35:06.310001 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:35:06.310001 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:35:06.310001 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:35:06.310001 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application with Lock( Aug 31 09:35:06.310001 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:35:06.310001 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:35:06.310001 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:35:06.310001 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:35:06.310001 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:35:06.310001 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:35:06.311921 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:35:06.311921 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:35:06.311921 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 31 09:35:06.311921 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 31 09:35:06.311921 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 31 09:35:06.311921 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 31 09:35:06.311921 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 31 09:35:06.311921 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 31 09:35:06.311921 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 31 09:35:06.311921 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application Aug 31 09:35:06.318554 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 437/865] 10.222.0.13 () {56 vars in 1096 bytes} [Wed Aug 31 09:35:06 2022] GET /identity/v3/users/swift => generated 83 bytes in 295 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:35:06.319834 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce192be-060a-4c42-978e-cac532f421e4 None admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:35:06.342955 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e0d69e8-97d3-4995-8531-225fbe035171 None admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:35:06.409690 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73545835-ad35-40cf-969b-2a67f50243ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:35:06.412129 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73545835-ad35-40cf-969b-2a67f50243ca None admin] REQUEST_METHOD: `POST` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:06.412797 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=100 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:35:06.413241 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73545835-ad35-40cf-969b-2a67f50243ca None admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:06.414491 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73545835-ad35-40cf-969b-2a67f50243ca None admin] PATH_INFO: `/v3/services` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:06.416353 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73545835-ad35-40cf-969b-2a67f50243ca None admin] RBAC: Authorizing `identity:create_service()` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:35:06.420246 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73545835-ad35-40cf-969b-2a67f50243ca None admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:35:06.456270 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 429/866] 10.222.0.13 () {60 vars in 1141 bytes} [Wed Aug 31 09:35:06 2022] POST /identity/v3/services => generated 251 bytes in 310 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 09:35:06.478274 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce192be-060a-4c42-978e-cac532f421e4 None admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:35:06.567059 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc95ec8-e538-4900-81dc-a7defbc285a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:35:06.568749 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc95ec8-e538-4900-81dc-a7defbc285a2 None admin] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:06.569717 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc95ec8-e538-4900-81dc-a7defbc285a2 None admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:06.570641 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc95ec8-e538-4900-81dc-a7defbc285a2 None admin] PATH_INFO: `/v3/users` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:06.572446 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fc95ec8-e538-4900-81dc-a7defbc285a2 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=88113) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:35:06.573371 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fc95ec8-e538-4900-81dc-a7defbc285a2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:35:06.576641 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fc95ec8-e538-4900-81dc-a7defbc285a2 None admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:35:06.617286 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 438/867] 10.222.0.13 () {56 vars in 1135 bytes} [Wed Aug 31 09:35:06 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 298 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:35:06.627312 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73545835-ad35-40cf-969b-2a67f50243ca None admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:35:06.771106 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:35:06.837016 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73545835-ad35-40cf-969b-2a67f50243ca None admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:35:06.910914 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d16ddd-5099-4c3c-bc05-45f5337c72de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:35:06.911986 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d16ddd-5099-4c3c-bc05-45f5337c72de None admin] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:06.912588 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d16ddd-5099-4c3c-bc05-45f5337c72de None admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:06.913077 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d16ddd-5099-4c3c-bc05-45f5337c72de None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:07.059087 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4d16ddd-5099-4c3c-bc05-45f5337c72de None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:35:07.062685 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4d16ddd-5099-4c3c-bc05-45f5337c72de None admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:35:07.158203 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=100 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:35:07.198899 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 430/868] 10.222.0.13 () {58 vars in 1282 bytes} [Wed Aug 31 09:35:06 2022] GET /identity/v3/auth/tokens => generated 1262 bytes in 573 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 31 09:35:07.208993 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc95ec8-e538-4900-81dc-a7defbc285a2 None admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:35:07.251309 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e133071-bf1d-4148-9370-61b03e4e51f4 None None] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:07.251659 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e133071-bf1d-4148-9370-61b03e4e51f4 None None] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:07.251854 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e133071-bf1d-4148-9370-61b03e4e51f4 None None] PATH_INFO: `/` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:07.253700 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 431/869] 10.222.0.13 () {54 vars in 927 bytes} [Wed Aug 31 09:35:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:35:07.259638 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83c4ed7-3124-4574-a4da-ddb340ab9e4d None None] REQUEST_METHOD: `POST` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:07.259979 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83c4ed7-3124-4574-a4da-ddb340ab9e4d None None] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:07.260186 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83c4ed7-3124-4574-a4da-ddb340ab9e4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:07.294429 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.auth.core [None req-a83c4ed7-3124-4574-a4da-ddb340ab9e4d None None] MFA Rules not processed for user `fd836cf590a145df9acf19cf3ae2ac56`. Rule list: `[]` (Enabled: `True`). {{(pid=88114) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:35:07.297909 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.fernet_utils [None req-a83c4ed7-3124-4574-a4da-ddb340ab9e4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88114) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:35:07.399471 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc95ec8-e538-4900-81dc-a7defbc285a2 None admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:35:07.399742 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 432/870] 10.222.0.13 () {58 vars in 1024 bytes} [Wed Aug 31 09:35:07 2022] POST /identity/v3/auth/tokens => generated 1262 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 31 09:35:07.435527 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=100 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:35:07.469969 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d05116-ff96-4498-8034-abe14050d90d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:35:07.471159 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d05116-ff96-4498-8034-abe14050d90d None admin] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:07.471663 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d05116-ff96-4498-8034-abe14050d90d None admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:07.472161 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d05116-ff96-4498-8034-abe14050d90d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:07.492982 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69d05116-ff96-4498-8034-abe14050d90d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:35:07.495732 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69d05116-ff96-4498-8034-abe14050d90d None admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:35:07.526603 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application [None req-69d05116-ff96-4498-8034-abe14050d90d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 09:35:07.526603 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:35:07.526603 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:35:07.526603 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:35:07.526603 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:35:07.526603 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:35:07.526603 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:35:07.526603 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:35:07.526603 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:35:07.526603 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:35:07.526603 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:35:07.526603 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:35:07.526603 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 09:35:07.526603 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 09:35:07.526603 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 09:35:07.526603 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 09:35:07.527638 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:35:07.527638 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:35:07.527638 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:35:07.527638 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:35:07.527638 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:35:07.527638 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:35:07.527638 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:35:07.527638 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application with Lock( Aug 31 09:35:07.527638 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:35:07.527638 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:35:07.527638 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:35:07.527638 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:35:07.527638 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:35:07.527638 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:35:07.527638 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:35:07.527638 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:35:07.527638 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 09:35:07.528541 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 09:35:07.528541 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 09:35:07.528541 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application Aug 31 09:35:07.529148 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 439/871] 10.222.0.13 () {56 vars in 1108 bytes} [Wed Aug 31 09:35:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 321 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:35:07.538541 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a83c4ed7-3124-4574-a4da-ddb340ab9e4d None None] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:35:07.694066 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d05116-ff96-4498-8034-abe14050d90d None admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:35:07.721523 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=100 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:35:07.726390 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a83c4ed7-3124-4574-a4da-ddb340ab9e4d None None] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:35:07.771820 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ed0547-9f90-48f8-b472-da7d3fd102a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:35:07.772548 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ed0547-9f90-48f8-b472-da7d3fd102a4 None admin] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:07.772893 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ed0547-9f90-48f8-b472-da7d3fd102a4 None admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:07.773234 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ed0547-9f90-48f8-b472-da7d3fd102a4 None admin] PATH_INFO: `/v3/domains` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:07.773999 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7ed0547-9f90-48f8-b472-da7d3fd102a4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88114) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:35:07.774373 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7ed0547-9f90-48f8-b472-da7d3fd102a4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:35:07.775839 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7ed0547-9f90-48f8-b472-da7d3fd102a4 None admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:35:07.789717 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 433/872] 10.222.0.13 () {56 vars in 1109 bytes} [Wed Aug 31 09:35:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:35:07.798244 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ed0547-9f90-48f8-b472-da7d3fd102a4 None admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:35:07.883281 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d05116-ff96-4498-8034-abe14050d90d None admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:35:07.928071 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e030139e-fbb9-47dd-b411-9ac7d28521f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:35:07.928732 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e030139e-fbb9-47dd-b411-9ac7d28521f2 None admin] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:07.929015 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e030139e-fbb9-47dd-b411-9ac7d28521f2 None admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:07.929341 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e030139e-fbb9-47dd-b411-9ac7d28521f2 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:07.939767 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ed0547-9f90-48f8-b472-da7d3fd102a4 None admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:35:07.945779 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e030139e-fbb9-47dd-b411-9ac7d28521f2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 09:35:07.951340 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application [None req-e030139e-fbb9-47dd-b411-9ac7d28521f2 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 31 09:35:07.951340 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:35:07.951340 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:35:07.951340 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:35:07.951340 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:35:07.951340 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:35:07.951340 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:35:07.951340 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:35:07.951340 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:35:07.951340 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:35:07.951340 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:35:07.951340 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:35:07.951340 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 09:35:07.951340 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 09:35:07.951340 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 09:35:07.951340 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 09:35:07.953081 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 09:35:07.953081 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 09:35:07.953081 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 09:35:07.953081 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 09:35:07.953081 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:35:07.953081 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:35:07.953081 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:35:07.953081 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:35:07.953081 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:35:07.953081 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:35:07.953081 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:35:07.953081 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application with Lock( Aug 31 09:35:07.953081 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:35:07.953081 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:35:07.953081 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:35:07.953081 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:35:07.954072 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:35:07.954072 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:35:07.954072 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:35:07.954072 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:35:07.954072 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 09:35:07.954072 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 09:35:07.954072 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 09:35:07.954072 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 09:35:07.954072 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 09:35:07.954072 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 09:35:07.954072 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 31 09:35:07.954072 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application Aug 31 09:35:07.954072 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 440/873] 10.222.0.13 () {56 vars in 1120 bytes} [Wed Aug 31 09:35:07 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 259 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 09:35:07.954846 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9474b50a-4155-4a0d-82ae-b5055916f711 None None] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:07.954846 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9474b50a-4155-4a0d-82ae-b5055916f711 None None] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:07.954846 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9474b50a-4155-4a0d-82ae-b5055916f711 None None] PATH_INFO: `/` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:07.957427 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 441/874] 10.222.0.13 () {54 vars in 927 bytes} [Wed Aug 31 09:35:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:35:07.959096 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9474b50a-4155-4a0d-82ae-b5055916f711 None None] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:35:08.001562 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=100 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:35:08.002248 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e7683bf-aa96-4e72-87d4-4e47fe3ee898 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:35:08.003136 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7683bf-aa96-4e72-87d4-4e47fe3ee898 None admin] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:08.003707 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7683bf-aa96-4e72-87d4-4e47fe3ee898 None admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:08.004027 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7683bf-aa96-4e72-87d4-4e47fe3ee898 None admin] PATH_INFO: `/v3/projects/service` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:08.015570 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e7683bf-aa96-4e72-87d4-4e47fe3ee898 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:35:08.017296 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e7683bf-aa96-4e72-87d4-4e47fe3ee898 None admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:35:08.034252 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application [None req-9e7683bf-aa96-4e72-87d4-4e47fe3ee898 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 09:35:08.034252 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:35:08.034252 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:35:08.034252 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:35:08.034252 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:35:08.034252 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:35:08.034252 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:35:08.034252 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:35:08.034252 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:35:08.034252 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:35:08.034252 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:35:08.034252 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:35:08.034252 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 31 09:35:08.034252 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 31 09:35:08.034252 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 31 09:35:08.036002 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 31 09:35:08.036002 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:35:08.036002 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:35:08.036002 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:35:08.036002 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:35:08.036002 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:35:08.036002 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:35:08.036002 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:35:08.036002 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application with Lock( Aug 31 09:35:08.036002 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:35:08.036002 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:35:08.036002 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:35:08.036002 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:35:08.036002 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:35:08.036002 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:35:08.036002 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:35:08.036002 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:35:08.037762 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 31 09:35:08.037762 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 31 09:35:08.037762 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 31 09:35:08.037762 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 31 09:35:08.037762 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 31 09:35:08.037762 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 31 09:35:08.037762 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 09:35:08.037762 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application Aug 31 09:35:08.041654 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 434/875] 10.222.0.13 () {56 vars in 1111 bytes} [Wed Aug 31 09:35:07 2022] GET /identity/v3/projects/service => generated 88 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 09:35:08.043915 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc572b9-ce3e-4631-b591-173cda3a4b26 None None] REQUEST_METHOD: `POST` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:08.044271 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc572b9-ce3e-4631-b591-173cda3a4b26 None None] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:08.044600 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc572b9-ce3e-4631-b591-173cda3a4b26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:08.073416 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.auth.core [None req-1dc572b9-ce3e-4631-b591-173cda3a4b26 None None] MFA Rules not processed for user `fd836cf590a145df9acf19cf3ae2ac56`. Rule list: `[]` (Enabled: `True`). {{(pid=88114) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:35:08.075053 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.fernet_utils [None req-1dc572b9-ce3e-4631-b591-173cda3a4b26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88114) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:35:08.099366 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=100 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:35:08.131312 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9474b50a-4155-4a0d-82ae-b5055916f711 None None] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:35:08.187337 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 435/876] 10.222.0.13 () {58 vars in 1024 bytes} [Wed Aug 31 09:35:08 2022] POST /identity/v3/auth/tokens => generated 1262 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 09:35:08.188720 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc572b9-ce3e-4631-b591-173cda3a4b26 None None] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:35:08.200959 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f4027e-652e-4c8a-83e2-d8afc6a96960 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:35:08.202535 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f4027e-652e-4c8a-83e2-d8afc6a96960 None admin] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:08.203245 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f4027e-652e-4c8a-83e2-d8afc6a96960 None admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:08.203932 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f4027e-652e-4c8a-83e2-d8afc6a96960 None admin] PATH_INFO: `/v3/roles` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:08.205340 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2f4027e-652e-4c8a-83e2-d8afc6a96960 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=88113) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:35:08.206091 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2f4027e-652e-4c8a-83e2-d8afc6a96960 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:35:08.208937 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2f4027e-652e-4c8a-83e2-d8afc6a96960 None admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:35:08.232791 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 442/877] 10.222.0.13 () {56 vars in 1115 bytes} [Wed Aug 31 09:35:07 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 274 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:35:08.234689 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f4027e-652e-4c8a-83e2-d8afc6a96960 None admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:35:08.381060 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc572b9-ce3e-4631-b591-173cda3a4b26 None None] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:35:08.416836 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=100 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:35:08.434851 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f4027e-652e-4c8a-83e2-d8afc6a96960 None admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:35:08.456284 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92fb9d76-6e74-483b-900a-d08425f93b64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:35:08.457962 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92fb9d76-6e74-483b-900a-d08425f93b64 None admin] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:08.458611 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92fb9d76-6e74-483b-900a-d08425f93b64 None admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:08.459244 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92fb9d76-6e74-483b-900a-d08425f93b64 None admin] PATH_INFO: `/v3/projects` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:08.460819 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92fb9d76-6e74-483b-900a-d08425f93b64 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88114) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:35:08.461550 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92fb9d76-6e74-483b-900a-d08425f93b64 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:35:08.465049 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92fb9d76-6e74-483b-900a-d08425f93b64 None admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:35:08.489926 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 436/878] 10.222.0.13 () {56 vars in 1148 bytes} [Wed Aug 31 09:35:08 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 302 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:35:08.491259 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92fb9d76-6e74-483b-900a-d08425f93b64 None admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:35:08.500626 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c18e6b-c0c8-41e1-b49f-5915206de836 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:35:08.501296 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c18e6b-c0c8-41e1-b49f-5915206de836 None admin] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:08.501609 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c18e6b-c0c8-41e1-b49f-5915206de836 None admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:08.501888 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c18e6b-c0c8-41e1-b49f-5915206de836 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:08.509717 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=100 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:35:08.515457 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5c18e6b-c0c8-41e1-b49f-5915206de836 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:35:08.517096 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5c18e6b-c0c8-41e1-b49f-5915206de836 None admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:35:08.535703 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application [None req-c5c18e6b-c0c8-41e1-b49f-5915206de836 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 09:35:08.535703 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:35:08.535703 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:35:08.535703 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:35:08.535703 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:35:08.535703 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:35:08.535703 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:35:08.535703 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:35:08.535703 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:35:08.535703 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:35:08.535703 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:35:08.535703 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:35:08.535703 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 09:35:08.535703 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 09:35:08.535703 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 09:35:08.535703 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 09:35:08.537703 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:35:08.537703 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:35:08.537703 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:35:08.537703 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:35:08.537703 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:35:08.537703 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:35:08.537703 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:35:08.537703 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application with Lock( Aug 31 09:35:08.537703 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:35:08.537703 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:35:08.537703 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:35:08.537703 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:35:08.537703 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:35:08.537703 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:35:08.537703 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:35:08.537703 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:35:08.537703 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 09:35:08.539403 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 09:35:08.539403 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 09:35:08.539403 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application Aug 31 09:35:08.542329 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 443/879] 10.222.0.13 () {56 vars in 1108 bytes} [Wed Aug 31 09:35:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 308 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:35:08.542836 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c18e6b-c0c8-41e1-b49f-5915206de836 None admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:35:08.687167 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92fb9d76-6e74-483b-900a-d08425f93b64 None admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:35:08.709886 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c18e6b-c0c8-41e1-b49f-5915206de836 None admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:35:08.753368 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41165694-d424-4889-84a7-6cff526dd289 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:35:08.754627 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41165694-d424-4889-84a7-6cff526dd289 None admin] REQUEST_METHOD: `POST` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:08.755090 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b57d40f-eb06-4851-9715-95d5f013d6b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:35:08.755368 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41165694-d424-4889-84a7-6cff526dd289 None admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:08.755876 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b57d40f-eb06-4851-9715-95d5f013d6b9 None admin] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:08.756090 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41165694-d424-4889-84a7-6cff526dd289 None admin] PATH_INFO: `/v3/services` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:08.756323 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b57d40f-eb06-4851-9715-95d5f013d6b9 None admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:08.756709 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b57d40f-eb06-4851-9715-95d5f013d6b9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:08.757585 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b57d40f-eb06-4851-9715-95d5f013d6b9 None admin] RBAC: Adding query filter params (role.id=5d4ee63a3add47fc9a6b8b16d39e0a1f, scope.project.id=49548eb44d62415582af820ab66decee, user.id=202bb9db7a8a4977ba788a290ed9a3fa) {{(pid=88113) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:35:08.757793 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41165694-d424-4889-84a7-6cff526dd289 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:35:08.758076 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b57d40f-eb06-4851-9715-95d5f013d6b9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:35:08.759613 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b57d40f-eb06-4851-9715-95d5f013d6b9 None admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:35:08.760695 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41165694-d424-4889-84a7-6cff526dd289 None admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:35:08.778089 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 444/880] 10.222.0.13 () {56 vars in 1374 bytes} [Wed Aug 31 09:35:08 2022] GET /identity/v3/role_assignments?user.id=202bb9db7a8a4977ba788a290ed9a3fa&scope.project.id=49548eb44d62415582af820ab66decee&role.id=5d4ee63a3add47fc9a6b8b16d39e0a1f => generated 608 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:35:08.778089 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b57d40f-eb06-4851-9715-95d5f013d6b9 None admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:35:08.799461 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=100 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:35:08.803714 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 437/881] 10.222.0.13 () {60 vars in 1141 bytes} [Wed Aug 31 09:35:08 2022] POST /identity/v3/services => generated 269 bytes in 313 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 09:35:08.847574 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d59f8e-838a-4e0c-a3d4-cbfb1d94a5ed None None] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:08.847837 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d59f8e-838a-4e0c-a3d4-cbfb1d94a5ed None None] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:08.848124 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d59f8e-838a-4e0c-a3d4-cbfb1d94a5ed None None] PATH_INFO: `/` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:08.853986 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 438/882] 10.222.0.13 () {54 vars in 927 bytes} [Wed Aug 31 09:35:08 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 31 09:35:08.855439 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708a6e1c-4c49-48ce-ae13-accb5da5c5a3 None None] REQUEST_METHOD: `POST` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:08.855729 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708a6e1c-4c49-48ce-ae13-accb5da5c5a3 None None] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:08.855992 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708a6e1c-4c49-48ce-ae13-accb5da5c5a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:08.878689 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.auth.core [None req-708a6e1c-4c49-48ce-ae13-accb5da5c5a3 None None] MFA Rules not processed for user `fd836cf590a145df9acf19cf3ae2ac56`. Rule list: `[]` (Enabled: `True`). {{(pid=88114) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:35:08.879986 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.fernet_utils [None req-708a6e1c-4c49-48ce-ae13-accb5da5c5a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88114) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:35:08.999765 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b57d40f-eb06-4851-9715-95d5f013d6b9 None admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:35:09.009143 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:35:09.037410 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 439/883] 10.222.0.13 () {58 vars in 1024 bytes} [Wed Aug 31 09:35:08 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 09:35:09.060104 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec93438-ee11-4e6d-9985-37bc1a0aa06a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:35:09.061768 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec93438-ee11-4e6d-9985-37bc1a0aa06a None admin] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:09.062221 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec93438-ee11-4e6d-9985-37bc1a0aa06a None admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:09.062662 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec93438-ee11-4e6d-9985-37bc1a0aa06a None admin] PATH_INFO: `/v3/domains` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:09.063874 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ec93438-ee11-4e6d-9985-37bc1a0aa06a None admin] RBAC: Adding query filter params (name=Default) {{(pid=88113) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:35:09.064368 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ec93438-ee11-4e6d-9985-37bc1a0aa06a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:35:09.066790 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ec93438-ee11-4e6d-9985-37bc1a0aa06a None admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:35:09.087408 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 445/884] 10.222.0.13 () {56 vars in 1109 bytes} [Wed Aug 31 09:35:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 320 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:35:09.097213 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-708a6e1c-4c49-48ce-ae13-accb5da5c5a3 None None] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:35:09.287603 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-708a6e1c-4c49-48ce-ae13-accb5da5c5a3 None None] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:35:09.350936 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf571a99-f5b2-4561-b0c4-2eb6600e7820 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:35:09.351999 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf571a99-f5b2-4561-b0c4-2eb6600e7820 None admin] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:09.352455 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf571a99-f5b2-4561-b0c4-2eb6600e7820 None admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:09.352942 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf571a99-f5b2-4561-b0c4-2eb6600e7820 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:09.366744 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf571a99-f5b2-4561-b0c4-2eb6600e7820 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:35:09.369316 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf571a99-f5b2-4561-b0c4-2eb6600e7820 None admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:35:09.392609 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application [None req-bf571a99-f5b2-4561-b0c4-2eb6600e7820 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 31 09:35:09.392609 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:35:09.392609 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:35:09.392609 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:35:09.392609 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:35:09.392609 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:35:09.392609 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:35:09.392609 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:35:09.392609 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:35:09.392609 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:35:09.392609 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:35:09.392609 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:35:09.392609 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 31 09:35:09.392609 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 31 09:35:09.392609 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 31 09:35:09.392609 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 31 09:35:09.393914 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:35:09.393914 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:35:09.393914 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 31 09:35:09.393914 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 09:35:09.393914 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 31 09:35:09.393914 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 09:35:09.393914 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:35:09.393914 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:35:09.393914 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:35:09.393914 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:35:09.393914 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:35:09.393914 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application with Lock( Aug 31 09:35:09.393914 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:35:09.393914 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:35:09.393914 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:35:09.393914 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:35:09.393914 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:35:09.393914 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:35:09.394862 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:35:09.394862 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:35:09.394862 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 31 09:35:09.394862 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 31 09:35:09.394862 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 31 09:35:09.394862 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 31 09:35:09.394862 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 31 09:35:09.394862 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 31 09:35:09.394862 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 31 09:35:09.394862 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application Aug 31 09:35:09.395535 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 440/885] 10.222.0.13 () {56 vars in 1117 bytes} [Wed Aug 31 09:35:09 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 299 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:35:09.405221 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec93438-ee11-4e6d-9985-37bc1a0aa06a None admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:35:09.491675 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf571a99-f5b2-4561-b0c4-2eb6600e7820 None admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:35:09.538653 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=100 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:35:09.612903 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec93438-ee11-4e6d-9985-37bc1a0aa06a None admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:35:09.628453 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=100 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:35:09.676962 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa4c238-8232-4230-b405-0c3028c210d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:35:09.677979 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa4c238-8232-4230-b405-0c3028c210d2 None admin] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:09.678441 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa4c238-8232-4230-b405-0c3028c210d2 None admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:09.678890 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa4c238-8232-4230-b405-0c3028c210d2 None admin] PATH_INFO: `/v3/users` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:09.680168 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fa4c238-8232-4230-b405-0c3028c210d2 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=88113) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:35:09.680936 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fa4c238-8232-4230-b405-0c3028c210d2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:35:09.683641 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fa4c238-8232-4230-b405-0c3028c210d2 None admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:35:09.714714 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 446/886] 10.222.0.13 () {56 vars in 1149 bytes} [Wed Aug 31 09:35:09 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 311 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:35:09.725531 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa4c238-8232-4230-b405-0c3028c210d2 None admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:35:09.735230 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf571a99-f5b2-4561-b0c4-2eb6600e7820 None admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:35:09.806177 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b71a1bbb-eab7-494e-8c3a-b75d89ce8b82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:35:09.807327 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71a1bbb-eab7-494e-8c3a-b75d89ce8b82 None admin] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:09.807900 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71a1bbb-eab7-494e-8c3a-b75d89ce8b82 None admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:09.808470 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71a1bbb-eab7-494e-8c3a-b75d89ce8b82 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:09.809731 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b71a1bbb-eab7-494e-8c3a-b75d89ce8b82 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:35:09.813146 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b71a1bbb-eab7-494e-8c3a-b75d89ce8b82 None admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:35:09.832475 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application [None req-b71a1bbb-eab7-494e-8c3a-b75d89ce8b82 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 31 09:35:09.832475 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:35:09.832475 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 31 09:35:09.832475 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 31 09:35:09.832475 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 31 09:35:09.832475 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 31 09:35:09.832475 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 31 09:35:09.832475 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 09:35:09.832475 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 31 09:35:09.832475 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application Aug 31 09:35:09.832475 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 31 09:35:09.832475 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application Aug 31 09:35:09.832475 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:35:09.832475 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:35:09.832475 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:35:09.832475 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:35:09.832475 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:35:09.834655 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:35:09.834655 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:35:09.834655 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:35:09.834655 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:35:09.834655 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:35:09.834655 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:35:09.834655 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 31 09:35:09.834655 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 31 09:35:09.834655 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 31 09:35:09.834655 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 31 09:35:09.834655 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:35:09.834655 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:35:09.834655 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:35:09.834655 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:35:09.834655 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:35:09.834655 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:35:09.836557 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:35:09.836557 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application with Lock( Aug 31 09:35:09.836557 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:35:09.836557 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:35:09.836557 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:35:09.836557 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:35:09.836557 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:35:09.836557 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:35:09.836557 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:35:09.836557 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:35:09.836557 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 31 09:35:09.836557 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 09:35:09.836557 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 31 09:35:09.836557 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application Aug 31 09:35:09.847204 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 441/887] 10.222.0.13 () {56 vars in 1114 bytes} [Wed Aug 31 09:35:09 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 356 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:35:09.852858 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b71a1bbb-eab7-494e-8c3a-b75d89ce8b82 None admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:35:09.914278 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa4c238-8232-4230-b405-0c3028c210d2 None admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:35:09.948278 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=100 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:35:09.960684 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac456620-2480-45d8-b4b5-c298823f248a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:35:09.961601 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac456620-2480-45d8-b4b5-c298823f248a None admin] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:09.962549 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac456620-2480-45d8-b4b5-c298823f248a None admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:09.962981 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac456620-2480-45d8-b4b5-c298823f248a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:10.054976 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b71a1bbb-eab7-494e-8c3a-b75d89ce8b82 None admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:35:10.055926 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac456620-2480-45d8-b4b5-c298823f248a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:35:10.057664 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac456620-2480-45d8-b4b5-c298823f248a None admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:35:10.097850 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=100 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:35:10.125567 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d536e2b8-27fb-4bbb-8386-0f98efc191a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:35:10.126528 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d536e2b8-27fb-4bbb-8386-0f98efc191a1 None admin] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:10.127143 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d536e2b8-27fb-4bbb-8386-0f98efc191a1 None admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:10.127633 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d536e2b8-27fb-4bbb-8386-0f98efc191a1 None admin] PATH_INFO: `/v3/services` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:10.128944 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d536e2b8-27fb-4bbb-8386-0f98efc191a1 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=88114) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:35:10.129554 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d536e2b8-27fb-4bbb-8386-0f98efc191a1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:35:10.132556 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d536e2b8-27fb-4bbb-8386-0f98efc191a1 None admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:35:10.150087 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 442/888] 10.222.0.13 () {56 vars in 1114 bytes} [Wed Aug 31 09:35:09 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 298 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 09:35:10.155097 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d536e2b8-27fb-4bbb-8386-0f98efc191a1 None admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:35:10.194553 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 447/889] 10.222.0.13 () {58 vars in 1282 bytes} [Wed Aug 31 09:35:09 2022] GET /identity/v3/auth/tokens => generated 1372 bytes in 471 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 31 09:35:10.202170 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac456620-2480-45d8-b4b5-c298823f248a None admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:35:10.254599 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=100 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:35:10.313515 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d536e2b8-27fb-4bbb-8386-0f98efc191a1 None admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:35:10.370215 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97bbb629-cedd-4a74-9e44-3f9f9b475f4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:35:10.371226 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97bbb629-cedd-4a74-9e44-3f9f9b475f4f None admin] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:10.371839 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97bbb629-cedd-4a74-9e44-3f9f9b475f4f None admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:10.372311 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97bbb629-cedd-4a74-9e44-3f9f9b475f4f None admin] PATH_INFO: `/v3/services` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:10.374385 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97bbb629-cedd-4a74-9e44-3f9f9b475f4f None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=88114) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:35:10.374922 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97bbb629-cedd-4a74-9e44-3f9f9b475f4f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:35:10.377585 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97bbb629-cedd-4a74-9e44-3f9f9b475f4f None admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:35:10.385511 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac456620-2480-45d8-b4b5-c298823f248a None admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:35:10.394217 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 443/890] 10.222.0.13 () {56 vars in 1114 bytes} [Wed Aug 31 09:35:10 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:35:10.398660 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97bbb629-cedd-4a74-9e44-3f9f9b475f4f None admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:35:10.437130 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64fb0f26-185a-4bd6-bb5e-821c3f5c4720 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:35:10.438191 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64fb0f26-185a-4bd6-bb5e-821c3f5c4720 None admin] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:10.438674 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64fb0f26-185a-4bd6-bb5e-821c3f5c4720 None admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:10.439178 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64fb0f26-185a-4bd6-bb5e-821c3f5c4720 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:10.453704 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64fb0f26-185a-4bd6-bb5e-821c3f5c4720 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:35:10.455814 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64fb0f26-185a-4bd6-bb5e-821c3f5c4720 None admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:35:10.475963 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application [None req-64fb0f26-185a-4bd6-bb5e-821c3f5c4720 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 09:35:10.475963 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:35:10.475963 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:35:10.475963 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:35:10.475963 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:35:10.475963 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:35:10.475963 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:35:10.475963 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:35:10.475963 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:35:10.475963 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:35:10.475963 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:35:10.475963 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:35:10.475963 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 09:35:10.475963 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 09:35:10.475963 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 09:35:10.475963 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 09:35:10.476983 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:35:10.476983 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:35:10.476983 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:35:10.476983 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:35:10.476983 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:35:10.476983 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:35:10.476983 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:35:10.476983 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application with Lock( Aug 31 09:35:10.476983 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:35:10.476983 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:35:10.476983 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:35:10.476983 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:35:10.476983 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:35:10.476983 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:35:10.476983 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:35:10.476983 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:35:10.476983 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 09:35:10.477931 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 09:35:10.477931 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 09:35:10.477931 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application Aug 31 09:35:10.486016 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 448/891] 10.222.0.13 () {56 vars in 1108 bytes} [Wed Aug 31 09:35:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 280 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 09:35:10.487377 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64fb0f26-185a-4bd6-bb5e-821c3f5c4720 None admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:35:10.507895 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=100 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:35:10.599121 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=100 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:35:10.624145 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97bbb629-cedd-4a74-9e44-3f9f9b475f4f None admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:35:10.627986 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64fb0f26-185a-4bd6-bb5e-821c3f5c4720 None admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:35:10.702604 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b79550-c23c-4947-9219-e12f8033c969 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:35:10.703522 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-245abb0e-55ee-4c8a-bc9b-d76a966f5a40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:35:10.703703 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b79550-c23c-4947-9219-e12f8033c969 None admin] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:10.704025 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b79550-c23c-4947-9219-e12f8033c969 None admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:10.704487 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b79550-c23c-4947-9219-e12f8033c969 None admin] PATH_INFO: `/v3/endpoints` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:10.705568 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245abb0e-55ee-4c8a-bc9b-d76a966f5a40 None admin] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:10.705752 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21b79550-c23c-4947-9219-e12f8033c969 None admin] RBAC: Adding query filter params (interface=public, service_id=4655cc1266704e2bbdc78504696a1143, region_id=RegionOne) {{(pid=88114) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:35:10.706260 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21b79550-c23c-4947-9219-e12f8033c969 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:35:10.706761 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245abb0e-55ee-4c8a-bc9b-d76a966f5a40 None admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:10.707454 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245abb0e-55ee-4c8a-bc9b-d76a966f5a40 None admin] PATH_INFO: `/v3/domains` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:10.708695 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21b79550-c23c-4947-9219-e12f8033c969 None admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:35:10.709045 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-245abb0e-55ee-4c8a-bc9b-d76a966f5a40 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88113) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:35:10.709839 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-245abb0e-55ee-4c8a-bc9b-d76a966f5a40 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:35:10.712776 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-245abb0e-55ee-4c8a-bc9b-d76a966f5a40 None admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:35:10.721636 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 444/892] 10.222.0.13 () {56 vars in 1251 bytes} [Wed Aug 31 09:35:10 2022] GET /identity/v3/endpoints?service_id=4655cc1266704e2bbdc78504696a1143&interface=public®ion_id=RegionOne => generated 194 bytes in 324 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:35:10.733107 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b79550-c23c-4947-9219-e12f8033c969 None admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:35:10.739340 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 449/893] 10.222.0.13 () {56 vars in 1109 bytes} [Wed Aug 31 09:35:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:35:10.750123 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-245abb0e-55ee-4c8a-bc9b-d76a966f5a40 None admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:35:10.945049 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b79550-c23c-4947-9219-e12f8033c969 None admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:35:10.962520 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-245abb0e-55ee-4c8a-bc9b-d76a966f5a40 None admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:35:10.981087 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=100 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:35:11.023314 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-316c7f35-04a5-4ffe-b5cb-eda4a1d346f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:35:11.024388 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316c7f35-04a5-4ffe-b5cb-eda4a1d346f7 None admin] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:11.024850 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316c7f35-04a5-4ffe-b5cb-eda4a1d346f7 None admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:11.025273 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316c7f35-04a5-4ffe-b5cb-eda4a1d346f7 None admin] PATH_INFO: `/v3/services` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:11.025941 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f20f67-662d-47fb-9e72-4952afb2bed6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:35:11.026328 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-316c7f35-04a5-4ffe-b5cb-eda4a1d346f7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:35:11.026999 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f20f67-662d-47fb-9e72-4952afb2bed6 None admin] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:11.027379 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f20f67-662d-47fb-9e72-4952afb2bed6 None admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:11.027750 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f20f67-662d-47fb-9e72-4952afb2bed6 None admin] PATH_INFO: `/v3/projects/service` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:11.028424 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-316c7f35-04a5-4ffe-b5cb-eda4a1d346f7 None admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:35:11.044105 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81f20f67-662d-47fb-9e72-4952afb2bed6 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:35:11.044105 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81f20f67-662d-47fb-9e72-4952afb2bed6 None admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:35:11.045556 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=100 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:35:11.046834 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 445/894] 10.222.0.13 () {56 vars in 1087 bytes} [Wed Aug 31 09:35:10 2022] GET /identity/v3/services => generated 1268 bytes in 316 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 09:35:11.064129 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application [None req-81f20f67-662d-47fb-9e72-4952afb2bed6 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 09:35:11.064129 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:35:11.064129 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:35:11.064129 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:35:11.064129 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:35:11.064129 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:35:11.064129 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:35:11.064129 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:35:11.064129 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:35:11.064129 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:35:11.064129 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:35:11.064129 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:35:11.064129 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 31 09:35:11.064129 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 31 09:35:11.064129 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 31 09:35:11.065310 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 31 09:35:11.065310 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:35:11.065310 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:35:11.065310 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:35:11.065310 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:35:11.065310 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:35:11.065310 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:35:11.065310 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:35:11.065310 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application with Lock( Aug 31 09:35:11.065310 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:35:11.065310 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:35:11.065310 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:35:11.065310 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:35:11.065310 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:35:11.065310 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:35:11.065310 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:35:11.065310 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:35:11.066491 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 31 09:35:11.066491 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 31 09:35:11.066491 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 31 09:35:11.066491 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 31 09:35:11.066491 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 31 09:35:11.066491 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 31 09:35:11.066491 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 09:35:11.066491 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application Aug 31 09:35:11.068535 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 450/895] 10.222.0.13 () {56 vars in 1111 bytes} [Wed Aug 31 09:35:10 2022] GET /identity/v3/projects/service => generated 88 bytes in 319 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:35:11.074404 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-316c7f35-04a5-4ffe-b5cb-eda4a1d346f7 None admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:35:11.199803 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-316c7f35-04a5-4ffe-b5cb-eda4a1d346f7 None admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:35:11.242546 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc22a3a-9c0a-4f92-9591-8cee63f40a99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:35:11.243545 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc22a3a-9c0a-4f92-9591-8cee63f40a99 None admin] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:11.244022 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc22a3a-9c0a-4f92-9591-8cee63f40a99 None admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:11.244480 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc22a3a-9c0a-4f92-9591-8cee63f40a99 None admin] PATH_INFO: `/v3/projects` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:11.245404 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adc22a3a-9c0a-4f92-9591-8cee63f40a99 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88114) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:35:11.245928 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adc22a3a-9c0a-4f92-9591-8cee63f40a99 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:35:11.247592 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adc22a3a-9c0a-4f92-9591-8cee63f40a99 None admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:35:11.261789 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 446/896] 10.222.0.13 () {56 vars in 1148 bytes} [Wed Aug 31 09:35:11 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:35:11.269264 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f20f67-662d-47fb-9e72-4952afb2bed6 None admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:35:11.294227 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68732b24-7f29-4b78-a1c9-07feeb213e79 None None] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:11.295601 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68732b24-7f29-4b78-a1c9-07feeb213e79 None None] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:11.296511 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68732b24-7f29-4b78-a1c9-07feeb213e79 None None] PATH_INFO: `/` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:11.299897 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 447/897] 10.222.0.13 () {54 vars in 927 bytes} [Wed Aug 31 09:35:11 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 31 09:35:11.306507 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1c2178-5bcb-4db9-b9ba-f59ddc2295fc None None] REQUEST_METHOD: `POST` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:11.306798 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1c2178-5bcb-4db9-b9ba-f59ddc2295fc None None] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:11.307066 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1c2178-5bcb-4db9-b9ba-f59ddc2295fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:11.331886 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.auth.core [None req-6a1c2178-5bcb-4db9-b9ba-f59ddc2295fc None None] MFA Rules not processed for user `fd836cf590a145df9acf19cf3ae2ac56`. Rule list: `[]` (Enabled: `True`). {{(pid=88114) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:35:11.333127 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.fernet_utils [None req-6a1c2178-5bcb-4db9-b9ba-f59ddc2295fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88114) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:35:11.386426 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=100 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:35:11.465192 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f20f67-662d-47fb-9e72-4952afb2bed6 None admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:35:11.469265 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 448/898] 10.222.0.13 () {58 vars in 1024 bytes} [Wed Aug 31 09:35:11 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 31 09:35:11.538819 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b0d0bd-98bc-4abe-b948-026ac4da6d85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:35:11.539756 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b0d0bd-98bc-4abe-b948-026ac4da6d85 None admin] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:11.540182 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b0d0bd-98bc-4abe-b948-026ac4da6d85 None admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:11.540644 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b0d0bd-98bc-4abe-b948-026ac4da6d85 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:11.542014 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24b0d0bd-98bc-4abe-b948-026ac4da6d85 None admin] RBAC: Adding query filter params (role.id=d41c162f8e1748989a09382ced5ef3a6, scope.project.id=49548eb44d62415582af820ab66decee, user.id=768cb67c1ef24251aaa113c12d81a250) {{(pid=88113) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:35:11.542611 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24b0d0bd-98bc-4abe-b948-026ac4da6d85 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:35:11.545921 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24b0d0bd-98bc-4abe-b948-026ac4da6d85 None admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:35:11.557888 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 451/899] 10.222.0.13 () {56 vars in 1374 bytes} [Wed Aug 31 09:35:11 2022] GET /identity/v3/role_assignments?user.id=768cb67c1ef24251aaa113c12d81a250&scope.project.id=49548eb44d62415582af820ab66decee&role.id=d41c162f8e1748989a09382ced5ef3a6 => generated 259 bytes in 289 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:35:11.832874 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a1c2178-5bcb-4db9-b9ba-f59ddc2295fc None None] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:35:11.965899 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41005c77-8e11-4859-8ed3-c3e0c4beeacc None None] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:11.969093 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41005c77-8e11-4859-8ed3-c3e0c4beeacc None None] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:11.969093 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41005c77-8e11-4859-8ed3-c3e0c4beeacc None None] PATH_INFO: `/` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:11.975015 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 452/900] 10.222.0.13 () {54 vars in 927 bytes} [Wed Aug 31 09:35:11 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:35:11.981076 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc96746-52bb-4a2c-ac39-ef17c2d9620a None None] REQUEST_METHOD: `POST` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:11.981687 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc96746-52bb-4a2c-ac39-ef17c2d9620a None None] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:11.982273 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc96746-52bb-4a2c-ac39-ef17c2d9620a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:12.002100 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.auth.core [None req-edc96746-52bb-4a2c-ac39-ef17c2d9620a None None] MFA Rules not processed for user `fd836cf590a145df9acf19cf3ae2ac56`. Rule list: `[]` (Enabled: `True`). {{(pid=88113) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:35:12.003886 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.fernet_utils [None req-edc96746-52bb-4a2c-ac39-ef17c2d9620a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:35:12.018574 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a1c2178-5bcb-4db9-b9ba-f59ddc2295fc None None] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:35:12.075584 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c77589c-7d8e-4cc2-8041-ec7a41dd0fa1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:35:12.076855 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=100 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:35:12.077115 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c77589c-7d8e-4cc2-8041-ec7a41dd0fa1 None admin] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:12.077423 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c77589c-7d8e-4cc2-8041-ec7a41dd0fa1 None admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:12.078483 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c77589c-7d8e-4cc2-8041-ec7a41dd0fa1 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:12.079124 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c77589c-7d8e-4cc2-8041-ec7a41dd0fa1 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:35:12.080541 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c77589c-7d8e-4cc2-8041-ec7a41dd0fa1 None admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:35:12.084860 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 453/901] 10.222.0.13 () {58 vars in 1024 bytes} [Wed Aug 31 09:35:11 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 09:35:12.093518 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application [None req-4c77589c-7d8e-4cc2-8041-ec7a41dd0fa1 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 31 09:35:12.093518 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:35:12.093518 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 31 09:35:12.093518 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 31 09:35:12.093518 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 31 09:35:12.093518 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 31 09:35:12.093518 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 31 09:35:12.093518 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 09:35:12.093518 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 31 09:35:12.093518 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application Aug 31 09:35:12.093518 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 31 09:35:12.093518 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application Aug 31 09:35:12.093518 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:35:12.093518 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:35:12.093518 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:35:12.093518 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:35:12.094608 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:35:12.094608 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:35:12.094608 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:35:12.094608 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:35:12.094608 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:35:12.094608 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:35:12.094608 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:35:12.094608 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 31 09:35:12.094608 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 31 09:35:12.094608 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 31 09:35:12.094608 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 31 09:35:12.094608 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:35:12.094608 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:35:12.094608 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:35:12.094608 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:35:12.094608 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:35:12.094608 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:35:12.095541 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:35:12.095541 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application with Lock( Aug 31 09:35:12.095541 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:35:12.095541 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:35:12.095541 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:35:12.095541 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:35:12.095541 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:35:12.095541 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:35:12.095541 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:35:12.095541 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:35:12.095541 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 31 09:35:12.095541 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 09:35:12.095541 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 31 09:35:12.095541 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application Aug 31 09:35:12.098438 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 449/902] 10.222.0.13 () {56 vars in 1132 bytes} [Wed Aug 31 09:35:11 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 266 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:35:12.106524 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc96746-52bb-4a2c-ac39-ef17c2d9620a None None] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:35:12.224132 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc96746-52bb-4a2c-ac39-ef17c2d9620a None None] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:35:12.282732 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a37beba0-5c73-4fdf-bc42-cbf49e57d5ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:35:12.284004 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37beba0-5c73-4fdf-bc42-cbf49e57d5ee None admin] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:12.284617 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37beba0-5c73-4fdf-bc42-cbf49e57d5ee None admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:12.285245 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37beba0-5c73-4fdf-bc42-cbf49e57d5ee None admin] PATH_INFO: `/v3/services` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:12.286356 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a37beba0-5c73-4fdf-bc42-cbf49e57d5ee None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=88113) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:35:12.287003 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a37beba0-5c73-4fdf-bc42-cbf49e57d5ee None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:35:12.289311 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a37beba0-5c73-4fdf-bc42-cbf49e57d5ee None admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:35:12.298536 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 454/903] 10.222.0.13 () {56 vars in 1126 bytes} [Wed Aug 31 09:35:12 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:35:12.306096 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c77589c-7d8e-4cc2-8041-ec7a41dd0fa1 None admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:35:12.342307 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=100 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:35:12.429995 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a37beba0-5c73-4fdf-bc42-cbf49e57d5ee None admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:35:12.493595 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c77589c-7d8e-4cc2-8041-ec7a41dd0fa1 None admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:35:12.500540 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=100 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:35:12.542697 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ccd124-be23-4659-b587-2798920c7590 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:35:12.543848 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ccd124-be23-4659-b587-2798920c7590 None admin] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:12.544361 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ccd124-be23-4659-b587-2798920c7590 None admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:12.545551 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ccd124-be23-4659-b587-2798920c7590 None admin] PATH_INFO: `/v3/services` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:12.546601 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29ccd124-be23-4659-b587-2798920c7590 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=88114) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:35:12.547129 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29ccd124-be23-4659-b587-2798920c7590 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:35:12.548918 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29ccd124-be23-4659-b587-2798920c7590 None admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:35:12.551991 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a37beba0-5c73-4fdf-bc42-cbf49e57d5ee None admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:35:12.556391 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 450/904] 10.222.0.13 () {56 vars in 1126 bytes} [Wed Aug 31 09:35:12 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:35:12.564537 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ccd124-be23-4659-b587-2798920c7590 None admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:35:12.593236 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f3eb24-c3cb-41b2-96db-780b2f9029a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:35:12.593947 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f3eb24-c3cb-41b2-96db-780b2f9029a8 None admin] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:12.594223 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f3eb24-c3cb-41b2-96db-780b2f9029a8 None admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:12.594551 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f3eb24-c3cb-41b2-96db-780b2f9029a8 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:12.595222 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91f3eb24-c3cb-41b2-96db-780b2f9029a8 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:35:12.597166 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91f3eb24-c3cb-41b2-96db-780b2f9029a8 None admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:35:12.607899 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application [None req-91f3eb24-c3cb-41b2-96db-780b2f9029a8 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 31 09:35:12.607899 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:35:12.607899 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 31 09:35:12.607899 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 31 09:35:12.607899 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 31 09:35:12.607899 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 31 09:35:12.607899 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 31 09:35:12.607899 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 09:35:12.607899 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 31 09:35:12.607899 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application Aug 31 09:35:12.607899 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 31 09:35:12.607899 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application Aug 31 09:35:12.607899 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:35:12.607899 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:35:12.607899 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:35:12.607899 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:35:12.607899 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:35:12.609141 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:35:12.609141 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:35:12.609141 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:35:12.609141 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:35:12.609141 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:35:12.609141 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:35:12.609141 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 31 09:35:12.609141 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 31 09:35:12.609141 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 31 09:35:12.609141 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 31 09:35:12.609141 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:35:12.609141 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:35:12.609141 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:35:12.609141 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:35:12.609141 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:35:12.609141 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:35:12.610075 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:35:12.610075 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application with Lock( Aug 31 09:35:12.610075 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:35:12.610075 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:35:12.610075 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:35:12.610075 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:35:12.610075 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:35:12.610075 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:35:12.610075 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:35:12.610075 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:35:12.610075 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 31 09:35:12.610075 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 09:35:12.610075 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 31 09:35:12.610075 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application Aug 31 09:35:12.611012 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 455/905] 10.222.0.13 () {56 vars in 1126 bytes} [Wed Aug 31 09:35:12 2022] GET /identity/v3/services/object-store => generated 93 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 09:35:12.617297 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f3eb24-c3cb-41b2-96db-780b2f9029a8 None admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:35:12.682744 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=100 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:35:12.690883 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ccd124-be23-4659-b587-2798920c7590 None admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:35:12.727696 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eac1433-f986-42e0-9e6c-58b5918499cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:35:12.728659 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eac1433-f986-42e0-9e6c-58b5918499cc None admin] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:12.729149 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eac1433-f986-42e0-9e6c-58b5918499cc None admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:12.729712 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eac1433-f986-42e0-9e6c-58b5918499cc None admin] PATH_INFO: `/v3/endpoints` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:12.730625 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eac1433-f986-42e0-9e6c-58b5918499cc None admin] RBAC: Adding query filter params (interface=public, service_id=3980507a14dc4a35bde5a3e5e5ce2870, region_id=RegionOne) {{(pid=88114) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:35:12.731145 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eac1433-f986-42e0-9e6c-58b5918499cc None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:35:12.732802 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eac1433-f986-42e0-9e6c-58b5918499cc None admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:35:12.743448 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f3eb24-c3cb-41b2-96db-780b2f9029a8 None admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:35:12.743893 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 451/906] 10.222.0.13 () {56 vars in 1251 bytes} [Wed Aug 31 09:35:12 2022] GET /identity/v3/endpoints?service_id=3980507a14dc4a35bde5a3e5e5ce2870&interface=public®ion_id=RegionOne => generated 194 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:35:12.745053 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eac1433-f986-42e0-9e6c-58b5918499cc None admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:35:12.763509 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=100 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:35:12.784132 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d27e30a9-6d78-463b-8cdb-b79e733cc7de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:35:12.784805 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27e30a9-6d78-463b-8cdb-b79e733cc7de None admin] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:12.785084 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27e30a9-6d78-463b-8cdb-b79e733cc7de None admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:12.785356 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27e30a9-6d78-463b-8cdb-b79e733cc7de None admin] PATH_INFO: `/v3/services` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:12.786156 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d27e30a9-6d78-463b-8cdb-b79e733cc7de None admin] RBAC: Adding query filter params (name=object-store) {{(pid=88113) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:35:12.786698 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d27e30a9-6d78-463b-8cdb-b79e733cc7de None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:35:12.790233 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d27e30a9-6d78-463b-8cdb-b79e733cc7de None admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:35:12.805673 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 456/907] 10.222.0.13 () {56 vars in 1122 bytes} [Wed Aug 31 09:35:12 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:35:12.809744 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d27e30a9-6d78-463b-8cdb-b79e733cc7de None admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:35:12.868009 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eac1433-f986-42e0-9e6c-58b5918499cc None admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:35:12.906457 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7755b5a-2642-43e5-bd1c-415f860a8a53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:35:12.907444 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7755b5a-2642-43e5-bd1c-415f860a8a53 None admin] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:12.907922 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7755b5a-2642-43e5-bd1c-415f860a8a53 None admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:12.908389 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7755b5a-2642-43e5-bd1c-415f860a8a53 None admin] PATH_INFO: `/v3/services` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:12.909208 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7755b5a-2642-43e5-bd1c-415f860a8a53 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:35:12.910806 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7755b5a-2642-43e5-bd1c-415f860a8a53 None admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:35:12.925709 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 452/908] 10.222.0.13 () {56 vars in 1087 bytes} [Wed Aug 31 09:35:12 2022] GET /identity/v3/services => generated 1268 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 09:35:12.993153 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d27e30a9-6d78-463b-8cdb-b79e733cc7de None admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:35:13.036674 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e0248b-028f-4486-a495-5a37f1534396 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:35:13.037302 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e0248b-028f-4486-a495-5a37f1534396 None admin] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:13.037626 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e0248b-028f-4486-a495-5a37f1534396 None admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:13.037898 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e0248b-028f-4486-a495-5a37f1534396 None admin] PATH_INFO: `/v3/services` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:13.038550 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99e0248b-028f-4486-a495-5a37f1534396 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=88113) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:35:13.038853 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99e0248b-028f-4486-a495-5a37f1534396 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:35:13.040347 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99e0248b-028f-4486-a495-5a37f1534396 None admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:35:13.047283 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 457/909] 10.222.0.13 () {56 vars in 1122 bytes} [Wed Aug 31 09:35:12 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 09:35:13.256738 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d735ac3a-b134-4fdc-a2b8-ae37b85167df None None] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:13.257284 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d735ac3a-b134-4fdc-a2b8-ae37b85167df None None] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:13.257284 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d735ac3a-b134-4fdc-a2b8-ae37b85167df None None] PATH_INFO: `/` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:13.259081 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 453/910] 10.222.0.13 () {54 vars in 927 bytes} [Wed Aug 31 09:35:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:35:13.264841 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14fcd986-27b0-4677-b346-f2812f75982f None None] REQUEST_METHOD: `POST` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:13.265139 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14fcd986-27b0-4677-b346-f2812f75982f None None] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:13.265417 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14fcd986-27b0-4677-b346-f2812f75982f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:13.291328 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.auth.core [None req-14fcd986-27b0-4677-b346-f2812f75982f None None] MFA Rules not processed for user `fd836cf590a145df9acf19cf3ae2ac56`. Rule list: `[]` (Enabled: `True`). {{(pid=88113) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:35:13.292801 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.fernet_utils [None req-14fcd986-27b0-4677-b346-f2812f75982f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:35:13.354621 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=100 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:35:13.391979 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 458/911] 10.222.0.13 () {58 vars in 1024 bytes} [Wed Aug 31 09:35:13 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 09:35:13.673275 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d735ac3a-b134-4fdc-a2b8-ae37b85167df None None] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:35:13.702124 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=100 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:35:13.741673 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc87802-ed34-4a5a-a081-9dc1ed656954 None None] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:13.741937 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc87802-ed34-4a5a-a081-9dc1ed656954 None None] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:13.742299 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc87802-ed34-4a5a-a081-9dc1ed656954 None None] PATH_INFO: `/` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:13.743609 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 459/912] 10.222.0.13 () {54 vars in 927 bytes} [Wed Aug 31 09:35:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:35:13.749537 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f5485c-c4d1-4ef8-86a6-fdb61a5ac49b None None] REQUEST_METHOD: `POST` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:13.749918 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f5485c-c4d1-4ef8-86a6-fdb61a5ac49b None None] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:13.750075 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f5485c-c4d1-4ef8-86a6-fdb61a5ac49b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:13.770742 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.auth.core [None req-c6f5485c-c4d1-4ef8-86a6-fdb61a5ac49b None None] MFA Rules not processed for user `fd836cf590a145df9acf19cf3ae2ac56`. Rule list: `[]` (Enabled: `True`). {{(pid=88113) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:35:13.772179 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.fernet_utils [None req-c6f5485c-c4d1-4ef8-86a6-fdb61a5ac49b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:35:13.805886 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d735ac3a-b134-4fdc-a2b8-ae37b85167df None None] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:35:13.851623 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ab1c0b-e08e-419e-a779-c2d49ac5c3c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:35:13.853226 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ab1c0b-e08e-419e-a779-c2d49ac5c3c8 None admin] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:13.853730 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ab1c0b-e08e-419e-a779-c2d49ac5c3c8 None admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:13.854444 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ab1c0b-e08e-419e-a779-c2d49ac5c3c8 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:13.855384 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81ab1c0b-e08e-419e-a779-c2d49ac5c3c8 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:35:13.858260 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81ab1c0b-e08e-419e-a779-c2d49ac5c3c8 None admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:35:13.875513 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application [None req-81ab1c0b-e08e-419e-a779-c2d49ac5c3c8 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 31 09:35:13.875513 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:35:13.875513 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 31 09:35:13.875513 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 31 09:35:13.875513 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 31 09:35:13.875513 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 31 09:35:13.875513 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 31 09:35:13.875513 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 09:35:13.875513 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 31 09:35:13.875513 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application Aug 31 09:35:13.875513 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 31 09:35:13.875513 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application Aug 31 09:35:13.875513 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:35:13.875513 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:35:13.875513 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:35:13.875513 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:35:13.875513 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:35:13.876577 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:35:13.876577 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:35:13.876577 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:35:13.876577 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:35:13.876577 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:35:13.876577 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:35:13.876577 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 31 09:35:13.876577 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 31 09:35:13.876577 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 31 09:35:13.876577 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 31 09:35:13.876577 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:35:13.876577 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:35:13.876577 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:35:13.876577 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:35:13.876577 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:35:13.876577 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:35:13.877452 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:35:13.877452 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application with Lock( Aug 31 09:35:13.877452 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:35:13.877452 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:35:13.877452 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:35:13.877452 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:35:13.877452 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:35:13.877452 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:35:13.877452 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:35:13.877452 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:35:13.877452 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 31 09:35:13.877452 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 09:35:13.877452 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 31 09:35:13.877452 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application Aug 31 09:35:13.884953 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 454/914] 10.222.0.13 () {56 vars in 1114 bytes} [Wed Aug 31 09:35:13 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:35:13.885634 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 460/914] 10.222.0.13 () {58 vars in 1024 bytes} [Wed Aug 31 09:35:13 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 09:35:13.886732 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ab1c0b-e08e-419e-a779-c2d49ac5c3c8 None admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:35:13.991649 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=100 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:35:14.024661 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ab1c0b-e08e-419e-a779-c2d49ac5c3c8 None admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:35:14.090350 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd80efb4-b320-459c-8a40-9da4c00e6bd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:35:14.091313 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd80efb4-b320-459c-8a40-9da4c00e6bd4 None admin] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:14.092328 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd80efb4-b320-459c-8a40-9da4c00e6bd4 None admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:14.092740 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd80efb4-b320-459c-8a40-9da4c00e6bd4 None admin] PATH_INFO: `/v3/services` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:14.093726 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd80efb4-b320-459c-8a40-9da4c00e6bd4 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=88114) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:35:14.094166 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd80efb4-b320-459c-8a40-9da4c00e6bd4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:35:14.096533 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd80efb4-b320-459c-8a40-9da4c00e6bd4 None admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:35:14.105885 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 455/915] 10.222.0.13 () {56 vars in 1114 bytes} [Wed Aug 31 09:35:13 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 09:35:14.112335 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f5485c-c4d1-4ef8-86a6-fdb61a5ac49b None None] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:35:14.179226 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd80efb4-b320-459c-8a40-9da4c00e6bd4 None admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:35:14.324281 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f5485c-c4d1-4ef8-86a6-fdb61a5ac49b None None] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:35:14.345427 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=100 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:35:14.403818 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c88d5f29-9824-46a6-b941-eaffa420bce8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:35:14.405083 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88d5f29-9824-46a6-b941-eaffa420bce8 None admin] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:14.405353 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88d5f29-9824-46a6-b941-eaffa420bce8 None admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:14.405646 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88d5f29-9824-46a6-b941-eaffa420bce8 None admin] PATH_INFO: `/v3/services` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:14.406290 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c88d5f29-9824-46a6-b941-eaffa420bce8 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=88113) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:35:14.406591 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c88d5f29-9824-46a6-b941-eaffa420bce8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:35:14.408145 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c88d5f29-9824-46a6-b941-eaffa420bce8 None admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:35:14.431199 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 461/916] 10.222.0.13 () {56 vars in 1114 bytes} [Wed Aug 31 09:35:14 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 315 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:35:14.434491 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c88d5f29-9824-46a6-b941-eaffa420bce8 None admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:35:14.437105 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd80efb4-b320-459c-8a40-9da4c00e6bd4 None admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:35:14.492823 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd9b2b2-6dbf-4fda-9af8-c0e05ddbf754 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:35:14.493142 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd9b2b2-6dbf-4fda-9af8-c0e05ddbf754 None admin] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:14.493296 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd9b2b2-6dbf-4fda-9af8-c0e05ddbf754 None admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:14.493635 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd9b2b2-6dbf-4fda-9af8-c0e05ddbf754 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:14.519459 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bd9b2b2-6dbf-4fda-9af8-c0e05ddbf754 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 09:35:14.524357 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application [None req-1bd9b2b2-6dbf-4fda-9af8-c0e05ddbf754 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 31 09:35:14.524357 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:35:14.524357 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:35:14.524357 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:35:14.524357 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:35:14.524357 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:35:14.524357 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:35:14.524357 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:35:14.524357 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:35:14.524357 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:35:14.524357 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:35:14.524357 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:35:14.524357 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 09:35:14.524357 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 09:35:14.524357 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 09:35:14.524357 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 09:35:14.526214 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 09:35:14.526214 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 09:35:14.526214 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 09:35:14.526214 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 09:35:14.526214 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:35:14.526214 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:35:14.526214 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:35:14.526214 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:35:14.526214 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:35:14.526214 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:35:14.526214 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:35:14.526214 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application with Lock( Aug 31 09:35:14.526214 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:35:14.526214 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:35:14.526214 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:35:14.526214 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:35:14.527820 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:35:14.527820 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:35:14.527820 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:35:14.527820 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:35:14.527820 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 09:35:14.527820 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 09:35:14.527820 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 09:35:14.527820 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 09:35:14.527820 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 09:35:14.527820 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 09:35:14.527820 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 31 09:35:14.527820 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application Aug 31 09:35:14.531314 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 456/917] 10.222.0.13 () {56 vars in 1120 bytes} [Wed Aug 31 09:35:14 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 350 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:35:14.536801 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd9b2b2-6dbf-4fda-9af8-c0e05ddbf754 None admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:35:14.565440 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=100 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:35:14.678658 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c88d5f29-9824-46a6-b941-eaffa420bce8 None admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:35:14.754335 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd9b2b2-6dbf-4fda-9af8-c0e05ddbf754 None admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:35:14.770719 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dbe4a6f-cc96-4a6d-b549-1687f7589636 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:35:14.771347 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dbe4a6f-cc96-4a6d-b549-1687f7589636 None admin] REQUEST_METHOD: `POST` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:14.771629 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dbe4a6f-cc96-4a6d-b549-1687f7589636 None admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:14.771884 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dbe4a6f-cc96-4a6d-b549-1687f7589636 None admin] PATH_INFO: `/v3/endpoints` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:14.772555 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dbe4a6f-cc96-4a6d-b549-1687f7589636 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:35:14.774668 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dbe4a6f-cc96-4a6d-b549-1687f7589636 None admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:35:14.805426 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 462/918] 10.222.0.13 () {60 vars in 1144 bytes} [Wed Aug 31 09:35:14 2022] POST /identity/v3/endpoints => generated 346 bytes in 371 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 09:35:14.852999 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e0728d1-2131-41aa-aaa7-16ac61635c59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:35:14.854138 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0728d1-2131-41aa-aaa7-16ac61635c59 None admin] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:14.854640 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0728d1-2131-41aa-aaa7-16ac61635c59 None admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:14.855122 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0728d1-2131-41aa-aaa7-16ac61635c59 None admin] PATH_INFO: `/v3/roles` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:14.856320 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e0728d1-2131-41aa-aaa7-16ac61635c59 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=88114) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:35:14.856861 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e0728d1-2131-41aa-aaa7-16ac61635c59 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:35:14.860317 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e0728d1-2131-41aa-aaa7-16ac61635c59 None admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:35:14.879941 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 457/919] 10.222.0.13 () {56 vars in 1115 bytes} [Wed Aug 31 09:35:14 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 345 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:35:14.885120 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dbe4a6f-cc96-4a6d-b549-1687f7589636 None admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:35:14.932256 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:35:15.032559 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dbe4a6f-cc96-4a6d-b549-1687f7589636 None admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:35:15.083342 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7773efb-a97d-435a-a755-ff1c0b040a0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:35:15.084001 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7773efb-a97d-435a-a755-ff1c0b040a0b None admin] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:15.084282 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7773efb-a97d-435a-a755-ff1c0b040a0b None admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:15.084559 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7773efb-a97d-435a-a755-ff1c0b040a0b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:15.098169 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7773efb-a97d-435a-a755-ff1c0b040a0b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:35:15.099834 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7773efb-a97d-435a-a755-ff1c0b040a0b None admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:35:15.117592 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application [None req-a7773efb-a97d-435a-a755-ff1c0b040a0b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 09:35:15.117592 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:35:15.117592 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:35:15.117592 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:35:15.117592 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:35:15.117592 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:35:15.117592 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:35:15.117592 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:35:15.117592 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:35:15.117592 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:35:15.117592 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:35:15.117592 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:35:15.117592 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 09:35:15.117592 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 09:35:15.117592 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 09:35:15.117592 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 09:35:15.120127 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:35:15.120127 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:35:15.120127 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:35:15.120127 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:35:15.120127 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:35:15.120127 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:35:15.120127 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:35:15.120127 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application with Lock( Aug 31 09:35:15.120127 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:35:15.120127 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:35:15.120127 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:35:15.120127 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:35:15.120127 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:35:15.120127 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:35:15.120127 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:35:15.120127 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:35:15.120127 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 09:35:15.122048 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 09:35:15.122048 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 09:35:15.122048 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application Aug 31 09:35:15.122382 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859de4d5-6dce-46c9-b257-b849fd68c2cd None None] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:15.122382 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859de4d5-6dce-46c9-b257-b849fd68c2cd None None] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:15.122382 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859de4d5-6dce-46c9-b257-b849fd68c2cd None None] PATH_INFO: `/` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:15.124242 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 463/920] 10.222.0.13 () {56 vars in 1108 bytes} [Wed Aug 31 09:35:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:35:15.124479 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 458/921] 10.222.0.13 () {54 vars in 927 bytes} [Wed Aug 31 09:35:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:35:15.129021 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7773efb-a97d-435a-a755-ff1c0b040a0b None admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:35:15.129440 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd562701-b1d8-4974-a244-a67d182b2af5 None None] REQUEST_METHOD: `POST` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:15.129440 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd562701-b1d8-4974-a244-a67d182b2af5 None None] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:15.129440 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd562701-b1d8-4974-a244-a67d182b2af5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:15.150443 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.auth.core [None req-bd562701-b1d8-4974-a244-a67d182b2af5 None None] MFA Rules not processed for user `fd836cf590a145df9acf19cf3ae2ac56`. Rule list: `[]` (Enabled: `True`). {{(pid=88114) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:35:15.151681 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.fernet_utils [None req-bd562701-b1d8-4974-a244-a67d182b2af5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88114) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:35:15.239485 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=100 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:35:15.271881 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7773efb-a97d-435a-a755-ff1c0b040a0b None admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:35:15.286811 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=100 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:35:15.299346 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 459/922] 10.222.0.13 () {58 vars in 1024 bytes} [Wed Aug 31 09:35:15 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 09:35:15.302844 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110a5111-5515-47c0-99ee-9f93e6c4afad None None] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:15.303449 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110a5111-5515-47c0-99ee-9f93e6c4afad None None] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:15.304258 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110a5111-5515-47c0-99ee-9f93e6c4afad None None] PATH_INFO: `/` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:15.307220 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 460/923] 10.222.0.13 () {54 vars in 927 bytes} [Wed Aug 31 09:35:15 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:35:15.315391 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dee26c9-4279-4520-aa9b-b7a495c382f1 None None] REQUEST_METHOD: `POST` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:15.315953 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dee26c9-4279-4520-aa9b-b7a495c382f1 None None] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:15.316463 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dee26c9-4279-4520-aa9b-b7a495c382f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:15.330598 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95114cd0-9595-4294-abc3-143325a496be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:35:15.331243 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95114cd0-9595-4294-abc3-143325a496be None admin] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:15.331521 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95114cd0-9595-4294-abc3-143325a496be None admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:15.331794 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95114cd0-9595-4294-abc3-143325a496be None admin] PATH_INFO: `/v3/domains` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:15.332486 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95114cd0-9595-4294-abc3-143325a496be None admin] RBAC: Adding query filter params (name=Default) {{(pid=88113) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:35:15.332791 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95114cd0-9595-4294-abc3-143325a496be None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:35:15.334379 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95114cd0-9595-4294-abc3-143325a496be None admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:35:15.343138 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.auth.core [None req-9dee26c9-4279-4520-aa9b-b7a495c382f1 None None] MFA Rules not processed for user `fd836cf590a145df9acf19cf3ae2ac56`. Rule list: `[]` (Enabled: `True`). {{(pid=88114) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:35:15.345677 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.fernet_utils [None req-9dee26c9-4279-4520-aa9b-b7a495c382f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88114) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:35:15.357636 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 464/924] 10.222.0.13 () {56 vars in 1109 bytes} [Wed Aug 31 09:35:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:35:15.359291 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95114cd0-9595-4294-abc3-143325a496be None admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:35:15.459078 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 461/925] 10.222.0.13 () {58 vars in 1024 bytes} [Wed Aug 31 09:35:15 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 31 09:35:15.521749 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95114cd0-9595-4294-abc3-143325a496be None admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:35:15.589491 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c00df2e5-be60-47ae-a9f3-0c6837c32675 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:35:15.590222 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dee26c9-4279-4520-aa9b-b7a495c382f1 None None] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:35:15.591235 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00df2e5-be60-47ae-a9f3-0c6837c32675 None admin] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:15.592528 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00df2e5-be60-47ae-a9f3-0c6837c32675 None admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:15.593198 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00df2e5-be60-47ae-a9f3-0c6837c32675 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:15.609186 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c00df2e5-be60-47ae-a9f3-0c6837c32675 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:35:15.612453 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c00df2e5-be60-47ae-a9f3-0c6837c32675 None admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:35:15.617925 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=100 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:35:15.641176 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application [None req-c00df2e5-be60-47ae-a9f3-0c6837c32675 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 31 09:35:15.641176 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:35:15.641176 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:35:15.641176 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:35:15.641176 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:35:15.641176 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:35:15.641176 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:35:15.641176 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:35:15.641176 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:35:15.641176 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:35:15.641176 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:35:15.641176 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:35:15.641176 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 31 09:35:15.641176 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 31 09:35:15.641176 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 31 09:35:15.641176 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 31 09:35:15.643626 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:35:15.643626 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:35:15.643626 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 31 09:35:15.643626 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 09:35:15.643626 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 31 09:35:15.643626 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 09:35:15.643626 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:35:15.643626 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:35:15.643626 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:35:15.643626 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:35:15.643626 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:35:15.643626 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application with Lock( Aug 31 09:35:15.643626 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:35:15.643626 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:35:15.643626 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:35:15.643626 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:35:15.643626 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:35:15.643626 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:35:15.645876 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:35:15.645876 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:35:15.645876 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 31 09:35:15.645876 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 31 09:35:15.645876 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 31 09:35:15.645876 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 31 09:35:15.645876 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 31 09:35:15.645876 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 31 09:35:15.645876 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 31 09:35:15.645876 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application Aug 31 09:35:15.653646 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 465/926] 10.222.0.13 () {56 vars in 1117 bytes} [Wed Aug 31 09:35:15 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 291 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:35:15.659838 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c00df2e5-be60-47ae-a9f3-0c6837c32675 None admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:35:15.716136 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dee26c9-4279-4520-aa9b-b7a495c382f1 None None] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:35:15.748240 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=100 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:35:15.757579 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-257e30d9-469e-4f23-a402-25d175bdf753 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:35:15.758228 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257e30d9-469e-4f23-a402-25d175bdf753 None admin] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:15.758494 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257e30d9-469e-4f23-a402-25d175bdf753 None admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:15.758762 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257e30d9-469e-4f23-a402-25d175bdf753 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:15.759389 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-257e30d9-469e-4f23-a402-25d175bdf753 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:35:15.760746 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-257e30d9-469e-4f23-a402-25d175bdf753 None admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:35:15.770928 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application [None req-257e30d9-469e-4f23-a402-25d175bdf753 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 31 09:35:15.770928 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:35:15.770928 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 31 09:35:15.770928 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 31 09:35:15.770928 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 31 09:35:15.770928 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 31 09:35:15.770928 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 31 09:35:15.770928 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 09:35:15.770928 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 31 09:35:15.770928 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application Aug 31 09:35:15.770928 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 31 09:35:15.770928 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application Aug 31 09:35:15.770928 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:35:15.770928 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:35:15.770928 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:35:15.770928 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:35:15.774473 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:35:15.774473 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:35:15.774473 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:35:15.774473 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:35:15.774473 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:35:15.774473 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:35:15.774473 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:35:15.774473 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 31 09:35:15.774473 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 31 09:35:15.774473 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 31 09:35:15.774473 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 31 09:35:15.774473 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:35:15.774473 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:35:15.774473 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:35:15.774473 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:35:15.774473 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:35:15.774473 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:35:15.776537 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:35:15.776537 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application with Lock( Aug 31 09:35:15.776537 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:35:15.776537 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:35:15.776537 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:35:15.776537 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:35:15.776537 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:35:15.776537 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:35:15.776537 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:35:15.776537 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:35:15.776537 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 31 09:35:15.776537 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 09:35:15.776537 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 31 09:35:15.776537 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application Aug 31 09:35:15.776537 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 462/927] 10.222.0.13 () {56 vars in 1132 bytes} [Wed Aug 31 09:35:15 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 09:35:15.778438 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-257e30d9-469e-4f23-a402-25d175bdf753 None admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:35:15.833602 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c00df2e5-be60-47ae-a9f3-0c6837c32675 None admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:35:15.872296 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-913e2a73-ae25-4ae7-8eae-8502f9b588c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:35:15.873476 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913e2a73-ae25-4ae7-8eae-8502f9b588c6 None admin] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:15.873759 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913e2a73-ae25-4ae7-8eae-8502f9b588c6 None admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:15.874031 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913e2a73-ae25-4ae7-8eae-8502f9b588c6 None admin] PATH_INFO: `/v3/users` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:15.874777 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-913e2a73-ae25-4ae7-8eae-8502f9b588c6 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=88113) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:35:15.875081 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-913e2a73-ae25-4ae7-8eae-8502f9b588c6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:35:15.876559 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-913e2a73-ae25-4ae7-8eae-8502f9b588c6 None admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:35:15.902185 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 466/928] 10.222.0.13 () {56 vars in 1149 bytes} [Wed Aug 31 09:35:15 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 09:35:15.902185 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-913e2a73-ae25-4ae7-8eae-8502f9b588c6 None admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:35:15.958184 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-257e30d9-469e-4f23-a402-25d175bdf753 None admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:35:15.983761 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=100 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:35:16.019551 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c01ea2be-c176-4dd9-a3b9-e1a6f4b44c8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:35:16.020557 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01ea2be-c176-4dd9-a3b9-e1a6f4b44c8a None admin] REQUEST_METHOD: `POST` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:16.021002 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01ea2be-c176-4dd9-a3b9-e1a6f4b44c8a None admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:16.021512 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01ea2be-c176-4dd9-a3b9-e1a6f4b44c8a None admin] PATH_INFO: `/v3/services` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:16.022664 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c01ea2be-c176-4dd9-a3b9-e1a6f4b44c8a None admin] RBAC: Authorizing `identity:create_service()` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:35:16.025296 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c01ea2be-c176-4dd9-a3b9-e1a6f4b44c8a None admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:35:16.035753 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-913e2a73-ae25-4ae7-8eae-8502f9b588c6 None admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:35:16.049531 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 463/929] 10.222.0.13 () {60 vars in 1141 bytes} [Wed Aug 31 09:35:15 2022] POST /identity/v3/services => generated 241 bytes in 271 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 09:35:16.049531 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c01ea2be-c176-4dd9-a3b9-e1a6f4b44c8a None admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:35:16.090039 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f7f83c9-2718-445f-81c0-94d7ada8f58d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:35:16.090769 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7f83c9-2718-445f-81c0-94d7ada8f58d None admin] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:16.091049 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7f83c9-2718-445f-81c0-94d7ada8f58d None admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:16.091323 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7f83c9-2718-445f-81c0-94d7ada8f58d None admin] PATH_INFO: `/v3/services` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:16.092546 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f7f83c9-2718-445f-81c0-94d7ada8f58d None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=88113) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:35:16.092852 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f7f83c9-2718-445f-81c0-94d7ada8f58d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:35:16.094546 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f7f83c9-2718-445f-81c0-94d7ada8f58d None admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:35:16.101016 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 467/930] 10.222.0.13 () {56 vars in 1126 bytes} [Wed Aug 31 09:35:15 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:35:16.110845 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f7f83c9-2718-445f-81c0-94d7ada8f58d None admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:35:16.141493 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:35:16.218848 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c01ea2be-c176-4dd9-a3b9-e1a6f4b44c8a None admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:35:16.274048 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6390f340-69f3-461b-89e0-1ff432384f35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:35:16.274818 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6390f340-69f3-461b-89e0-1ff432384f35 None admin] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:16.275102 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6390f340-69f3-461b-89e0-1ff432384f35 None admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:16.275445 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6390f340-69f3-461b-89e0-1ff432384f35 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:16.289916 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6390f340-69f3-461b-89e0-1ff432384f35 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:35:16.292393 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6390f340-69f3-461b-89e0-1ff432384f35 None admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:35:16.315435 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f7f83c9-2718-445f-81c0-94d7ada8f58d None admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:35:16.322715 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application [None req-6390f340-69f3-461b-89e0-1ff432384f35 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 09:35:16.322715 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:35:16.322715 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:35:16.322715 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:35:16.322715 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:35:16.322715 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:35:16.322715 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:35:16.322715 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:35:16.322715 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:35:16.322715 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:35:16.322715 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:35:16.322715 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:35:16.322715 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 09:35:16.322715 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 09:35:16.322715 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 09:35:16.322715 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 09:35:16.324814 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:35:16.324814 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:35:16.324814 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:35:16.324814 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:35:16.324814 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:35:16.324814 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:35:16.324814 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:35:16.324814 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application with Lock( Aug 31 09:35:16.324814 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:35:16.324814 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:35:16.324814 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:35:16.324814 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:35:16.324814 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:35:16.324814 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:35:16.324814 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:35:16.324814 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:35:16.324814 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 09:35:16.327799 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 09:35:16.327799 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 09:35:16.327799 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application Aug 31 09:35:16.337580 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 464/931] 10.222.0.13 () {56 vars in 1108 bytes} [Wed Aug 31 09:35:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 286 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:35:16.338468 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=100 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:35:16.342066 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6390f340-69f3-461b-89e0-1ff432384f35 None admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:35:16.369990 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a13b9736-f8c0-411d-b929-0f38f5c9b09d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:35:16.371384 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13b9736-f8c0-411d-b929-0f38f5c9b09d None admin] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:16.372119 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13b9736-f8c0-411d-b929-0f38f5c9b09d None admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:16.373422 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13b9736-f8c0-411d-b929-0f38f5c9b09d None admin] PATH_INFO: `/v3/services` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:16.374817 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a13b9736-f8c0-411d-b929-0f38f5c9b09d None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=88113) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:35:16.375566 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a13b9736-f8c0-411d-b929-0f38f5c9b09d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:35:16.378678 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a13b9736-f8c0-411d-b929-0f38f5c9b09d None admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:35:16.397594 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 468/932] 10.222.0.13 () {56 vars in 1126 bytes} [Wed Aug 31 09:35:16 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 282 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 09:35:16.401825 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a13b9736-f8c0-411d-b929-0f38f5c9b09d None admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:35:16.491532 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6390f340-69f3-461b-89e0-1ff432384f35 None admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:35:16.498662 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=100 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:35:16.567457 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a13b9736-f8c0-411d-b929-0f38f5c9b09d None admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:35:16.571143 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-470b8d39-f30f-4fe5-ae58-ec6a119ba0fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:35:16.572537 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470b8d39-f30f-4fe5-ae58-ec6a119ba0fd None admin] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:16.573296 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470b8d39-f30f-4fe5-ae58-ec6a119ba0fd None admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:16.574128 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470b8d39-f30f-4fe5-ae58-ec6a119ba0fd None admin] PATH_INFO: `/v3/domains` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:16.575423 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-470b8d39-f30f-4fe5-ae58-ec6a119ba0fd None admin] RBAC: Adding query filter params (name=Default) {{(pid=88114) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:35:16.576035 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-470b8d39-f30f-4fe5-ae58-ec6a119ba0fd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:35:16.579507 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-470b8d39-f30f-4fe5-ae58-ec6a119ba0fd None admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:35:16.607215 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 465/933] 10.222.0.13 () {56 vars in 1109 bytes} [Wed Aug 31 09:35:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:35:16.614372 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-470b8d39-f30f-4fe5-ae58-ec6a119ba0fd None admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:35:16.618095 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feba0dce-fa40-4f23-bf9a-47e641e8c6b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:35:16.618789 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feba0dce-fa40-4f23-bf9a-47e641e8c6b9 None admin] REQUEST_METHOD: `POST` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:16.620132 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feba0dce-fa40-4f23-bf9a-47e641e8c6b9 None admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:16.620399 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feba0dce-fa40-4f23-bf9a-47e641e8c6b9 None admin] PATH_INFO: `/v3/endpoints` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:16.621114 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feba0dce-fa40-4f23-bf9a-47e641e8c6b9 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:35:16.623204 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feba0dce-fa40-4f23-bf9a-47e641e8c6b9 None admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:35:16.658457 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 469/934] 10.222.0.13 () {60 vars in 1144 bytes} [Wed Aug 31 09:35:16 2022] POST /identity/v3/endpoints => generated 347 bytes in 258 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 09:35:16.755246 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-470b8d39-f30f-4fe5-ae58-ec6a119ba0fd None admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:35:16.806595 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9857c1d-8eff-43a5-b212-918f2805ca36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:35:16.807266 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9857c1d-8eff-43a5-b212-918f2805ca36 None admin] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:16.807545 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9857c1d-8eff-43a5-b212-918f2805ca36 None admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:16.807814 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9857c1d-8eff-43a5-b212-918f2805ca36 None admin] PATH_INFO: `/v3/projects/service` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:16.816461 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9857c1d-8eff-43a5-b212-918f2805ca36 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:35:16.818125 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9857c1d-8eff-43a5-b212-918f2805ca36 None admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:35:16.834450 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application [None req-d9857c1d-8eff-43a5-b212-918f2805ca36 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 09:35:16.834450 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:35:16.834450 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:35:16.834450 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:35:16.834450 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:35:16.834450 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:35:16.834450 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:35:16.834450 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:35:16.834450 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:35:16.834450 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:35:16.834450 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:35:16.834450 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:35:16.834450 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 31 09:35:16.834450 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 31 09:35:16.834450 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 31 09:35:16.836541 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 31 09:35:16.836541 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:35:16.836541 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:35:16.836541 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:35:16.836541 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:35:16.836541 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:35:16.836541 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:35:16.836541 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:35:16.836541 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application with Lock( Aug 31 09:35:16.836541 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:35:16.836541 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:35:16.836541 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:35:16.836541 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:35:16.836541 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:35:16.836541 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:35:16.836541 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:35:16.836541 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:35:16.838581 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 31 09:35:16.838581 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 31 09:35:16.838581 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 31 09:35:16.838581 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 31 09:35:16.838581 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 31 09:35:16.838581 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 31 09:35:16.838581 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 09:35:16.838581 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application Aug 31 09:35:16.838581 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 466/935] 10.222.0.13 () {56 vars in 1111 bytes} [Wed Aug 31 09:35:16 2022] GET /identity/v3/projects/service => generated 88 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:35:16.855013 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feba0dce-fa40-4f23-bf9a-47e641e8c6b9 None admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:35:16.940627 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:35:17.057837 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feba0dce-fa40-4f23-bf9a-47e641e8c6b9 None admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:35:17.118576 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53186da5-213a-4782-9613-2f50c79be86f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:35:17.119571 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53186da5-213a-4782-9613-2f50c79be86f None admin] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:17.120057 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53186da5-213a-4782-9613-2f50c79be86f None admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:17.120549 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53186da5-213a-4782-9613-2f50c79be86f None admin] PATH_INFO: `/v3/projects` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:17.121925 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53186da5-213a-4782-9613-2f50c79be86f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88113) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:35:17.122303 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53186da5-213a-4782-9613-2f50c79be86f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:35:17.125576 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53186da5-213a-4782-9613-2f50c79be86f None admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:35:17.151876 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 470/936] 10.222.0.13 () {56 vars in 1148 bytes} [Wed Aug 31 09:35:16 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 298 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:35:17.161979 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9857c1d-8eff-43a5-b212-918f2805ca36 None admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:35:17.196929 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=100 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:35:17.290291 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9857c1d-8eff-43a5-b212-918f2805ca36 None admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:35:17.331075 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f23de39a-a999-4471-be65-d531eb2643b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:35:17.332046 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23de39a-a999-4471-be65-d531eb2643b2 None admin] REQUEST_METHOD: `PUT` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:17.332488 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23de39a-a999-4471-be65-d531eb2643b2 None admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:17.332920 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23de39a-a999-4471-be65-d531eb2643b2 None admin] PATH_INFO: `/v3/projects/49548eb44d62415582af820ab66decee/users/768cb67c1ef24251aaa113c12d81a250/roles/d41c162f8e1748989a09382ced5ef3a6` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:17.335460 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f23de39a-a999-4471-be65-d531eb2643b2 None admin] RBAC: Authorizing `identity:create_grant(project_id=49548eb44d62415582af820ab66decee, user_id=768cb67c1ef24251aaa113c12d81a250, role_id=d41c162f8e1748989a09382ced5ef3a6)` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:35:17.337093 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f23de39a-a999-4471-be65-d531eb2643b2 None admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:35:17.354362 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 467/937] 10.222.0.13 () {58 vars in 1439 bytes} [Wed Aug 31 09:35:17 2022] PUT /identity/v3/projects/49548eb44d62415582af820ab66decee/users/768cb67c1ef24251aaa113c12d81a250/roles/d41c162f8e1748989a09382ced5ef3a6 => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:35:18.334512 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1de4aa6-4e96-467f-9316-be53735f9e62 None None] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:18.335083 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1de4aa6-4e96-467f-9316-be53735f9e62 None None] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:18.335716 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1de4aa6-4e96-467f-9316-be53735f9e62 None None] PATH_INFO: `/` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:18.338909 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 471/938] 10.222.0.13 () {54 vars in 927 bytes} [Wed Aug 31 09:35:18 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:35:18.348390 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154883f7-ecd0-41e0-a94e-55459c256080 None None] REQUEST_METHOD: `POST` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:18.349145 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154883f7-ecd0-41e0-a94e-55459c256080 None None] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:18.349145 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154883f7-ecd0-41e0-a94e-55459c256080 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:18.369966 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.auth.core [None req-154883f7-ecd0-41e0-a94e-55459c256080 None None] MFA Rules not processed for user `fd836cf590a145df9acf19cf3ae2ac56`. Rule list: `[]` (Enabled: `True`). {{(pid=88114) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:35:18.371188 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.fernet_utils [None req-154883f7-ecd0-41e0-a94e-55459c256080 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88114) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:35:18.451360 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 468/939] 10.222.0.13 () {58 vars in 1024 bytes} [Wed Aug 31 09:35:18 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 31 09:35:18.729250 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f285f1-978b-4726-a0e7-6853dd483e15 None None] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:18.729760 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f285f1-978b-4726-a0e7-6853dd483e15 None None] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:18.730256 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f285f1-978b-4726-a0e7-6853dd483e15 None None] PATH_INFO: `/` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:18.733432 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 472/940] 10.222.0.13 () {54 vars in 927 bytes} [Wed Aug 31 09:35:18 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:35:18.741801 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca25c64b-565f-47d9-9787-5b3dc88a0368 None None] REQUEST_METHOD: `POST` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:18.742084 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca25c64b-565f-47d9-9787-5b3dc88a0368 None None] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:18.742447 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca25c64b-565f-47d9-9787-5b3dc88a0368 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:18.764212 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.auth.core [None req-ca25c64b-565f-47d9-9787-5b3dc88a0368 None None] MFA Rules not processed for user `fd836cf590a145df9acf19cf3ae2ac56`. Rule list: `[]` (Enabled: `True`). {{(pid=88114) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:35:18.765359 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.fernet_utils [None req-ca25c64b-565f-47d9-9787-5b3dc88a0368 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88114) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:35:18.769461 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:35:18.860084 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 469/941] 10.222.0.13 () {58 vars in 1024 bytes} [Wed Aug 31 09:35:18 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 31 09:35:18.980421 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f285f1-978b-4726-a0e7-6853dd483e15 None None] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:35:19.173845 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=100 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:35:19.182621 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f285f1-978b-4726-a0e7-6853dd483e15 None None] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:35:19.213812 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca25c64b-565f-47d9-9787-5b3dc88a0368 None None] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:35:19.256889 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5230cee9-5457-4f9d-b2c8-a8eceb6d5a59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:35:19.258129 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5230cee9-5457-4f9d-b2c8-a8eceb6d5a59 None admin] REQUEST_METHOD: `POST` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:19.258409 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5230cee9-5457-4f9d-b2c8-a8eceb6d5a59 None admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:19.258674 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5230cee9-5457-4f9d-b2c8-a8eceb6d5a59 None admin] PATH_INFO: `/v3/projects` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:19.259344 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5230cee9-5457-4f9d-b2c8-a8eceb6d5a59 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:35:19.260811 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5230cee9-5457-4f9d-b2c8-a8eceb6d5a59 None admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:35:19.265097 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: WARNING keystone.server.flask.common [None req-5230cee9-5457-4f9d-b2c8-a8eceb6d5a59 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Aug 31 09:35:19.289992 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.sql.core [None req-5230cee9-5457-4f9d-b2c8-a8eceb6d5a59 None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Aug 31 09:35:19.289992 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Aug 31 09:35:19.289992 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [parameters: {'id': '1411cd8d86ad4473a2aad4999e4c16eb', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Aug 31 09:35:19.289992 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=88113) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Aug 31 09:35:19.313079 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application [None req-5230cee9-5457-4f9d-b2c8-a8eceb6d5a59 None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 1411cd8d86ad4473a2aad4999e4c16eb.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 1411cd8d86ad4473a2aad4999e4c16eb. Aug 31 09:35:19.313079 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:35:19.313079 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 31 09:35:19.313079 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 31 09:35:19.313079 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 31 09:35:19.313079 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 31 09:35:19.313079 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 31 09:35:19.313079 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application result = self._query(query) Aug 31 09:35:19.313079 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 31 09:35:19.313079 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application conn.query(q) Aug 31 09:35:19.313079 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 31 09:35:19.313079 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 31 09:35:19.313079 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 31 09:35:19.314672 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application result.read() Aug 31 09:35:19.314672 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 31 09:35:19.314672 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 31 09:35:19.314672 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 31 09:35:19.314672 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 31 09:35:19.314672 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 31 09:35:19.314672 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 31 09:35:19.314672 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 31 09:35:19.314672 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 31 09:35:19.314672 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Aug 31 09:35:19.314672 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application Aug 31 09:35:19.314672 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Aug 31 09:35:19.314672 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application Aug 31 09:35:19.314672 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:35:19.314672 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Aug 31 09:35:19.314672 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return method(*args, **kwargs) Aug 31 09:35:19.314672 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Aug 31 09:35:19.314672 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return project_ref.to_dict() Aug 31 09:35:19.316480 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 31 09:35:19.316480 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application next(self.gen) Aug 31 09:35:19.316480 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Aug 31 09:35:19.316480 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application yield resource Aug 31 09:35:19.316480 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 31 09:35:19.316480 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application next(self.gen) Aug 31 09:35:19.316480 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Aug 31 09:35:19.316480 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application self.session.rollback() Aug 31 09:35:19.316480 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Aug 31 09:35:19.316480 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application self.force_reraise() Aug 31 09:35:19.316480 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Aug 31 09:35:19.316480 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application raise self.value Aug 31 09:35:19.316480 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Aug 31 09:35:19.316480 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Aug 31 09:35:19.316480 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Aug 31 09:35:19.316480 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application session.commit() Aug 31 09:35:19.316480 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Aug 31 09:35:19.318123 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Aug 31 09:35:19.318123 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Aug 31 09:35:19.318123 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application self._prepare_impl() Aug 31 09:35:19.318123 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Aug 31 09:35:19.318123 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application self.session.flush() Aug 31 09:35:19.318123 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Aug 31 09:35:19.318123 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application self._flush(objects) Aug 31 09:35:19.318123 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Aug 31 09:35:19.318123 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Aug 31 09:35:19.318123 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Aug 31 09:35:19.318123 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application compat.raise_( Aug 31 09:35:19.318123 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 31 09:35:19.318123 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application raise exception Aug 31 09:35:19.318123 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Aug 31 09:35:19.318123 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application flush_context.execute() Aug 31 09:35:19.318123 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Aug 31 09:35:19.318123 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application rec.execute(self) Aug 31 09:35:19.321999 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Aug 31 09:35:19.321999 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Aug 31 09:35:19.321999 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Aug 31 09:35:19.321999 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application _emit_insert_statements( Aug 31 09:35:19.321999 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Aug 31 09:35:19.321999 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application c = connection._execute_20( Aug 31 09:35:19.321999 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Aug 31 09:35:19.321999 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Aug 31 09:35:19.321999 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Aug 31 09:35:19.321999 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Aug 31 09:35:19.321999 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Aug 31 09:35:19.321999 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application ret = self._execute_context( Aug 31 09:35:19.321999 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Aug 31 09:35:19.321999 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Aug 31 09:35:19.321999 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Aug 31 09:35:19.324396 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Aug 31 09:35:19.324396 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 31 09:35:19.324396 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application raise exception Aug 31 09:35:19.324396 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 31 09:35:19.324396 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 31 09:35:19.324396 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 31 09:35:19.324396 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 31 09:35:19.324396 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 31 09:35:19.324396 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application result = self._query(query) Aug 31 09:35:19.324396 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 31 09:35:19.324396 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application conn.query(q) Aug 31 09:35:19.324396 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 31 09:35:19.324396 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 31 09:35:19.324396 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 31 09:35:19.324396 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application result.read() Aug 31 09:35:19.324396 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 31 09:35:19.324396 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 31 09:35:19.327714 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 31 09:35:19.327714 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 31 09:35:19.327714 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 31 09:35:19.327714 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 31 09:35:19.327714 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 31 09:35:19.327714 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 31 09:35:19.327714 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Aug 31 09:35:19.327714 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Aug 31 09:35:19.327714 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application [parameters: {'id': '1411cd8d86ad4473a2aad4999e4c16eb', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Aug 31 09:35:19.327714 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Aug 31 09:35:19.327714 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application Aug 31 09:35:19.327714 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 31 09:35:19.327714 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application Aug 31 09:35:19.327714 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:35:19.327714 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Aug 31 09:35:19.330158 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Aug 31 09:35:19.330158 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Aug 31 09:35:19.330158 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Aug 31 09:35:19.330158 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Aug 31 09:35:19.330158 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application Aug 31 09:35:19.330158 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 31 09:35:19.330158 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application Aug 31 09:35:19.330158 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:35:19.330158 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:35:19.330158 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:35:19.330158 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:35:19.330158 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:35:19.330158 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:35:19.330158 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:35:19.330158 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:35:19.330158 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:35:19.330158 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:35:19.334270 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:35:19.334270 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Aug 31 09:35:19.334270 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Aug 31 09:35:19.334270 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:35:19.334270 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:35:19.334270 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Aug 31 09:35:19.334270 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application raise exception.Conflict( Aug 31 09:35:19.334270 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 1411cd8d86ad4473a2aad4999e4c16eb. Aug 31 09:35:19.334270 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application Aug 31 09:35:19.334270 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 473/942] 10.222.0.13 () {60 vars in 1140 bytes} [Wed Aug 31 09:35:18 2022] POST /identity/v3/projects => generated 262 bytes in 333 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:35:19.334270 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5230cee9-5457-4f9d-b2c8-a8eceb6d5a59 None admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:35:19.419259 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca25c64b-565f-47d9-9787-5b3dc88a0368 None None] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:35:19.468180 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad74a4a-721b-4b4c-81ed-50dcc81507b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:35:19.469331 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad74a4a-721b-4b4c-81ed-50dcc81507b3 None admin] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:19.469725 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad74a4a-721b-4b4c-81ed-50dcc81507b3 None admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:19.470065 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad74a4a-721b-4b4c-81ed-50dcc81507b3 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:19.470758 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ad74a4a-721b-4b4c-81ed-50dcc81507b3 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:35:19.472260 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ad74a4a-721b-4b4c-81ed-50dcc81507b3 None admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:35:19.483504 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application [None req-7ad74a4a-721b-4b4c-81ed-50dcc81507b3 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 31 09:35:19.483504 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:35:19.483504 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 31 09:35:19.483504 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 31 09:35:19.483504 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 31 09:35:19.483504 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 31 09:35:19.483504 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 31 09:35:19.483504 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 09:35:19.483504 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 31 09:35:19.483504 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application Aug 31 09:35:19.483504 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 31 09:35:19.483504 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application Aug 31 09:35:19.483504 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:35:19.483504 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:35:19.483504 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:35:19.483504 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:35:19.483504 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:35:19.485459 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:35:19.485459 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:35:19.485459 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:35:19.485459 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:35:19.485459 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:35:19.485459 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:35:19.485459 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 31 09:35:19.485459 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 31 09:35:19.485459 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 31 09:35:19.485459 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 31 09:35:19.485459 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:35:19.485459 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:35:19.485459 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:35:19.485459 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:35:19.485459 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:35:19.485459 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:35:19.487094 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:35:19.487094 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application with Lock( Aug 31 09:35:19.487094 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:35:19.487094 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:35:19.487094 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:35:19.487094 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:35:19.487094 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:35:19.487094 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:35:19.487094 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:35:19.487094 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:35:19.487094 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 31 09:35:19.487094 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 09:35:19.487094 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 31 09:35:19.487094 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application Aug 31 09:35:19.489652 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 470/943] 10.222.0.13 () {56 vars in 1126 bytes} [Wed Aug 31 09:35:19 2022] GET /identity/v3/services/object-store => generated 93 bytes in 272 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:35:19.491029 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad74a4a-721b-4b4c-81ed-50dcc81507b3 None admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:35:19.495060 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=100 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:35:19.514439 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5230cee9-5457-4f9d-b2c8-a8eceb6d5a59 None admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:35:19.592396 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f578a5-aaa9-4195-b418-aa73e54a2ba5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:35:19.594236 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f578a5-aaa9-4195-b418-aa73e54a2ba5 None admin] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:19.595147 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f578a5-aaa9-4195-b418-aa73e54a2ba5 None admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:19.595907 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f578a5-aaa9-4195-b418-aa73e54a2ba5 None admin] PATH_INFO: `/v3/projects/service` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:19.610456 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0f578a5-aaa9-4195-b418-aa73e54a2ba5 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:35:19.614132 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0f578a5-aaa9-4195-b418-aa73e54a2ba5 None admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:35:19.637087 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application [None req-e0f578a5-aaa9-4195-b418-aa73e54a2ba5 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 09:35:19.637087 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:35:19.637087 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:35:19.637087 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:35:19.637087 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:35:19.637087 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:35:19.637087 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:35:19.637087 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:35:19.637087 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:35:19.637087 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:35:19.637087 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:35:19.637087 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:35:19.637087 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 31 09:35:19.637087 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 31 09:35:19.637087 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 31 09:35:19.639734 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 31 09:35:19.639734 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:35:19.639734 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:35:19.639734 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:35:19.639734 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:35:19.639734 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:35:19.639734 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:35:19.639734 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:35:19.639734 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application with Lock( Aug 31 09:35:19.639734 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:35:19.639734 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:35:19.639734 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:35:19.639734 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:35:19.639734 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:35:19.639734 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:35:19.639734 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:35:19.639734 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:35:19.641536 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 31 09:35:19.641536 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 31 09:35:19.641536 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 31 09:35:19.641536 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 31 09:35:19.641536 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 31 09:35:19.641536 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 31 09:35:19.641536 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 09:35:19.641536 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application Aug 31 09:35:19.652954 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 474/944] 10.222.0.13 () {56 vars in 1111 bytes} [Wed Aug 31 09:35:19 2022] GET /identity/v3/projects/service => generated 88 bytes in 330 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:35:19.657898 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f578a5-aaa9-4195-b418-aa73e54a2ba5 None admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:35:19.687673 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad74a4a-721b-4b4c-81ed-50dcc81507b3 None admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:35:19.690178 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:35:19.737234 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ad0482-e6d4-4cb2-9677-f77a57c0f981 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:35:19.737946 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ad0482-e6d4-4cb2-9677-f77a57c0f981 None admin] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:19.738238 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ad0482-e6d4-4cb2-9677-f77a57c0f981 None admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:19.738493 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ad0482-e6d4-4cb2-9677-f77a57c0f981 None admin] PATH_INFO: `/v3/services` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:19.739139 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88ad0482-e6d4-4cb2-9677-f77a57c0f981 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=88114) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:35:19.739441 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88ad0482-e6d4-4cb2-9677-f77a57c0f981 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:35:19.740836 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88ad0482-e6d4-4cb2-9677-f77a57c0f981 None admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:35:19.751305 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 471/945] 10.222.0.13 () {56 vars in 1122 bytes} [Wed Aug 31 09:35:19 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:35:19.755667 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ef2624-bab4-4a7d-8c75-18fdde2f53df None None] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:19.756118 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ef2624-bab4-4a7d-8c75-18fdde2f53df None None] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:19.756530 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ef2624-bab4-4a7d-8c75-18fdde2f53df None None] PATH_INFO: `/` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:19.758661 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 472/946] 10.222.0.13 () {54 vars in 927 bytes} [Wed Aug 31 09:35:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:35:19.761133 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68ef2624-bab4-4a7d-8c75-18fdde2f53df None None] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:35:19.833513 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f578a5-aaa9-4195-b418-aa73e54a2ba5 None admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:35:19.871681 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ecbbe8-e345-4e70-9ef3-7a0f63722ed6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:35:19.872304 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ecbbe8-e345-4e70-9ef3-7a0f63722ed6 None admin] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:19.872738 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ecbbe8-e345-4e70-9ef3-7a0f63722ed6 None admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:19.873076 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ecbbe8-e345-4e70-9ef3-7a0f63722ed6 None admin] PATH_INFO: `/v3/projects` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:19.873782 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77ecbbe8-e345-4e70-9ef3-7a0f63722ed6 None admin] RBAC: Adding query filter params (name=service) {{(pid=88113) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:35:19.874085 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77ecbbe8-e345-4e70-9ef3-7a0f63722ed6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:35:19.875530 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77ecbbe8-e345-4e70-9ef3-7a0f63722ed6 None admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:35:19.878456 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=100 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:35:19.898760 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 475/947] 10.222.0.13 () {56 vars in 1112 bytes} [Wed Aug 31 09:35:19 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 09:35:19.900107 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe1f21d-aa1c-498e-a8e9-a05a001d1db6 None None] REQUEST_METHOD: `POST` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:19.900456 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe1f21d-aa1c-498e-a8e9-a05a001d1db6 None None] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:19.900622 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe1f21d-aa1c-498e-a8e9-a05a001d1db6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:19.922739 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.auth.core [None req-7fe1f21d-aa1c-498e-a8e9-a05a001d1db6 None None] MFA Rules not processed for user `fd836cf590a145df9acf19cf3ae2ac56`. Rule list: `[]` (Enabled: `True`). {{(pid=88113) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:35:19.924458 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.fernet_utils [None req-7fe1f21d-aa1c-498e-a8e9-a05a001d1db6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:35:19.939043 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68ef2624-bab4-4a7d-8c75-18fdde2f53df None None] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:35:20.000844 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb7588aa-6c40-40de-bce7-283ac43220fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:35:20.001846 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7588aa-6c40-40de-bce7-283ac43220fe None admin] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:20.002255 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7588aa-6c40-40de-bce7-283ac43220fe None admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:20.002657 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7588aa-6c40-40de-bce7-283ac43220fe None admin] PATH_INFO: `/v3/services` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:20.003650 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb7588aa-6c40-40de-bce7-283ac43220fe None admin] RBAC: Adding query filter params (type=object-store) {{(pid=88114) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:35:20.004121 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb7588aa-6c40-40de-bce7-283ac43220fe None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:35:20.006471 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb7588aa-6c40-40de-bce7-283ac43220fe None admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:35:20.021432 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 473/948] 10.222.0.13 () {56 vars in 1122 bytes} [Wed Aug 31 09:35:19 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 09:35:20.021432 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3456dde-f481-44ee-a32f-212034af9941 None None] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:20.021432 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3456dde-f481-44ee-a32f-212034af9941 None None] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:20.021432 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3456dde-f481-44ee-a32f-212034af9941 None None] PATH_INFO: `/` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:20.021432 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 474/949] 10.222.0.13 () {54 vars in 927 bytes} [Wed Aug 31 09:35:20 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:35:20.027597 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3456dde-f481-44ee-a32f-212034af9941 None None] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:35:20.033475 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 476/950] 10.222.0.13 () {58 vars in 1024 bytes} [Wed Aug 31 09:35:19 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 09:35:20.034803 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e6e21e-b55d-4199-bb7d-36d4a1dd92b0 None None] REQUEST_METHOD: `POST` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:20.035085 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e6e21e-b55d-4199-bb7d-36d4a1dd92b0 None None] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:20.035357 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e6e21e-b55d-4199-bb7d-36d4a1dd92b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:20.044263 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=100 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:35:20.067761 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.auth.core [None req-c0e6e21e-b55d-4199-bb7d-36d4a1dd92b0 None None] MFA Rules not processed for user `fd836cf590a145df9acf19cf3ae2ac56`. Rule list: `[]` (Enabled: `True`). {{(pid=88113) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:35:20.070131 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.fernet_utils [None req-c0e6e21e-b55d-4199-bb7d-36d4a1dd92b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:35:20.178688 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 477/951] 10.222.0.13 () {58 vars in 1024 bytes} [Wed Aug 31 09:35:20 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 09:35:20.214446 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3456dde-f481-44ee-a32f-212034af9941 None None] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:35:20.270108 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=100 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:35:20.277952 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b68f0e00-2230-4fd4-925c-f1e0d3063882 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:35:20.279601 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68f0e00-2230-4fd4-925c-f1e0d3063882 None admin] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:20.280266 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68f0e00-2230-4fd4-925c-f1e0d3063882 None admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:20.281025 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68f0e00-2230-4fd4-925c-f1e0d3063882 None admin] PATH_INFO: `/v3/endpoints` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:20.282058 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b68f0e00-2230-4fd4-925c-f1e0d3063882 None admin] RBAC: Adding query filter params (interface=public, service_id=a0f5cdb3f7914db5b814282cf8b2490d, region_id=RegionOne) {{(pid=88114) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:35:20.282407 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b68f0e00-2230-4fd4-925c-f1e0d3063882 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:35:20.284361 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b68f0e00-2230-4fd4-925c-f1e0d3063882 None admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:35:20.291906 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 475/952] 10.222.0.13 () {56 vars in 1251 bytes} [Wed Aug 31 09:35:20 2022] GET /identity/v3/endpoints?service_id=a0f5cdb3f7914db5b814282cf8b2490d&interface=public®ion_id=RegionOne => generated 194 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 09:35:20.297374 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e6e21e-b55d-4199-bb7d-36d4a1dd92b0 None None] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:35:20.407715 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e6e21e-b55d-4199-bb7d-36d4a1dd92b0 None None] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:35:20.448548 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c01384b7-3cf8-4020-bdeb-3492c9a69bbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:35:20.449600 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01384b7-3cf8-4020-bdeb-3492c9a69bbe None admin] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:20.450020 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01384b7-3cf8-4020-bdeb-3492c9a69bbe None admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:20.450445 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01384b7-3cf8-4020-bdeb-3492c9a69bbe None admin] PATH_INFO: `/v3/services` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:20.451814 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c01384b7-3cf8-4020-bdeb-3492c9a69bbe None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:35:20.453397 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c01384b7-3cf8-4020-bdeb-3492c9a69bbe None admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:35:20.454975 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=100 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:35:20.455459 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b68f0e00-2230-4fd4-925c-f1e0d3063882 None admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:35:20.461371 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 478/953] 10.222.0.13 () {56 vars in 1087 bytes} [Wed Aug 31 09:35:20 2022] GET /identity/v3/services => generated 1497 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 09:35:20.491255 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c01384b7-3cf8-4020-bdeb-3492c9a69bbe None admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:35:20.642173 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c01384b7-3cf8-4020-bdeb-3492c9a69bbe None admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:35:20.649040 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b68f0e00-2230-4fd4-925c-f1e0d3063882 None admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:35:20.689709 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-906a278a-36a9-45c2-8f6d-3636196b391a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:35:20.691281 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906a278a-36a9-45c2-8f6d-3636196b391a None admin] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:20.691281 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906a278a-36a9-45c2-8f6d-3636196b391a None admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:20.691578 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906a278a-36a9-45c2-8f6d-3636196b391a None admin] PATH_INFO: `/v3/services/compute` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:20.692493 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-906a278a-36a9-45c2-8f6d-3636196b391a None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:35:20.695231 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-906a278a-36a9-45c2-8f6d-3636196b391a None admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:35:20.706756 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f508a957-f4bb-4dea-abd4-54189856ec5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:35:20.707487 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f508a957-f4bb-4dea-abd4-54189856ec5f None admin] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:20.707862 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f508a957-f4bb-4dea-abd4-54189856ec5f None admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:20.708231 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f508a957-f4bb-4dea-abd4-54189856ec5f None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:20.715243 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application [None req-906a278a-36a9-45c2-8f6d-3636196b391a None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Aug 31 09:35:20.715243 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:35:20.715243 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 31 09:35:20.715243 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 31 09:35:20.715243 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 31 09:35:20.715243 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 31 09:35:20.715243 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 31 09:35:20.715243 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 09:35:20.715243 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 31 09:35:20.715243 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application Aug 31 09:35:20.715243 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 31 09:35:20.715243 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application Aug 31 09:35:20.715243 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:35:20.715243 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:35:20.715243 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:35:20.715243 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:35:20.715243 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:35:20.717254 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:35:20.717254 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:35:20.717254 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:35:20.717254 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:35:20.717254 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:35:20.717254 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:35:20.717254 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 31 09:35:20.717254 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 31 09:35:20.717254 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 31 09:35:20.717254 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 31 09:35:20.717254 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:35:20.717254 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:35:20.717254 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:35:20.717254 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:35:20.717254 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:35:20.717254 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:35:20.718215 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:35:20.718215 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application with Lock( Aug 31 09:35:20.718215 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:35:20.718215 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:35:20.718215 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:35:20.718215 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:35:20.718215 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:35:20.718215 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:35:20.718215 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:35:20.718215 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:35:20.718215 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 31 09:35:20.718215 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 09:35:20.718215 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 31 09:35:20.718215 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application Aug 31 09:35:20.719092 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 479/954] 10.222.0.13 () {56 vars in 1111 bytes} [Wed Aug 31 09:35:20 2022] GET /identity/v3/services/compute => generated 88 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 09:35:20.726292 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f508a957-f4bb-4dea-abd4-54189856ec5f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 09:35:20.728404 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-906a278a-36a9-45c2-8f6d-3636196b391a None admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:35:20.732891 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application [None req-f508a957-f4bb-4dea-abd4-54189856ec5f None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 31 09:35:20.732891 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:35:20.732891 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:35:20.732891 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:35:20.732891 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:35:20.732891 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:35:20.732891 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:35:20.732891 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:35:20.732891 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:35:20.732891 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:35:20.732891 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:35:20.732891 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:35:20.732891 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 09:35:20.732891 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 09:35:20.732891 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 09:35:20.732891 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 09:35:20.733920 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 09:35:20.733920 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 09:35:20.733920 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 09:35:20.733920 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 09:35:20.733920 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:35:20.733920 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:35:20.733920 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:35:20.733920 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:35:20.733920 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:35:20.733920 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:35:20.733920 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:35:20.733920 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application with Lock( Aug 31 09:35:20.733920 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:35:20.733920 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:35:20.733920 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:35:20.733920 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:35:20.734763 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:35:20.734763 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:35:20.734763 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:35:20.734763 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:35:20.734763 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 09:35:20.734763 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 09:35:20.734763 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 09:35:20.734763 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 09:35:20.734763 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 09:35:20.734763 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 09:35:20.734763 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 31 09:35:20.734763 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application Aug 31 09:35:20.737536 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 476/955] 10.222.0.13 () {56 vars in 1120 bytes} [Wed Aug 31 09:35:20 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 283 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:35:20.745215 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f508a957-f4bb-4dea-abd4-54189856ec5f None admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:35:20.811103 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=100 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:35:20.858994 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=100 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:35:20.862565 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f508a957-f4bb-4dea-abd4-54189856ec5f None admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:35:20.904874 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc7c993-b742-4dc7-9579-a0703e2de10a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:35:20.906682 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc7c993-b742-4dc7-9579-a0703e2de10a None admin] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:20.907120 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc7c993-b742-4dc7-9579-a0703e2de10a None admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:20.907717 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc7c993-b742-4dc7-9579-a0703e2de10a None admin] PATH_INFO: `/v3/roles` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:20.908977 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddc7c993-b742-4dc7-9579-a0703e2de10a None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=88114) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:35:20.909476 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddc7c993-b742-4dc7-9579-a0703e2de10a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:35:20.910431 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-906a278a-36a9-45c2-8f6d-3636196b391a None admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:35:20.911267 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddc7c993-b742-4dc7-9579-a0703e2de10a None admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:35:20.927931 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 477/956] 10.222.0.13 () {56 vars in 1115 bytes} [Wed Aug 31 09:35:20 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 09:35:20.930142 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc7c993-b742-4dc7-9579-a0703e2de10a None admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:35:20.952280 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80810dcd-9264-4c75-8850-081f872da3a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:35:20.953196 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80810dcd-9264-4c75-8850-081f872da3a3 None admin] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:20.953668 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80810dcd-9264-4c75-8850-081f872da3a3 None admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:20.954099 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80810dcd-9264-4c75-8850-081f872da3a3 None admin] PATH_INFO: `/v3/services` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:20.954896 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80810dcd-9264-4c75-8850-081f872da3a3 None admin] RBAC: Adding query filter params (name=compute) {{(pid=88113) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:35:20.955433 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80810dcd-9264-4c75-8850-081f872da3a3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:35:20.957158 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80810dcd-9264-4c75-8850-081f872da3a3 None admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:35:20.967799 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 480/957] 10.222.0.13 () {56 vars in 1112 bytes} [Wed Aug 31 09:35:20 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:35:20.970424 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80810dcd-9264-4c75-8850-081f872da3a3 None admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:35:21.067298 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc7c993-b742-4dc7-9579-a0703e2de10a None admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:35:21.093814 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=100 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:35:21.112470 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f6e735-30b8-435d-80d5-ca11ee027641 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:35:21.113299 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f6e735-30b8-435d-80d5-ca11ee027641 None admin] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:21.113737 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f6e735-30b8-435d-80d5-ca11ee027641 None admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:21.114118 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f6e735-30b8-435d-80d5-ca11ee027641 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:21.128905 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78f6e735-30b8-435d-80d5-ca11ee027641 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:35:21.130740 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78f6e735-30b8-435d-80d5-ca11ee027641 None admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:35:21.149481 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80810dcd-9264-4c75-8850-081f872da3a3 None admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:35:21.150650 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application [None req-78f6e735-30b8-435d-80d5-ca11ee027641 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 09:35:21.150650 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:35:21.150650 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:35:21.150650 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:35:21.150650 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:35:21.150650 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:35:21.150650 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:35:21.150650 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:35:21.150650 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:35:21.150650 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:35:21.150650 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:35:21.150650 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:35:21.150650 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 09:35:21.150650 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 09:35:21.150650 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 09:35:21.150650 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 09:35:21.152582 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:35:21.152582 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:35:21.152582 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:35:21.152582 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:35:21.152582 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:35:21.152582 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:35:21.152582 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:35:21.152582 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application with Lock( Aug 31 09:35:21.152582 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:35:21.152582 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:35:21.152582 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:35:21.152582 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:35:21.152582 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:35:21.152582 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:35:21.152582 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:35:21.152582 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:35:21.152582 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 09:35:21.155927 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 09:35:21.155927 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 09:35:21.155927 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application Aug 31 09:35:21.159500 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 478/958] 10.222.0.13 () {56 vars in 1108 bytes} [Wed Aug 31 09:35:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 09:35:21.160436 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f6e735-30b8-435d-80d5-ca11ee027641 None admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:35:21.214096 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=100 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:35:21.233249 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-384c1d52-b04a-4c61-a31b-4aa3a5535664 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:35:21.234845 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384c1d52-b04a-4c61-a31b-4aa3a5535664 None admin] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:21.235572 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384c1d52-b04a-4c61-a31b-4aa3a5535664 None admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:21.236283 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384c1d52-b04a-4c61-a31b-4aa3a5535664 None admin] PATH_INFO: `/v3/services` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:21.237732 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-384c1d52-b04a-4c61-a31b-4aa3a5535664 None admin] RBAC: Adding query filter params (type=compute) {{(pid=88113) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:35:21.238663 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-384c1d52-b04a-4c61-a31b-4aa3a5535664 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:35:21.241664 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-384c1d52-b04a-4c61-a31b-4aa3a5535664 None admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:35:21.255752 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 481/959] 10.222.0.13 () {56 vars in 1112 bytes} [Wed Aug 31 09:35:20 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 281 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 09:35:21.306111 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f6e735-30b8-435d-80d5-ca11ee027641 None admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:35:21.367837 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d30f4d4-616f-47c6-aba8-353d21beed05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:35:21.368842 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d30f4d4-616f-47c6-aba8-353d21beed05 None admin] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:21.369293 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d30f4d4-616f-47c6-aba8-353d21beed05 None admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:21.369863 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d30f4d4-616f-47c6-aba8-353d21beed05 None admin] PATH_INFO: `/v3/domains` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:21.371007 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d30f4d4-616f-47c6-aba8-353d21beed05 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88114) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:35:21.371511 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d30f4d4-616f-47c6-aba8-353d21beed05 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:35:21.374035 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d30f4d4-616f-47c6-aba8-353d21beed05 None admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:35:21.397113 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 479/960] 10.222.0.13 () {56 vars in 1109 bytes} [Wed Aug 31 09:35:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:35:21.402757 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-384c1d52-b04a-4c61-a31b-4aa3a5535664 None admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:35:21.533507 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-384c1d52-b04a-4c61-a31b-4aa3a5535664 None admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:35:21.583991 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f298b73-da3a-4d2a-a0fb-ba46bb5b23a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:35:21.584634 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f298b73-da3a-4d2a-a0fb-ba46bb5b23a9 None admin] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:21.584915 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f298b73-da3a-4d2a-a0fb-ba46bb5b23a9 None admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:21.585183 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f298b73-da3a-4d2a-a0fb-ba46bb5b23a9 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:21.595444 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f298b73-da3a-4d2a-a0fb-ba46bb5b23a9 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:35:21.596912 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f298b73-da3a-4d2a-a0fb-ba46bb5b23a9 None admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:35:21.609881 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application [None req-7f298b73-da3a-4d2a-a0fb-ba46bb5b23a9 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 31 09:35:21.609881 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:35:21.609881 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:35:21.609881 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:35:21.609881 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:35:21.609881 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:35:21.609881 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:35:21.609881 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:35:21.609881 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:35:21.609881 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:35:21.609881 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:35:21.609881 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:35:21.609881 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 31 09:35:21.609881 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 31 09:35:21.609881 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 31 09:35:21.609881 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 31 09:35:21.610928 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:35:21.610928 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:35:21.610928 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 31 09:35:21.610928 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 09:35:21.610928 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 31 09:35:21.610928 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 09:35:21.610928 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:35:21.610928 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:35:21.610928 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:35:21.610928 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:35:21.610928 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:35:21.610928 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application with Lock( Aug 31 09:35:21.610928 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:35:21.610928 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:35:21.610928 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:35:21.610928 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:35:21.610928 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:35:21.610928 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:35:21.611894 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:35:21.611894 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:35:21.611894 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 31 09:35:21.611894 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 31 09:35:21.611894 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 31 09:35:21.611894 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 31 09:35:21.611894 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 31 09:35:21.611894 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 31 09:35:21.611894 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 31 09:35:21.611894 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application Aug 31 09:35:21.611894 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 482/961] 10.222.0.13 () {56 vars in 1117 bytes} [Wed Aug 31 09:35:21 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:35:21.617172 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d30f4d4-616f-47c6-aba8-353d21beed05 None admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:35:21.656563 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=100 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:35:21.735734 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d30f4d4-616f-47c6-aba8-353d21beed05 None admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:35:21.774488 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9697062-4c9f-40cb-8f9c-f2680cacaeb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:35:21.775100 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9697062-4c9f-40cb-8f9c-f2680cacaeb5 None admin] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:21.775423 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9697062-4c9f-40cb-8f9c-f2680cacaeb5 None admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:21.775692 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9697062-4c9f-40cb-8f9c-f2680cacaeb5 None admin] PATH_INFO: `/v3/users` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:21.776418 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9697062-4c9f-40cb-8f9c-f2680cacaeb5 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=88114) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:35:21.776715 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9697062-4c9f-40cb-8f9c-f2680cacaeb5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:35:21.778164 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9697062-4c9f-40cb-8f9c-f2680cacaeb5 None admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:35:21.800016 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 480/962] 10.222.0.13 () {56 vars in 1149 bytes} [Wed Aug 31 09:35:21 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:35:21.805433 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f298b73-da3a-4d2a-a0fb-ba46bb5b23a9 None admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:35:21.889850 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=100 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:35:21.950801 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f298b73-da3a-4d2a-a0fb-ba46bb5b23a9 None admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:35:21.989361 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a3826a-d683-4ae1-b4db-aff249c0ac72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:35:21.990424 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a3826a-d683-4ae1-b4db-aff249c0ac72 None admin] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:21.990921 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a3826a-d683-4ae1-b4db-aff249c0ac72 None admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:21.991402 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a3826a-d683-4ae1-b4db-aff249c0ac72 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:22.072234 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5a3826a-d683-4ae1-b4db-aff249c0ac72 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:35:22.074091 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5a3826a-d683-4ae1-b4db-aff249c0ac72 None admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:35:22.137299 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=100 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:35:22.196162 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 483/963] 10.222.0.13 () {58 vars in 1282 bytes} [Wed Aug 31 09:35:21 2022] GET /identity/v3/auth/tokens => generated 1474 bytes in 391 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 31 09:35:22.202476 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9697062-4c9f-40cb-8f9c-f2680cacaeb5 None admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:35:22.311882 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=100 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:35:22.328581 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9697062-4c9f-40cb-8f9c-f2680cacaeb5 None admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:35:22.349479 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44df715-56c0-4ec8-925a-62403144d930 None None] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:22.349737 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44df715-56c0-4ec8-925a-62403144d930 None None] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:22.350020 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44df715-56c0-4ec8-925a-62403144d930 None None] PATH_INFO: `/` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:22.351565 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 484/964] 10.222.0.13 () {54 vars in 927 bytes} [Wed Aug 31 09:35:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:35:22.358374 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f215145-ef49-4e06-9b57-013035d445ae None None] REQUEST_METHOD: `POST` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:22.358654 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f215145-ef49-4e06-9b57-013035d445ae None None] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:22.358920 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f215145-ef49-4e06-9b57-013035d445ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:22.380397 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.auth.core [None req-9f215145-ef49-4e06-9b57-013035d445ae None None] MFA Rules not processed for user `fd836cf590a145df9acf19cf3ae2ac56`. Rule list: `[]` (Enabled: `True`). {{(pid=88113) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:35:22.381638 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.fernet_utils [None req-9f215145-ef49-4e06-9b57-013035d445ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:35:22.397116 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b13101-6283-4eea-aa50-b5cb2b092485 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:35:22.398796 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b13101-6283-4eea-aa50-b5cb2b092485 None admin] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:22.399562 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b13101-6283-4eea-aa50-b5cb2b092485 None admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:22.400283 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b13101-6283-4eea-aa50-b5cb2b092485 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:22.421661 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9b13101-6283-4eea-aa50-b5cb2b092485 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:35:22.425129 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9b13101-6283-4eea-aa50-b5cb2b092485 None admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:35:22.456474 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application [None req-f9b13101-6283-4eea-aa50-b5cb2b092485 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 09:35:22.456474 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:35:22.456474 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:35:22.456474 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:35:22.456474 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:35:22.456474 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:35:22.456474 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:35:22.456474 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:35:22.456474 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:35:22.456474 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:35:22.456474 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:35:22.456474 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:35:22.456474 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 09:35:22.456474 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 09:35:22.456474 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 09:35:22.456474 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 09:35:22.458322 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:35:22.458322 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:35:22.458322 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:35:22.458322 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:35:22.458322 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:35:22.458322 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:35:22.458322 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:35:22.458322 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application with Lock( Aug 31 09:35:22.458322 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:35:22.458322 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:35:22.458322 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:35:22.458322 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:35:22.458322 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:35:22.458322 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:35:22.458322 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:35:22.458322 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:35:22.458322 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 09:35:22.460089 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 09:35:22.460089 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 09:35:22.460089 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application Aug 31 09:35:22.469409 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 485/966] 10.222.0.13 () {58 vars in 1024 bytes} [Wed Aug 31 09:35:22 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 31 09:35:22.469788 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 481/966] 10.222.0.13 () {56 vars in 1108 bytes} [Wed Aug 31 09:35:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 263 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:35:22.474472 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f215145-ef49-4e06-9b57-013035d445ae None None] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:35:22.562212 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a1d350-ad2c-424c-8910-70e95d0103f4 None None] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:22.562486 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a1d350-ad2c-424c-8910-70e95d0103f4 None None] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:22.562797 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a1d350-ad2c-424c-8910-70e95d0103f4 None None] PATH_INFO: `/` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:22.565588 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 482/967] 10.222.0.13 () {54 vars in 927 bytes} [Wed Aug 31 09:35:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 31 09:35:22.570308 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ccd13bf-af2c-490d-b242-63f1adafe7ea None None] REQUEST_METHOD: `POST` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:22.570613 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ccd13bf-af2c-490d-b242-63f1adafe7ea None None] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:22.570890 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ccd13bf-af2c-490d-b242-63f1adafe7ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:22.589205 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=100 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:35:22.601064 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.auth.core [None req-4ccd13bf-af2c-490d-b242-63f1adafe7ea None None] MFA Rules not processed for user `fd836cf590a145df9acf19cf3ae2ac56`. Rule list: `[]` (Enabled: `True`). {{(pid=88114) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:35:22.603488 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.fernet_utils [None req-4ccd13bf-af2c-490d-b242-63f1adafe7ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88114) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:35:22.614475 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f215145-ef49-4e06-9b57-013035d445ae None None] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:35:22.684628 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a78645a-0b2e-42f9-8a05-ce9e724a239d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:35:22.686335 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a78645a-0b2e-42f9-8a05-ce9e724a239d None admin] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:22.686730 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a78645a-0b2e-42f9-8a05-ce9e724a239d None admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:22.687159 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a78645a-0b2e-42f9-8a05-ce9e724a239d None admin] PATH_INFO: `/v3/domains` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:22.688358 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a78645a-0b2e-42f9-8a05-ce9e724a239d None admin] RBAC: Adding query filter params (name=Default) {{(pid=88113) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:35:22.689038 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a78645a-0b2e-42f9-8a05-ce9e724a239d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:35:22.691152 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 483/968] 10.222.0.13 () {58 vars in 1024 bytes} [Wed Aug 31 09:35:22 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 09:35:22.698488 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a78645a-0b2e-42f9-8a05-ce9e724a239d None admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:35:22.719327 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 486/969] 10.222.0.13 () {56 vars in 1109 bytes} [Wed Aug 31 09:35:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:35:22.724916 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ccd13bf-af2c-490d-b242-63f1adafe7ea None None] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:35:22.742270 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a78645a-0b2e-42f9-8a05-ce9e724a239d None admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:35:22.813761 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=100 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:35:22.871574 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ccd13bf-af2c-490d-b242-63f1adafe7ea None None] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:35:22.899439 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a78645a-0b2e-42f9-8a05-ce9e724a239d None admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:35:22.914828 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24820d50-5dd6-460f-92fb-c144257f1ce6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:35:22.915829 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24820d50-5dd6-460f-92fb-c144257f1ce6 None admin] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:22.916316 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24820d50-5dd6-460f-92fb-c144257f1ce6 None admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:22.916822 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24820d50-5dd6-460f-92fb-c144257f1ce6 None admin] PATH_INFO: `/v3/projects/service` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:22.926471 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24820d50-5dd6-460f-92fb-c144257f1ce6 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:35:22.929007 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24820d50-5dd6-460f-92fb-c144257f1ce6 None admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:35:22.943109 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application [None req-24820d50-5dd6-460f-92fb-c144257f1ce6 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 09:35:22.943109 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:35:22.943109 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:35:22.943109 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:35:22.943109 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:35:22.943109 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:35:22.943109 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:35:22.943109 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:35:22.943109 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:35:22.943109 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:35:22.943109 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:35:22.943109 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:35:22.943109 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 31 09:35:22.943109 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 31 09:35:22.943109 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 31 09:35:22.944101 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 31 09:35:22.944101 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:35:22.944101 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:35:22.944101 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:35:22.944101 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:35:22.944101 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:35:22.944101 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:35:22.944101 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:35:22.944101 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application with Lock( Aug 31 09:35:22.944101 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:35:22.944101 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:35:22.944101 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:35:22.944101 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:35:22.944101 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:35:22.944101 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:35:22.944101 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:35:22.944101 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:35:22.945020 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 31 09:35:22.945020 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 31 09:35:22.945020 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 31 09:35:22.945020 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 31 09:35:22.945020 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 31 09:35:22.945020 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 31 09:35:22.945020 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 09:35:22.945020 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application Aug 31 09:35:22.947832 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 484/970] 10.222.0.13 () {56 vars in 1111 bytes} [Wed Aug 31 09:35:22 2022] GET /identity/v3/projects/service => generated 88 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:35:22.959806 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24820d50-5dd6-460f-92fb-c144257f1ce6 None admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:35:22.982752 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e27fa160-234d-4e6a-82ff-791232699575 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:35:22.983942 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27fa160-234d-4e6a-82ff-791232699575 None admin] REQUEST_METHOD: `POST` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:22.984423 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27fa160-234d-4e6a-82ff-791232699575 None admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:22.984845 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27fa160-234d-4e6a-82ff-791232699575 None admin] PATH_INFO: `/v3/users` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:22.985883 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e27fa160-234d-4e6a-82ff-791232699575 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:35:22.987897 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e27fa160-234d-4e6a-82ff-791232699575 None admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:35:23.004291 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.sql.core [None req-e27fa160-234d-4e6a-82ff-791232699575 None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Aug 31 09:35:23.004291 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Aug 31 09:35:23.004291 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [parameters: {'user_id': '4f7da46cb8ff44de9131992a117ad659', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Aug 31 09:35:23.004291 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=88113) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Aug 31 09:35:23.014012 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application [None req-e27fa160-234d-4e6a-82ff-791232699575 None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Aug 31 09:35:23.014012 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:35:23.014012 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 31 09:35:23.014012 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 31 09:35:23.014012 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 31 09:35:23.014012 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 31 09:35:23.014012 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 31 09:35:23.014012 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application result = self._query(query) Aug 31 09:35:23.014012 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 31 09:35:23.014012 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application conn.query(q) Aug 31 09:35:23.014012 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 31 09:35:23.014012 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 31 09:35:23.014012 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 31 09:35:23.014012 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application result.read() Aug 31 09:35:23.014012 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 31 09:35:23.014998 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 31 09:35:23.014998 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 31 09:35:23.014998 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 31 09:35:23.014998 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 31 09:35:23.014998 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 31 09:35:23.014998 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 31 09:35:23.014998 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 31 09:35:23.014998 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Aug 31 09:35:23.014998 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application Aug 31 09:35:23.014998 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Aug 31 09:35:23.014998 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application Aug 31 09:35:23.014998 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:35:23.014998 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Aug 31 09:35:23.014998 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return method(*args, **kwargs) Aug 31 09:35:23.014998 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Aug 31 09:35:23.014998 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Aug 31 09:35:23.014998 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 31 09:35:23.014998 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application next(self.gen) Aug 31 09:35:23.015949 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Aug 31 09:35:23.015949 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application yield resource Aug 31 09:35:23.015949 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 31 09:35:23.015949 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application next(self.gen) Aug 31 09:35:23.015949 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Aug 31 09:35:23.015949 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application self.session.rollback() Aug 31 09:35:23.015949 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Aug 31 09:35:23.015949 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application self.force_reraise() Aug 31 09:35:23.015949 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Aug 31 09:35:23.015949 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application raise self.value Aug 31 09:35:23.015949 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Aug 31 09:35:23.015949 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Aug 31 09:35:23.015949 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Aug 31 09:35:23.015949 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application session.commit() Aug 31 09:35:23.015949 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Aug 31 09:35:23.015949 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Aug 31 09:35:23.015949 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Aug 31 09:35:23.017592 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application self._prepare_impl() Aug 31 09:35:23.017592 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Aug 31 09:35:23.017592 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application self.session.flush() Aug 31 09:35:23.017592 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Aug 31 09:35:23.017592 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application self._flush(objects) Aug 31 09:35:23.017592 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Aug 31 09:35:23.017592 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Aug 31 09:35:23.017592 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Aug 31 09:35:23.017592 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application compat.raise_( Aug 31 09:35:23.017592 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 31 09:35:23.017592 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application raise exception Aug 31 09:35:23.017592 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Aug 31 09:35:23.017592 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application flush_context.execute() Aug 31 09:35:23.017592 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Aug 31 09:35:23.017592 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application rec.execute(self) Aug 31 09:35:23.017592 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Aug 31 09:35:23.017592 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Aug 31 09:35:23.018806 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Aug 31 09:35:23.018806 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application _emit_insert_statements( Aug 31 09:35:23.018806 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Aug 31 09:35:23.018806 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application result = connection._execute_20( Aug 31 09:35:23.018806 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Aug 31 09:35:23.018806 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Aug 31 09:35:23.018806 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Aug 31 09:35:23.018806 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Aug 31 09:35:23.018806 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Aug 31 09:35:23.018806 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application ret = self._execute_context( Aug 31 09:35:23.018806 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Aug 31 09:35:23.018806 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Aug 31 09:35:23.018806 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Aug 31 09:35:23.018806 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Aug 31 09:35:23.018806 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 31 09:35:23.018806 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application raise exception Aug 31 09:35:23.019652 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 31 09:35:23.019652 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 31 09:35:23.019652 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 31 09:35:23.019652 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 31 09:35:23.019652 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 31 09:35:23.019652 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application result = self._query(query) Aug 31 09:35:23.019652 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 31 09:35:23.019652 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application conn.query(q) Aug 31 09:35:23.019652 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 31 09:35:23.019652 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 31 09:35:23.019652 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 31 09:35:23.019652 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application result.read() Aug 31 09:35:23.019652 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 31 09:35:23.019652 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 31 09:35:23.019652 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 31 09:35:23.019652 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 31 09:35:23.023355 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 31 09:35:23.023355 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 31 09:35:23.023355 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 31 09:35:23.023355 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 31 09:35:23.023355 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Aug 31 09:35:23.023355 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Aug 31 09:35:23.023355 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application [parameters: {'user_id': '4f7da46cb8ff44de9131992a117ad659', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Aug 31 09:35:23.023355 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Aug 31 09:35:23.023355 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application Aug 31 09:35:23.023355 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 31 09:35:23.023355 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application Aug 31 09:35:23.023355 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:35:23.023355 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:35:23.023355 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:35:23.023355 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:35:23.027204 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:35:23.027204 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:35:23.027204 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:35:23.027204 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:35:23.027204 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:35:23.027204 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:35:23.027204 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:35:23.027204 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Aug 31 09:35:23.027204 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Aug 31 09:35:23.027204 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:35:23.027204 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:35:23.027204 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 31 09:35:23.027204 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 09:35:23.027204 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 31 09:35:23.027204 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 09:35:23.027204 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Aug 31 09:35:23.027204 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Aug 31 09:35:23.030501 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Aug 31 09:35:23.030501 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Aug 31 09:35:23.030501 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Aug 31 09:35:23.030501 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Aug 31 09:35:23.030501 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Aug 31 09:35:23.030501 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application Aug 31 09:35:23.030501 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 487/971] 10.222.0.13 () {60 vars in 1131 bytes} [Wed Aug 31 09:35:22 2022] POST /identity/v3/users => generated 160 bytes in 279 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Aug 31 09:35:23.030501 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e27fa160-234d-4e6a-82ff-791232699575 None admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:35:23.034305 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:35:23.127499 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24820d50-5dd6-460f-92fb-c144257f1ce6 None admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:35:23.171002 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=100 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:35:23.179340 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f07f4b-5163-4dd1-b758-7dd661278d74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:35:23.180225 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f07f4b-5163-4dd1-b758-7dd661278d74 None admin] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:23.180917 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f07f4b-5163-4dd1-b758-7dd661278d74 None admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:23.181191 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f07f4b-5163-4dd1-b758-7dd661278d74 None admin] PATH_INFO: `/v3/projects` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:23.182062 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5f07f4b-5163-4dd1-b758-7dd661278d74 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88114) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:35:23.182367 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5f07f4b-5163-4dd1-b758-7dd661278d74 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:35:23.183810 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5f07f4b-5163-4dd1-b758-7dd661278d74 None admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:35:23.184292 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e27fa160-234d-4e6a-82ff-791232699575 None admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:35:23.205114 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 485/972] 10.222.0.13 () {56 vars in 1148 bytes} [Wed Aug 31 09:35:22 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 09:35:23.205979 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f07f4b-5163-4dd1-b758-7dd661278d74 None admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:35:23.245723 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d891f305-7a6e-46ba-a1fa-f93445e22930 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:35:23.246478 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d891f305-7a6e-46ba-a1fa-f93445e22930 None admin] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:23.246894 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d891f305-7a6e-46ba-a1fa-f93445e22930 None admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:23.248056 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d891f305-7a6e-46ba-a1fa-f93445e22930 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:23.248959 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d891f305-7a6e-46ba-a1fa-f93445e22930 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:35:23.250648 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d891f305-7a6e-46ba-a1fa-f93445e22930 None admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:35:23.260913 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application [None req-d891f305-7a6e-46ba-a1fa-f93445e22930 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 31 09:35:23.260913 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:35:23.260913 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 31 09:35:23.260913 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 31 09:35:23.260913 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 31 09:35:23.260913 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 31 09:35:23.260913 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 31 09:35:23.260913 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 09:35:23.260913 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 31 09:35:23.260913 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application Aug 31 09:35:23.260913 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 31 09:35:23.260913 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application Aug 31 09:35:23.260913 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:35:23.260913 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:35:23.260913 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:35:23.260913 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:35:23.260913 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:35:23.263010 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:35:23.263010 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:35:23.263010 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:35:23.263010 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:35:23.263010 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:35:23.263010 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:35:23.263010 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 31 09:35:23.263010 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 31 09:35:23.263010 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 31 09:35:23.263010 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 31 09:35:23.263010 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:35:23.263010 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:35:23.263010 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:35:23.263010 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:35:23.263010 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:35:23.263010 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:35:23.264874 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:35:23.264874 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application with Lock( Aug 31 09:35:23.264874 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:35:23.264874 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:35:23.264874 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:35:23.264874 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:35:23.264874 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:35:23.264874 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:35:23.264874 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:35:23.264874 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:35:23.264874 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 31 09:35:23.264874 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 09:35:23.264874 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 31 09:35:23.264874 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application Aug 31 09:35:23.268030 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 488/973] 10.222.0.13 () {56 vars in 1126 bytes} [Wed Aug 31 09:35:23 2022] GET /identity/v3/services/object-store => generated 93 bytes in 247 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:35:23.269293 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d891f305-7a6e-46ba-a1fa-f93445e22930 None admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:35:23.353075 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f07f4b-5163-4dd1-b758-7dd661278d74 None admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:35:23.370100 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=100 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:35:23.410378 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c8e4c32-8035-46f0-b384-5600ec1d04ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:35:23.411340 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8e4c32-8035-46f0-b384-5600ec1d04ce None admin] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:23.411772 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8e4c32-8035-46f0-b384-5600ec1d04ce None admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:23.412194 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8e4c32-8035-46f0-b384-5600ec1d04ce None admin] PATH_INFO: `/v3/users/cinder` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:23.421592 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c8e4c32-8035-46f0-b384-5600ec1d04ce None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:35:23.423348 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c8e4c32-8035-46f0-b384-5600ec1d04ce None admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:35:23.429225 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d891f305-7a6e-46ba-a1fa-f93445e22930 None admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:35:23.437818 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application [None req-1c8e4c32-8035-46f0-b384-5600ec1d04ce None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 31 09:35:23.437818 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:35:23.437818 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:35:23.437818 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:35:23.437818 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:35:23.437818 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:35:23.437818 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:35:23.437818 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:35:23.437818 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:35:23.437818 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:35:23.437818 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:35:23.437818 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:35:23.437818 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 31 09:35:23.437818 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 31 09:35:23.437818 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 31 09:35:23.437818 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 31 09:35:23.438901 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:35:23.438901 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:35:23.438901 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 31 09:35:23.438901 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 09:35:23.438901 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 31 09:35:23.438901 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 09:35:23.438901 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:35:23.438901 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:35:23.438901 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:35:23.438901 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:35:23.438901 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:35:23.438901 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application with Lock( Aug 31 09:35:23.438901 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:35:23.438901 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:35:23.438901 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:35:23.438901 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:35:23.438901 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:35:23.438901 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:35:23.439847 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:35:23.439847 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:35:23.439847 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 31 09:35:23.439847 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 31 09:35:23.439847 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 31 09:35:23.439847 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 31 09:35:23.439847 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 31 09:35:23.439847 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 31 09:35:23.439847 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 31 09:35:23.439847 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application Aug 31 09:35:23.446470 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 486/974] 10.222.0.13 () {56 vars in 1099 bytes} [Wed Aug 31 09:35:23 2022] GET /identity/v3/users/cinder => generated 84 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:35:23.446470 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c8e4c32-8035-46f0-b384-5600ec1d04ce None admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:35:23.476029 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21fdb6dc-4961-4725-96bc-c372c8b7e81e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:35:23.476528 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21fdb6dc-4961-4725-96bc-c372c8b7e81e None admin] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:23.476805 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21fdb6dc-4961-4725-96bc-c372c8b7e81e None admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:23.477076 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21fdb6dc-4961-4725-96bc-c372c8b7e81e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:23.477849 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21fdb6dc-4961-4725-96bc-c372c8b7e81e None admin] RBAC: Adding query filter params (role.id=d41c162f8e1748989a09382ced5ef3a6, scope.project.id=49548eb44d62415582af820ab66decee, user.id=768cb67c1ef24251aaa113c12d81a250) {{(pid=88113) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:35:23.478239 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21fdb6dc-4961-4725-96bc-c372c8b7e81e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:35:23.479615 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21fdb6dc-4961-4725-96bc-c372c8b7e81e None admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:35:23.497924 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 489/975] 10.222.0.13 () {56 vars in 1374 bytes} [Wed Aug 31 09:35:23 2022] GET /identity/v3/role_assignments?user.id=768cb67c1ef24251aaa113c12d81a250&scope.project.id=49548eb44d62415582af820ab66decee&role.id=d41c162f8e1748989a09382ced5ef3a6 => generated 608 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:35:23.497924 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21fdb6dc-4961-4725-96bc-c372c8b7e81e None admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:35:23.554390 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=100 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:35:23.638662 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c8e4c32-8035-46f0-b384-5600ec1d04ce None admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:35:23.672577 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21fdb6dc-4961-4725-96bc-c372c8b7e81e None admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:35:23.699000 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af69e93-664a-4ee6-8023-7cf8e721e396 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:35:23.699835 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af69e93-664a-4ee6-8023-7cf8e721e396 None admin] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:23.700002 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af69e93-664a-4ee6-8023-7cf8e721e396 None admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:23.700136 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af69e93-664a-4ee6-8023-7cf8e721e396 None admin] PATH_INFO: `/v3/services` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:23.703238 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8af69e93-664a-4ee6-8023-7cf8e721e396 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=88114) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:35:23.703238 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8af69e93-664a-4ee6-8023-7cf8e721e396 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:35:23.703563 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8af69e93-664a-4ee6-8023-7cf8e721e396 None admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:35:23.709681 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 487/976] 10.222.0.13 () {56 vars in 1122 bytes} [Wed Aug 31 09:35:23 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:35:23.710957 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d27ed8-c4f9-482c-9ae8-3791f79856b8 None None] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:23.711229 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d27ed8-c4f9-482c-9ae8-3791f79856b8 None None] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:23.711498 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d27ed8-c4f9-482c-9ae8-3791f79856b8 None None] PATH_INFO: `/` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:23.713588 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 488/977] 10.222.0.13 () {54 vars in 927 bytes} [Wed Aug 31 09:35:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:35:23.718783 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d27ed8-c4f9-482c-9ae8-3791f79856b8 None None] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:35:23.719061 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=100 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:35:23.726853 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8863bb05-d329-4944-9464-a9ad9fd442c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:35:23.728459 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8863bb05-d329-4944-9464-a9ad9fd442c6 None admin] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:23.728459 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8863bb05-d329-4944-9464-a9ad9fd442c6 None admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:23.728750 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8863bb05-d329-4944-9464-a9ad9fd442c6 None admin] PATH_INFO: `/v3/users` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:23.729655 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8863bb05-d329-4944-9464-a9ad9fd442c6 None admin] RBAC: Adding query filter params (name=cinder) {{(pid=88113) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:35:23.729655 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8863bb05-d329-4944-9464-a9ad9fd442c6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:35:23.731340 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8863bb05-d329-4944-9464-a9ad9fd442c6 None admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:35:23.774383 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 490/978] 10.222.0.13 () {56 vars in 1101 bytes} [Wed Aug 31 09:35:23 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 284 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:35:23.775849 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46462389-3620-4bd2-95e9-4a48e23bc9b7 None None] REQUEST_METHOD: `POST` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:23.776140 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46462389-3620-4bd2-95e9-4a48e23bc9b7 None None] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:23.776412 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46462389-3620-4bd2-95e9-4a48e23bc9b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:23.804115 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.auth.core [None req-46462389-3620-4bd2-95e9-4a48e23bc9b7 None None] MFA Rules not processed for user `fd836cf590a145df9acf19cf3ae2ac56`. Rule list: `[]` (Enabled: `True`). {{(pid=88113) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:35:23.808544 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.fernet_utils [None req-46462389-3620-4bd2-95e9-4a48e23bc9b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:35:23.928121 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d27ed8-c4f9-482c-9ae8-3791f79856b8 None None] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:35:23.958424 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=100 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:35:23.982586 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 491/979] 10.222.0.13 () {58 vars in 1024 bytes} [Wed Aug 31 09:35:23 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 09:35:24.014203 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6186b94-ce67-42c6-aa9d-890b7bc84c3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:35:24.014815 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6186b94-ce67-42c6-aa9d-890b7bc84c3e None admin] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:24.015303 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6186b94-ce67-42c6-aa9d-890b7bc84c3e None admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:24.015789 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6186b94-ce67-42c6-aa9d-890b7bc84c3e None admin] PATH_INFO: `/v3/services` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:24.016951 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6186b94-ce67-42c6-aa9d-890b7bc84c3e None admin] RBAC: Adding query filter params (type=object-store) {{(pid=88114) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:35:24.017530 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6186b94-ce67-42c6-aa9d-890b7bc84c3e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:35:24.020497 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6186b94-ce67-42c6-aa9d-890b7bc84c3e None admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:35:24.031568 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 489/980] 10.222.0.13 () {56 vars in 1122 bytes} [Wed Aug 31 09:35:23 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 313 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:35:24.037116 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46462389-3620-4bd2-95e9-4a48e23bc9b7 None None] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:35:24.226182 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46462389-3620-4bd2-95e9-4a48e23bc9b7 None None] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:35:24.258241 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=100 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:35:24.290251 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb0d56f-9498-4333-a00c-c007b3cf2c8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:35:24.291278 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb0d56f-9498-4333-a00c-c007b3cf2c8a None admin] REQUEST_METHOD: `POST` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:24.291759 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb0d56f-9498-4333-a00c-c007b3cf2c8a None admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:24.292236 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb0d56f-9498-4333-a00c-c007b3cf2c8a None admin] PATH_INFO: `/v3/endpoints` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:24.293498 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecb0d56f-9498-4333-a00c-c007b3cf2c8a None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:35:24.296723 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecb0d56f-9498-4333-a00c-c007b3cf2c8a None admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:35:24.337418 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 492/981] 10.222.0.13 () {60 vars in 1144 bytes} [Wed Aug 31 09:35:24 2022] POST /identity/v3/endpoints => generated 349 bytes in 295 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 09:35:24.428466 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6186b94-ce67-42c6-aa9d-890b7bc84c3e None admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:35:24.620298 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6186b94-ce67-42c6-aa9d-890b7bc84c3e None admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:35:24.695106 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f06f7418-d934-4779-adf5-7161173bcc2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:35:24.695744 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06f7418-d934-4779-adf5-7161173bcc2c None admin] REQUEST_METHOD: `POST` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:24.696014 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06f7418-d934-4779-adf5-7161173bcc2c None admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:24.696463 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06f7418-d934-4779-adf5-7161173bcc2c None admin] PATH_INFO: `/v3/services` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:24.697667 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f06f7418-d934-4779-adf5-7161173bcc2c None admin] RBAC: Authorizing `identity:create_service()` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:35:24.700181 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f06f7418-d934-4779-adf5-7161173bcc2c None admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:35:24.725949 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 490/982] 10.222.0.13 () {60 vars in 1141 bytes} [Wed Aug 31 09:35:24 2022] POST /identity/v3/services => generated 242 bytes in 298 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 09:35:25.791681 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693579a7-4073-45f5-b111-3dc9dc4a1975 None None] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:25.792263 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693579a7-4073-45f5-b111-3dc9dc4a1975 None None] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:25.792627 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693579a7-4073-45f5-b111-3dc9dc4a1975 None None] PATH_INFO: `/` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:25.795771 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 493/983] 10.222.0.13 () {54 vars in 927 bytes} [Wed Aug 31 09:35:25 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:35:25.801719 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea1a2e6-a494-4a84-9669-1881dade00a7 None None] REQUEST_METHOD: `POST` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:25.802523 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea1a2e6-a494-4a84-9669-1881dade00a7 None None] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:25.802523 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea1a2e6-a494-4a84-9669-1881dade00a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:25.824502 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.auth.core [None req-2ea1a2e6-a494-4a84-9669-1881dade00a7 None None] MFA Rules not processed for user `fd836cf590a145df9acf19cf3ae2ac56`. Rule list: `[]` (Enabled: `True`). {{(pid=88114) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:35:25.826429 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.fernet_utils [None req-2ea1a2e6-a494-4a84-9669-1881dade00a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88114) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:35:25.883811 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:35:25.966226 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 491/984] 10.222.0.13 () {58 vars in 1024 bytes} [Wed Aug 31 09:35:25 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 31 09:35:26.257755 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-693579a7-4073-45f5-b111-3dc9dc4a1975 None None] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:35:26.374663 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e685c50-7981-48f6-bb98-5b44d59ee2c4 None None] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:26.374958 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e685c50-7981-48f6-bb98-5b44d59ee2c4 None None] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:26.375238 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e685c50-7981-48f6-bb98-5b44d59ee2c4 None None] PATH_INFO: `/` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:26.376600 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 492/985] 10.222.0.13 () {54 vars in 927 bytes} [Wed Aug 31 09:35:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 31 09:35:26.382446 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee7978e-f870-47a5-97d1-3105fcad7f6a None None] REQUEST_METHOD: `POST` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:26.382735 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee7978e-f870-47a5-97d1-3105fcad7f6a None None] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:26.383013 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee7978e-f870-47a5-97d1-3105fcad7f6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:26.403152 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.auth.core [None req-eee7978e-f870-47a5-97d1-3105fcad7f6a None None] MFA Rules not processed for user `fd836cf590a145df9acf19cf3ae2ac56`. Rule list: `[]` (Enabled: `True`). {{(pid=88114) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:35:26.404917 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.fernet_utils [None req-eee7978e-f870-47a5-97d1-3105fcad7f6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88114) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:35:26.454501 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-693579a7-4073-45f5-b111-3dc9dc4a1975 None None] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:35:26.494289 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 493/986] 10.222.0.13 () {58 vars in 1024 bytes} [Wed Aug 31 09:35:26 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 09:35:26.521074 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad1b438c-90e7-4e8d-a9c9-b365110a42c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:35:26.522103 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1b438c-90e7-4e8d-a9c9-b365110a42c0 None admin] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:26.522594 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1b438c-90e7-4e8d-a9c9-b365110a42c0 None admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:26.523079 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1b438c-90e7-4e8d-a9c9-b365110a42c0 None admin] PATH_INFO: `/v3/services/image` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:26.524148 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad1b438c-90e7-4e8d-a9c9-b365110a42c0 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:35:26.526848 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad1b438c-90e7-4e8d-a9c9-b365110a42c0 None admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:35:26.545614 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application [None req-ad1b438c-90e7-4e8d-a9c9-b365110a42c0 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Aug 31 09:35:26.545614 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:35:26.545614 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 31 09:35:26.545614 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 31 09:35:26.545614 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 31 09:35:26.545614 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 31 09:35:26.545614 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 31 09:35:26.545614 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 09:35:26.545614 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 31 09:35:26.545614 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application Aug 31 09:35:26.545614 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 31 09:35:26.545614 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application Aug 31 09:35:26.545614 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:35:26.545614 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:35:26.545614 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:35:26.545614 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:35:26.545614 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:35:26.547654 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:35:26.547654 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:35:26.547654 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:35:26.547654 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:35:26.547654 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:35:26.547654 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:35:26.547654 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 31 09:35:26.547654 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 31 09:35:26.547654 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 31 09:35:26.547654 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 31 09:35:26.547654 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:35:26.547654 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:35:26.547654 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:35:26.547654 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:35:26.547654 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:35:26.547654 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:35:26.549474 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:35:26.549474 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application with Lock( Aug 31 09:35:26.549474 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:35:26.549474 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:35:26.549474 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:35:26.549474 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:35:26.549474 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:35:26.549474 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:35:26.549474 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:35:26.549474 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:35:26.549474 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 31 09:35:26.549474 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 09:35:26.549474 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 31 09:35:26.549474 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application Aug 31 09:35:26.549474 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 494/987] 10.222.0.13 () {56 vars in 1105 bytes} [Wed Aug 31 09:35:26 2022] GET /identity/v3/services/image => generated 86 bytes in 291 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:35:26.554744 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee7978e-f870-47a5-97d1-3105fcad7f6a None None] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:35:26.670542 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=100 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:35:26.684248 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee7978e-f870-47a5-97d1-3105fcad7f6a None None] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:35:26.725280 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-839698ac-168a-4610-8c03-143087d38462 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:35:26.725940 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839698ac-168a-4610-8c03-143087d38462 None admin] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:26.726211 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839698ac-168a-4610-8c03-143087d38462 None admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:26.726478 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839698ac-168a-4610-8c03-143087d38462 None admin] PATH_INFO: `/v3/services` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:26.727143 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-839698ac-168a-4610-8c03-143087d38462 None admin] RBAC: Adding query filter params (name=image) {{(pid=88114) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:35:26.727438 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-839698ac-168a-4610-8c03-143087d38462 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:35:26.729062 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-839698ac-168a-4610-8c03-143087d38462 None admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:35:26.741433 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 494/988] 10.222.0.13 () {56 vars in 1108 bytes} [Wed Aug 31 09:35:26 2022] GET /identity/v3/services?name=image => generated 122 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:35:26.745193 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad1b438c-90e7-4e8d-a9c9-b365110a42c0 None admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:35:26.795120 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:35:26.877305 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad1b438c-90e7-4e8d-a9c9-b365110a42c0 None admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:35:26.916171 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce8d4f4c-d877-4a9d-9d88-6177945f1d28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:35:26.916778 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8d4f4c-d877-4a9d-9d88-6177945f1d28 None admin] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:26.917051 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8d4f4c-d877-4a9d-9d88-6177945f1d28 None admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:26.917338 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8d4f4c-d877-4a9d-9d88-6177945f1d28 None admin] PATH_INFO: `/v3/services` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:26.917962 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce8d4f4c-d877-4a9d-9d88-6177945f1d28 None admin] RBAC: Adding query filter params (type=image) {{(pid=88113) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:35:26.918257 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce8d4f4c-d877-4a9d-9d88-6177945f1d28 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:35:26.920204 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce8d4f4c-d877-4a9d-9d88-6177945f1d28 None admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:35:26.931390 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 495/989] 10.222.0.13 () {56 vars in 1108 bytes} [Wed Aug 31 09:35:26 2022] GET /identity/v3/services?type=image => generated 122 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 09:35:26.949980 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-839698ac-168a-4610-8c03-143087d38462 None admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:35:27.140583 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-839698ac-168a-4610-8c03-143087d38462 None admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:35:27.198777 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5338e46-0519-4fb2-905b-74fe1672c0d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:35:27.199810 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5338e46-0519-4fb2-905b-74fe1672c0d7 None admin] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:27.200273 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5338e46-0519-4fb2-905b-74fe1672c0d7 None admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:27.201557 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5338e46-0519-4fb2-905b-74fe1672c0d7 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:27.202339 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5338e46-0519-4fb2-905b-74fe1672c0d7 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:35:27.204828 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5338e46-0519-4fb2-905b-74fe1672c0d7 None admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:35:27.221726 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application [None req-a5338e46-0519-4fb2-905b-74fe1672c0d7 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 31 09:35:27.221726 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:35:27.221726 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 31 09:35:27.221726 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 31 09:35:27.221726 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 31 09:35:27.221726 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 31 09:35:27.221726 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 31 09:35:27.221726 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 09:35:27.221726 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 31 09:35:27.221726 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application Aug 31 09:35:27.221726 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 31 09:35:27.221726 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application Aug 31 09:35:27.221726 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:35:27.221726 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:35:27.221726 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:35:27.221726 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:35:27.221726 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:35:27.223538 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:35:27.223538 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:35:27.223538 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:35:27.223538 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:35:27.223538 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:35:27.223538 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:35:27.223538 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 31 09:35:27.223538 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 31 09:35:27.223538 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 31 09:35:27.223538 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 31 09:35:27.223538 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:35:27.223538 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:35:27.223538 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:35:27.223538 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:35:27.223538 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:35:27.223538 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:35:27.225245 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:35:27.225245 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application with Lock( Aug 31 09:35:27.225245 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:35:27.225245 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:35:27.225245 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:35:27.225245 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:35:27.225245 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:35:27.225245 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:35:27.225245 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:35:27.225245 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:35:27.225245 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 31 09:35:27.225245 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 09:35:27.225245 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 31 09:35:27.225245 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application Aug 31 09:35:27.225245 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 495/990] 10.222.0.13 () {56 vars in 1126 bytes} [Wed Aug 31 09:35:26 2022] GET /identity/v3/services/object-store => generated 93 bytes in 275 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:35:27.234110 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce8d4f4c-d877-4a9d-9d88-6177945f1d28 None admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:35:27.433017 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce8d4f4c-d877-4a9d-9d88-6177945f1d28 None admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:35:27.436054 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=100 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:35:27.479177 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3834cc5e-3e3b-416a-8b1c-4f5de229b320 None None] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:27.479781 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3834cc5e-3e3b-416a-8b1c-4f5de229b320 None None] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:27.480353 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3834cc5e-3e3b-416a-8b1c-4f5de229b320 None None] PATH_INFO: `/` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:27.485831 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 496/991] 10.222.0.13 () {54 vars in 927 bytes} [Wed Aug 31 09:35:27 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:35:27.491941 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c23deb-dbbe-4a7c-b4e0-116aaed021d6 None None] REQUEST_METHOD: `POST` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:27.492452 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c23deb-dbbe-4a7c-b4e0-116aaed021d6 None None] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:27.492914 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c23deb-dbbe-4a7c-b4e0-116aaed021d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:27.499454 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4417476-c544-4391-9e24-4232280338d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:35:27.500501 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4417476-c544-4391-9e24-4232280338d8 None admin] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:27.500998 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4417476-c544-4391-9e24-4232280338d8 None admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:27.501524 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4417476-c544-4391-9e24-4232280338d8 None admin] PATH_INFO: `/v3/services` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:27.502723 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4417476-c544-4391-9e24-4232280338d8 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=88113) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:35:27.503272 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4417476-c544-4391-9e24-4232280338d8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:35:27.505895 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4417476-c544-4391-9e24-4232280338d8 None admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:35:27.523118 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 496/992] 10.222.0.13 () {56 vars in 1122 bytes} [Wed Aug 31 09:35:27 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 285 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:35:27.524816 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4417476-c544-4391-9e24-4232280338d8 None admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:35:27.535579 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.auth.core [None req-91c23deb-dbbe-4a7c-b4e0-116aaed021d6 None None] MFA Rules not processed for user `fd836cf590a145df9acf19cf3ae2ac56`. Rule list: `[]` (Enabled: `True`). {{(pid=88114) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:35:27.537716 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.fernet_utils [None req-91c23deb-dbbe-4a7c-b4e0-116aaed021d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88114) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:35:27.560546 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=100 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:35:27.649625 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 497/993] 10.222.0.13 () {58 vars in 1024 bytes} [Wed Aug 31 09:35:27 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 364 bytes (3 switches on core 0) Aug 31 09:35:27.676408 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4417476-c544-4391-9e24-4232280338d8 None admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:35:27.714766 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b9da60-b2b0-4d47-afff-d60fc4369e00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:35:27.715273 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b9da60-b2b0-4d47-afff-d60fc4369e00 None admin] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:27.715545 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b9da60-b2b0-4d47-afff-d60fc4369e00 None admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:27.715810 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b9da60-b2b0-4d47-afff-d60fc4369e00 None admin] PATH_INFO: `/v3/services` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:27.716490 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4b9da60-b2b0-4d47-afff-d60fc4369e00 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=88113) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:35:27.716786 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4b9da60-b2b0-4d47-afff-d60fc4369e00 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:35:27.718204 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4b9da60-b2b0-4d47-afff-d60fc4369e00 None admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:35:27.724536 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 497/994] 10.222.0.13 () {56 vars in 1122 bytes} [Wed Aug 31 09:35:27 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 09:35:27.730512 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c23deb-dbbe-4a7c-b4e0-116aaed021d6 None None] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:35:27.855259 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c23deb-dbbe-4a7c-b4e0-116aaed021d6 None None] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:35:27.910299 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4792d09c-a9f1-452f-9787-5eba3cdaf59d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:35:27.910920 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4792d09c-a9f1-452f-9787-5eba3cdaf59d None admin] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:27.911196 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4792d09c-a9f1-452f-9787-5eba3cdaf59d None admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:27.911474 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4792d09c-a9f1-452f-9787-5eba3cdaf59d None admin] PATH_INFO: `/v3/endpoints` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:27.912193 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4792d09c-a9f1-452f-9787-5eba3cdaf59d None admin] RBAC: Adding query filter params (interface=admin, service_id=a0f5cdb3f7914db5b814282cf8b2490d, region_id=RegionOne) {{(pid=88114) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:35:27.912495 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4792d09c-a9f1-452f-9787-5eba3cdaf59d None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:35:27.913964 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4792d09c-a9f1-452f-9787-5eba3cdaf59d None admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:35:27.925429 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 498/995] 10.222.0.13 () {56 vars in 1249 bytes} [Wed Aug 31 09:35:27 2022] GET /identity/v3/endpoints?service_id=a0f5cdb3f7914db5b814282cf8b2490d&interface=admin®ion_id=RegionOne => generated 193 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 09:35:27.930877 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b9da60-b2b0-4d47-afff-d60fc4369e00 None admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:35:27.989600 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=100 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:35:28.024659 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4792d09c-a9f1-452f-9787-5eba3cdaf59d None admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:35:28.052029 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=100 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:35:28.094668 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b9da60-b2b0-4d47-afff-d60fc4369e00 None admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:35:28.142088 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a48693-b8d1-4447-a8de-b4a039d82575 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:35:28.143387 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a48693-b8d1-4447-a8de-b4a039d82575 None admin] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:28.143876 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a48693-b8d1-4447-a8de-b4a039d82575 None admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:28.144356 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a48693-b8d1-4447-a8de-b4a039d82575 None admin] PATH_INFO: `/v3/services` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:28.145538 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40a48693-b8d1-4447-a8de-b4a039d82575 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:35:28.148135 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40a48693-b8d1-4447-a8de-b4a039d82575 None admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:35:28.159899 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 498/996] 10.222.0.13 () {56 vars in 1087 bytes} [Wed Aug 31 09:35:27 2022] GET /identity/v3/services => generated 1727 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 09:35:28.246338 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4792d09c-a9f1-452f-9787-5eba3cdaf59d None admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:35:28.296248 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b2995c0-3451-4f9e-b05c-7fe9b55b09ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:35:28.296929 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2995c0-3451-4f9e-b05c-7fe9b55b09ef None admin] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:28.297208 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2995c0-3451-4f9e-b05c-7fe9b55b09ef None admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:28.297540 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2995c0-3451-4f9e-b05c-7fe9b55b09ef None admin] PATH_INFO: `/v3/services/compute` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:28.298159 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b2995c0-3451-4f9e-b05c-7fe9b55b09ef None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:35:28.299722 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b2995c0-3451-4f9e-b05c-7fe9b55b09ef None admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:35:28.311070 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application [None req-5b2995c0-3451-4f9e-b05c-7fe9b55b09ef None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Aug 31 09:35:28.311070 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:35:28.311070 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 31 09:35:28.311070 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 31 09:35:28.311070 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 31 09:35:28.311070 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 31 09:35:28.311070 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 31 09:35:28.311070 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 09:35:28.311070 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 31 09:35:28.311070 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application Aug 31 09:35:28.311070 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 31 09:35:28.311070 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application Aug 31 09:35:28.311070 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:35:28.311070 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:35:28.311070 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:35:28.311070 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:35:28.311070 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:35:28.313062 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:35:28.313062 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:35:28.313062 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:35:28.313062 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:35:28.313062 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:35:28.313062 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:35:28.313062 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 31 09:35:28.313062 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 31 09:35:28.313062 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 31 09:35:28.313062 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 31 09:35:28.313062 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:35:28.313062 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:35:28.313062 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:35:28.313062 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:35:28.313062 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:35:28.313062 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:35:28.314722 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:35:28.314722 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application with Lock( Aug 31 09:35:28.314722 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:35:28.314722 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:35:28.314722 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:35:28.314722 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:35:28.314722 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:35:28.314722 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:35:28.314722 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:35:28.314722 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:35:28.314722 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 31 09:35:28.314722 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 09:35:28.314722 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 31 09:35:28.314722 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application Aug 31 09:35:28.314722 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 499/997] 10.222.0.13 () {56 vars in 1111 bytes} [Wed Aug 31 09:35:28 2022] GET /identity/v3/services/compute => generated 88 bytes in 289 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 09:35:28.323060 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a48693-b8d1-4447-a8de-b4a039d82575 None admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:35:28.465858 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=100 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:35:28.477036 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a48693-b8d1-4447-a8de-b4a039d82575 None admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:35:28.520177 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e166132-6de1-4a24-bf41-2a047ab203e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:35:28.520898 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e166132-6de1-4a24-bf41-2a047ab203e2 None admin] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:28.521310 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e166132-6de1-4a24-bf41-2a047ab203e2 None admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:28.521742 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e166132-6de1-4a24-bf41-2a047ab203e2 None admin] PATH_INFO: `/v3/services` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:28.522527 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e166132-6de1-4a24-bf41-2a047ab203e2 None admin] RBAC: Adding query filter params (name=compute) {{(pid=88113) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:35:28.522963 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e166132-6de1-4a24-bf41-2a047ab203e2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:35:28.524489 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e166132-6de1-4a24-bf41-2a047ab203e2 None admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:35:28.532013 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 499/998] 10.222.0.13 () {56 vars in 1112 bytes} [Wed Aug 31 09:35:28 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:35:28.540949 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b2995c0-3451-4f9e-b05c-7fe9b55b09ef None admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:35:28.681694 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=100 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:35:28.709542 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b2995c0-3451-4f9e-b05c-7fe9b55b09ef None admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:35:28.752808 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc6cad0-9168-4d3e-80fc-1177a6d7e276 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:35:28.753468 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc6cad0-9168-4d3e-80fc-1177a6d7e276 None admin] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:28.753748 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc6cad0-9168-4d3e-80fc-1177a6d7e276 None admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:28.754017 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc6cad0-9168-4d3e-80fc-1177a6d7e276 None admin] PATH_INFO: `/v3/services` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:28.754665 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bc6cad0-9168-4d3e-80fc-1177a6d7e276 None admin] RBAC: Adding query filter params (type=compute) {{(pid=88114) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:35:28.755681 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bc6cad0-9168-4d3e-80fc-1177a6d7e276 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:35:28.757173 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bc6cad0-9168-4d3e-80fc-1177a6d7e276 None admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:35:28.769424 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 500/999] 10.222.0.13 () {56 vars in 1112 bytes} [Wed Aug 31 09:35:28 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:35:28.774858 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e166132-6de1-4a24-bf41-2a047ab203e2 None admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:35:28.922213 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e166132-6de1-4a24-bf41-2a047ab203e2 None admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:35:28.976547 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ded642e0-8726-4afe-82ca-5e80dbd8074e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:35:28.977418 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded642e0-8726-4afe-82ca-5e80dbd8074e None admin] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:28.977660 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded642e0-8726-4afe-82ca-5e80dbd8074e None admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:28.977900 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded642e0-8726-4afe-82ca-5e80dbd8074e None admin] PATH_INFO: `/v3/endpoints` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:28.978629 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ded642e0-8726-4afe-82ca-5e80dbd8074e None admin] RBAC: Adding query filter params (interface=public, service_id=314432c904d24f19814ad284f1b059b5, region_id=RegionOne) {{(pid=88113) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:35:28.978935 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ded642e0-8726-4afe-82ca-5e80dbd8074e None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:35:28.980354 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ded642e0-8726-4afe-82ca-5e80dbd8074e None admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:35:28.987230 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 500/1000] 10.222.0.13 () {56 vars in 1251 bytes} [Wed Aug 31 09:35:28 2022] GET /identity/v3/endpoints?service_id=314432c904d24f19814ad284f1b059b5&interface=public®ion_id=RegionOne => generated 194 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:35:28.992748 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc6cad0-9168-4d3e-80fc-1177a6d7e276 None admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:35:29.122264 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc6cad0-9168-4d3e-80fc-1177a6d7e276 None admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:35:29.163757 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b9d9a79-b8c5-4c05-90ef-eb74512d1605 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:35:29.164433 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9d9a79-b8c5-4c05-90ef-eb74512d1605 None admin] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:29.164713 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9d9a79-b8c5-4c05-90ef-eb74512d1605 None admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:29.164985 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9d9a79-b8c5-4c05-90ef-eb74512d1605 None admin] PATH_INFO: `/v3/services` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:29.165618 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b9d9a79-b8c5-4c05-90ef-eb74512d1605 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:35:29.166991 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b9d9a79-b8c5-4c05-90ef-eb74512d1605 None admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:35:29.173853 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 501/1001] 10.222.0.13 () {56 vars in 1087 bytes} [Wed Aug 31 09:35:28 2022] GET /identity/v3/services => generated 1727 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 09:35:29.456731 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a198dce-8eec-44bc-bdef-e989a667eee8 None None] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:29.457012 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a198dce-8eec-44bc-bdef-e989a667eee8 None None] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:29.457265 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a198dce-8eec-44bc-bdef-e989a667eee8 None None] PATH_INFO: `/` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:29.458959 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 501/1002] 10.222.0.13 () {54 vars in 927 bytes} [Wed Aug 31 09:35:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:35:29.464663 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d419e7c-b720-4183-ba9a-710ed889ccd8 None None] REQUEST_METHOD: `POST` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:29.465002 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d419e7c-b720-4183-ba9a-710ed889ccd8 None None] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:29.465202 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d419e7c-b720-4183-ba9a-710ed889ccd8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:29.473299 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=100 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:35:29.485852 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.auth.core [None req-5d419e7c-b720-4183-ba9a-710ed889ccd8 None None] MFA Rules not processed for user `fd836cf590a145df9acf19cf3ae2ac56`. Rule list: `[]` (Enabled: `True`). {{(pid=88114) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:35:29.487052 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.fernet_utils [None req-5d419e7c-b720-4183-ba9a-710ed889ccd8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88114) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:35:29.579578 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 502/1003] 10.222.0.13 () {58 vars in 1024 bytes} [Wed Aug 31 09:35:29 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 31 09:35:29.851816 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a198dce-8eec-44bc-bdef-e989a667eee8 None None] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:35:29.897179 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=100 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:35:30.048582 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a198dce-8eec-44bc-bdef-e989a667eee8 None None] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:35:30.114600 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-074184fb-5154-4734-a253-ce08d3576a3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:35:30.115528 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074184fb-5154-4734-a253-ce08d3576a3c None admin] REQUEST_METHOD: `POST` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:30.116001 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074184fb-5154-4734-a253-ce08d3576a3c None admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:30.116469 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074184fb-5154-4734-a253-ce08d3576a3c None admin] PATH_INFO: `/v3/services` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:30.117650 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-074184fb-5154-4734-a253-ce08d3576a3c None admin] RBAC: Authorizing `identity:create_service()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:35:30.120229 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-074184fb-5154-4734-a253-ce08d3576a3c None admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:35:30.150292 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 502/1004] 10.222.0.13 () {60 vars in 1141 bytes} [Wed Aug 31 09:35:29 2022] POST /identity/v3/services => generated 242 bytes in 299 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 09:35:30.332332 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4c8b27-a876-499d-adb2-539e00c3f059 None None] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:30.332963 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4c8b27-a876-499d-adb2-539e00c3f059 None None] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:30.332963 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4c8b27-a876-499d-adb2-539e00c3f059 None None] PATH_INFO: `/` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:30.334577 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 503/1005] 10.222.0.13 () {54 vars in 927 bytes} [Wed Aug 31 09:35:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:35:30.340299 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec24b3d-944a-4393-bcc5-fa2a23957ee9 None None] REQUEST_METHOD: `POST` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:30.340706 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec24b3d-944a-4393-bcc5-fa2a23957ee9 None None] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:30.341684 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec24b3d-944a-4393-bcc5-fa2a23957ee9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:30.361374 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.auth.core [None req-fec24b3d-944a-4393-bcc5-fa2a23957ee9 None None] MFA Rules not processed for user `fd836cf590a145df9acf19cf3ae2ac56`. Rule list: `[]` (Enabled: `True`). {{(pid=88113) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:35:30.362746 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.fernet_utils [None req-fec24b3d-944a-4393-bcc5-fa2a23957ee9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:35:30.466793 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 503/1006] 10.222.0.13 () {58 vars in 1024 bytes} [Wed Aug 31 09:35:30 2022] POST /identity/v3/auth/tokens => generated 1666 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 09:35:30.742226 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e4c8b27-a876-499d-adb2-539e00c3f059 None None] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:35:30.892888 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e4c8b27-a876-499d-adb2-539e00c3f059 None None] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:35:30.955228 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab45024f-42df-4bb9-9747-d0dc74ebf97d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:35:30.956332 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab45024f-42df-4bb9-9747-d0dc74ebf97d None admin] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:30.956825 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab45024f-42df-4bb9-9747-d0dc74ebf97d None admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:30.957343 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab45024f-42df-4bb9-9747-d0dc74ebf97d None admin] PATH_INFO: `/v3/services/object-store` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:30.958329 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab45024f-42df-4bb9-9747-d0dc74ebf97d None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:35:30.960957 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab45024f-42df-4bb9-9747-d0dc74ebf97d None admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:35:30.964325 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=100 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:35:30.980153 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application [None req-ab45024f-42df-4bb9-9747-d0dc74ebf97d None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 31 09:35:30.980153 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:35:30.980153 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 31 09:35:30.980153 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 31 09:35:30.980153 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 31 09:35:30.980153 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 31 09:35:30.980153 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 31 09:35:30.980153 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 09:35:30.980153 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 31 09:35:30.980153 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application Aug 31 09:35:30.980153 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 31 09:35:30.980153 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application Aug 31 09:35:30.980153 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:35:30.980153 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:35:30.980153 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:35:30.980153 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:35:30.980153 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:35:30.981481 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:35:30.981481 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:35:30.981481 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:35:30.981481 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:35:30.981481 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:35:30.981481 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:35:30.981481 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 31 09:35:30.981481 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 31 09:35:30.981481 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 31 09:35:30.981481 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 31 09:35:30.981481 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:35:30.981481 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:35:30.981481 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:35:30.981481 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:35:30.981481 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:35:30.981481 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:35:30.984078 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:35:30.984078 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application with Lock( Aug 31 09:35:30.984078 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:35:30.984078 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:35:30.984078 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:35:30.984078 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:35:30.984078 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:35:30.984078 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:35:30.984078 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:35:30.984078 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:35:30.984078 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 31 09:35:30.984078 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 09:35:30.984078 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 31 09:35:30.984078 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application Aug 31 09:35:30.993496 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 504/1007] 10.222.0.13 () {56 vars in 1126 bytes} [Wed Aug 31 09:35:30 2022] GET /identity/v3/services/object-store => generated 93 bytes in 248 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 09:35:30.995965 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fec24b3d-944a-4393-bcc5-fa2a23957ee9 None None] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:35:31.015276 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:35:31.182976 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fec24b3d-944a-4393-bcc5-fa2a23957ee9 None None] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:35:31.237580 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce1541a7-1975-4a91-b0d6-70cc7b841f1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:35:31.238222 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1541a7-1975-4a91-b0d6-70cc7b841f1a None admin] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:31.238515 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1541a7-1975-4a91-b0d6-70cc7b841f1a None admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:31.238768 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1541a7-1975-4a91-b0d6-70cc7b841f1a None admin] PATH_INFO: `/v3/services` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:31.239424 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce1541a7-1975-4a91-b0d6-70cc7b841f1a None admin] RBAC: Adding query filter params (name=object-store) {{(pid=88113) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:35:31.239729 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce1541a7-1975-4a91-b0d6-70cc7b841f1a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:35:31.241114 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce1541a7-1975-4a91-b0d6-70cc7b841f1a None admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:35:31.249183 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 504/1008] 10.222.0.13 () {56 vars in 1122 bytes} [Wed Aug 31 09:35:30 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:35:31.253506 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab45024f-42df-4bb9-9747-d0dc74ebf97d None admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:35:31.437659 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab45024f-42df-4bb9-9747-d0dc74ebf97d None admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:35:31.439127 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c55ffe-4548-4005-badf-933f2533061c None None] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:31.439422 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c55ffe-4548-4005-badf-933f2533061c None None] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:31.439718 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c55ffe-4548-4005-badf-933f2533061c None None] PATH_INFO: `/` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:31.441063 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 505/1009] 10.222.0.13 () {54 vars in 927 bytes} [Wed Aug 31 09:35:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 31 09:35:31.446391 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504b4eb0-2d59-4ebd-9741-69f467e481fd None None] REQUEST_METHOD: `POST` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:31.446679 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504b4eb0-2d59-4ebd-9741-69f467e481fd None None] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:31.446937 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504b4eb0-2d59-4ebd-9741-69f467e481fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:31.469529 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.auth.core [None req-504b4eb0-2d59-4ebd-9741-69f467e481fd None None] MFA Rules not processed for user `fd836cf590a145df9acf19cf3ae2ac56`. Rule list: `[]` (Enabled: `True`). {{(pid=88113) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:35:31.470470 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.fernet_utils [None req-504b4eb0-2d59-4ebd-9741-69f467e481fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:35:31.502619 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61aa3796-14fc-485d-b7cb-da5beea8db22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:35:31.503552 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61aa3796-14fc-485d-b7cb-da5beea8db22 None admin] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:31.504051 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61aa3796-14fc-485d-b7cb-da5beea8db22 None admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:31.504340 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61aa3796-14fc-485d-b7cb-da5beea8db22 None admin] PATH_INFO: `/v3/services` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:31.505539 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61aa3796-14fc-485d-b7cb-da5beea8db22 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=88114) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:35:31.506099 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61aa3796-14fc-485d-b7cb-da5beea8db22 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:35:31.508513 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61aa3796-14fc-485d-b7cb-da5beea8db22 None admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:35:31.526205 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 505/1010] 10.222.0.13 () {56 vars in 1122 bytes} [Wed Aug 31 09:35:31 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:35:31.529645 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61aa3796-14fc-485d-b7cb-da5beea8db22 None admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:35:31.570541 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 506/1011] 10.222.0.13 () {58 vars in 1024 bytes} [Wed Aug 31 09:35:31 2022] POST /identity/v3/auth/tokens => generated 1666 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 31 09:35:31.634608 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=100 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:35:31.707258 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61aa3796-14fc-485d-b7cb-da5beea8db22 None admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:35:31.768537 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b47545-70d8-444b-8891-145d71310798 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:35:31.769502 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b47545-70d8-444b-8891-145d71310798 None admin] REQUEST_METHOD: `POST` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:31.769991 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b47545-70d8-444b-8891-145d71310798 None admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:31.770459 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b47545-70d8-444b-8891-145d71310798 None admin] PATH_INFO: `/v3/endpoints` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:31.771796 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1b47545-70d8-444b-8891-145d71310798 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:35:31.774378 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1b47545-70d8-444b-8891-145d71310798 None admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:35:31.809435 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 506/1012] 10.222.0.13 () {60 vars in 1144 bytes} [Wed Aug 31 09:35:31 2022] POST /identity/v3/endpoints => generated 325 bytes in 274 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 09:35:31.958799 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-504b4eb0-2d59-4ebd-9741-69f467e481fd None None] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:35:31.961286 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=100 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:35:32.112168 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-504b4eb0-2d59-4ebd-9741-69f467e481fd None None] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:35:32.159088 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f297bd1-4543-4515-be89-2bbe21857bbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:35:32.159738 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f297bd1-4543-4515-be89-2bbe21857bbb None admin] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:32.160002 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f297bd1-4543-4515-be89-2bbe21857bbb None admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:32.160271 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f297bd1-4543-4515-be89-2bbe21857bbb None admin] PATH_INFO: `/v3/services/compute` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:32.160852 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f297bd1-4543-4515-be89-2bbe21857bbb None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:35:32.162262 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f297bd1-4543-4515-be89-2bbe21857bbb None admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:35:32.171921 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application [None req-0f297bd1-4543-4515-be89-2bbe21857bbb None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Aug 31 09:35:32.171921 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:35:32.171921 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 31 09:35:32.171921 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 31 09:35:32.171921 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 31 09:35:32.171921 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 31 09:35:32.171921 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 31 09:35:32.171921 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 09:35:32.171921 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 31 09:35:32.171921 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application Aug 31 09:35:32.171921 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 31 09:35:32.171921 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application Aug 31 09:35:32.171921 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:35:32.171921 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:35:32.171921 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:35:32.171921 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:35:32.171921 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:35:32.173066 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:35:32.173066 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:35:32.173066 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:35:32.173066 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:35:32.173066 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:35:32.173066 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:35:32.173066 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 31 09:35:32.173066 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 31 09:35:32.173066 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 31 09:35:32.173066 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 31 09:35:32.173066 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:35:32.173066 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:35:32.173066 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:35:32.173066 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:35:32.173066 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:35:32.173066 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:35:32.174137 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:35:32.174137 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application with Lock( Aug 31 09:35:32.174137 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:35:32.174137 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:35:32.174137 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:35:32.174137 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:35:32.174137 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:35:32.174137 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:35:32.174137 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:35:32.174137 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:35:32.174137 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 31 09:35:32.174137 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 09:35:32.174137 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 31 09:35:32.174137 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application Aug 31 09:35:32.174137 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 507/1013] 10.222.0.13 () {56 vars in 1111 bytes} [Wed Aug 31 09:35:31 2022] GET /identity/v3/services/compute => generated 88 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 09:35:32.182349 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b47545-70d8-444b-8891-145d71310798 None admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:35:32.303590 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b47545-70d8-444b-8891-145d71310798 None admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:35:32.303852 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4215edea-ab90-4b81-98e8-89a409435d6a None None] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:32.304548 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4215edea-ab90-4b81-98e8-89a409435d6a None None] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:32.304548 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4215edea-ab90-4b81-98e8-89a409435d6a None None] PATH_INFO: `/` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:32.305986 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 508/1014] 10.222.0.13 () {54 vars in 927 bytes} [Wed Aug 31 09:35:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:35:32.311868 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc28631-a303-4899-9b36-63209b290026 None None] REQUEST_METHOD: `POST` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:32.312254 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc28631-a303-4899-9b36-63209b290026 None None] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:32.312411 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc28631-a303-4899-9b36-63209b290026 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:32.332092 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.auth.core [None req-8bc28631-a303-4899-9b36-63209b290026 None None] MFA Rules not processed for user `fd836cf590a145df9acf19cf3ae2ac56`. Rule list: `[]` (Enabled: `True`). {{(pid=88113) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:35:32.333284 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.fernet_utils [None req-8bc28631-a303-4899-9b36-63209b290026 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:35:32.333756 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:35:32.351045 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55846595-72d7-45d6-8f32-a043f2efb9dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:35:32.351770 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55846595-72d7-45d6-8f32-a043f2efb9dc None admin] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:32.351964 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55846595-72d7-45d6-8f32-a043f2efb9dc None admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:32.352304 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55846595-72d7-45d6-8f32-a043f2efb9dc None admin] PATH_INFO: `/v3/services` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:32.352844 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55846595-72d7-45d6-8f32-a043f2efb9dc None admin] RBAC: Adding query filter params (name=compute) {{(pid=88114) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:35:32.353146 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55846595-72d7-45d6-8f32-a043f2efb9dc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:35:32.354581 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55846595-72d7-45d6-8f32-a043f2efb9dc None admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:35:32.360871 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 507/1015] 10.222.0.13 () {56 vars in 1112 bytes} [Wed Aug 31 09:35:32 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:35:32.366665 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55846595-72d7-45d6-8f32-a043f2efb9dc None admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:35:32.407819 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=100 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:35:32.423377 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 509/1016] 10.222.0.13 () {58 vars in 1024 bytes} [Wed Aug 31 09:35:32 2022] POST /identity/v3/auth/tokens => generated 1813 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 31 09:35:32.528183 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55846595-72d7-45d6-8f32-a043f2efb9dc None admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:35:32.593291 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-919bade1-d84b-4083-8670-128b9927ba24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:35:32.594444 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919bade1-d84b-4083-8670-128b9927ba24 None admin] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:32.595024 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919bade1-d84b-4083-8670-128b9927ba24 None admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:32.595583 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919bade1-d84b-4083-8670-128b9927ba24 None admin] PATH_INFO: `/v3/services` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:32.596799 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-919bade1-d84b-4083-8670-128b9927ba24 None admin] RBAC: Adding query filter params (type=compute) {{(pid=88114) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:35:32.597452 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-919bade1-d84b-4083-8670-128b9927ba24 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:35:32.600141 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-919bade1-d84b-4083-8670-128b9927ba24 None admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:35:32.615098 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 508/1017] 10.222.0.13 () {56 vars in 1112 bytes} [Wed Aug 31 09:35:32 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 09:35:32.618586 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc28631-a303-4899-9b36-63209b290026 None None] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:35:32.700768 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-919bade1-d84b-4083-8670-128b9927ba24 None admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:35:32.738590 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc28631-a303-4899-9b36-63209b290026 None None] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:35:32.767110 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=100 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:35:32.784472 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b125508-f6da-4a17-9759-30d0c0253d75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:35:32.785352 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b125508-f6da-4a17-9759-30d0c0253d75 None admin] REQUEST_METHOD: `POST` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:32.785826 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b125508-f6da-4a17-9759-30d0c0253d75 None admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:32.786087 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b125508-f6da-4a17-9759-30d0c0253d75 None admin] PATH_INFO: `/v3/endpoints` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:32.786781 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b125508-f6da-4a17-9759-30d0c0253d75 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:35:32.788176 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b125508-f6da-4a17-9759-30d0c0253d75 None admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:35:32.821553 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 510/1018] 10.222.0.13 () {60 vars in 1144 bytes} [Wed Aug 31 09:35:32 2022] POST /identity/v3/endpoints => generated 334 bytes in 206 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 09:35:32.851462 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-919bade1-d84b-4083-8670-128b9927ba24 None admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:35:32.906257 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57542833-323c-4349-bf6a-cb812fa59b77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:35:32.906799 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57542833-323c-4349-bf6a-cb812fa59b77 None admin] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:32.907071 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57542833-323c-4349-bf6a-cb812fa59b77 None admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:32.907336 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57542833-323c-4349-bf6a-cb812fa59b77 None admin] PATH_INFO: `/v3/services/image` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:32.907922 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57542833-323c-4349-bf6a-cb812fa59b77 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:35:32.909325 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57542833-323c-4349-bf6a-cb812fa59b77 None admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:35:32.919464 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application [None req-57542833-323c-4349-bf6a-cb812fa59b77 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Aug 31 09:35:32.919464 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:35:32.919464 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 31 09:35:32.919464 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 31 09:35:32.919464 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 31 09:35:32.919464 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 31 09:35:32.919464 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 31 09:35:32.919464 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 09:35:32.919464 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 31 09:35:32.919464 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application Aug 31 09:35:32.919464 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 31 09:35:32.919464 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application Aug 31 09:35:32.919464 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:35:32.919464 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:35:32.919464 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:35:32.919464 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:35:32.919464 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:35:32.920614 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:35:32.920614 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:35:32.920614 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:35:32.920614 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:35:32.920614 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:35:32.920614 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:35:32.920614 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 31 09:35:32.920614 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 31 09:35:32.920614 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 31 09:35:32.920614 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 31 09:35:32.920614 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:35:32.920614 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:35:32.920614 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:35:32.920614 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:35:32.920614 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:35:32.920614 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:35:32.921969 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:35:32.921969 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application with Lock( Aug 31 09:35:32.921969 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:35:32.921969 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:35:32.921969 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:35:32.921969 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:35:32.921969 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:35:32.921969 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:35:32.921969 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:35:32.921969 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:35:32.921969 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 31 09:35:32.921969 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 09:35:32.921969 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 31 09:35:32.921969 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application Aug 31 09:35:32.921969 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 509/1019] 10.222.0.13 () {56 vars in 1105 bytes} [Wed Aug 31 09:35:32 2022] GET /identity/v3/services/image => generated 86 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 09:35:32.931158 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b125508-f6da-4a17-9759-30d0c0253d75 None admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:35:33.034624 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:35:33.108968 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b125508-f6da-4a17-9759-30d0c0253d75 None admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:35:33.150945 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5098671-66ef-4301-a4cc-0f9c136302d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:35:33.151570 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5098671-66ef-4301-a4cc-0f9c136302d0 None admin] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:33.151847 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5098671-66ef-4301-a4cc-0f9c136302d0 None admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:33.152123 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5098671-66ef-4301-a4cc-0f9c136302d0 None admin] PATH_INFO: `/v3/services` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:33.152780 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5098671-66ef-4301-a4cc-0f9c136302d0 None admin] RBAC: Adding query filter params (name=image) {{(pid=88113) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:35:33.153082 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5098671-66ef-4301-a4cc-0f9c136302d0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:35:33.155316 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5098671-66ef-4301-a4cc-0f9c136302d0 None admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:35:33.162019 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 511/1020] 10.222.0.13 () {56 vars in 1108 bytes} [Wed Aug 31 09:35:32 2022] GET /identity/v3/services?name=image => generated 122 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:35:33.170400 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57542833-323c-4349-bf6a-cb812fa59b77 None admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:35:33.319648 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57542833-323c-4349-bf6a-cb812fa59b77 None admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:35:33.332545 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=100 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:35:33.365267 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c53b2e55-db07-4002-b676-8a81ca690013 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:35:33.365963 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53b2e55-db07-4002-b676-8a81ca690013 None admin] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:33.366252 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53b2e55-db07-4002-b676-8a81ca690013 None admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:33.366508 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53b2e55-db07-4002-b676-8a81ca690013 None admin] PATH_INFO: `/v3/services` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:33.367222 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c53b2e55-db07-4002-b676-8a81ca690013 None admin] RBAC: Adding query filter params (type=image) {{(pid=88114) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:35:33.367554 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c53b2e55-db07-4002-b676-8a81ca690013 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:35:33.368949 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c53b2e55-db07-4002-b676-8a81ca690013 None admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:35:33.375805 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 510/1021] 10.222.0.13 () {56 vars in 1108 bytes} [Wed Aug 31 09:35:33 2022] GET /identity/v3/services?type=image => generated 350 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 09:35:33.381802 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5098671-66ef-4301-a4cc-0f9c136302d0 None admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:35:33.548715 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5098671-66ef-4301-a4cc-0f9c136302d0 None admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:35:33.563576 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=100 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:35:33.590090 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e178eb0-753b-40d8-ab05-845887d79de3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:35:33.590714 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e178eb0-753b-40d8-ab05-845887d79de3 None admin] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:33.591561 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e178eb0-753b-40d8-ab05-845887d79de3 None admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:33.591828 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e178eb0-753b-40d8-ab05-845887d79de3 None admin] PATH_INFO: `/v3/endpoints` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:33.592550 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e178eb0-753b-40d8-ab05-845887d79de3 None admin] RBAC: Adding query filter params (interface=public, service_id=9cd620de5d9240bc822b64f7c2ba27b7, region_id=RegionOne) {{(pid=88113) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:35:33.592855 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e178eb0-753b-40d8-ab05-845887d79de3 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:35:33.594288 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e178eb0-753b-40d8-ab05-845887d79de3 None admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:35:33.601238 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 512/1022] 10.222.0.13 () {56 vars in 1251 bytes} [Wed Aug 31 09:35:33 2022] GET /identity/v3/endpoints?service_id=9cd620de5d9240bc822b64f7c2ba27b7&interface=public®ion_id=RegionOne => generated 194 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:35:33.608476 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c53b2e55-db07-4002-b676-8a81ca690013 None admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:35:33.758245 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c53b2e55-db07-4002-b676-8a81ca690013 None admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:35:33.810748 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a8c37d-65ff-42d9-8820-9eefcc4934fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:35:33.811175 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a8c37d-65ff-42d9-8820-9eefcc4934fa None admin] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:33.811662 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a8c37d-65ff-42d9-8820-9eefcc4934fa None admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:33.812133 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a8c37d-65ff-42d9-8820-9eefcc4934fa None admin] PATH_INFO: `/v3/services` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:33.812765 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81a8c37d-65ff-42d9-8820-9eefcc4934fa None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:35:33.814196 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81a8c37d-65ff-42d9-8820-9eefcc4934fa None admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:35:33.821782 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 511/1023] 10.222.0.13 () {56 vars in 1087 bytes} [Wed Aug 31 09:35:33 2022] GET /identity/v3/services => generated 1957 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 09:35:33.963474 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0eb1fc0-3f8d-4ccd-ad7b-b7b25611b90e None None] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:33.963974 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0eb1fc0-3f8d-4ccd-ad7b-b7b25611b90e None None] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:33.964471 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0eb1fc0-3f8d-4ccd-ad7b-b7b25611b90e None None] PATH_INFO: `/` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:33.967797 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 513/1024] 10.222.0.13 () {54 vars in 927 bytes} [Wed Aug 31 09:35:33 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:35:33.977787 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049471b0-2a30-40f7-bae4-2fd46123aa71 None None] REQUEST_METHOD: `POST` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:33.978286 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049471b0-2a30-40f7-bae4-2fd46123aa71 None None] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:33.978778 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049471b0-2a30-40f7-bae4-2fd46123aa71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:34.008437 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.auth.core [None req-049471b0-2a30-40f7-bae4-2fd46123aa71 None None] MFA Rules not processed for user `fd836cf590a145df9acf19cf3ae2ac56`. Rule list: `[]` (Enabled: `True`). {{(pid=88114) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:35:34.009686 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.fernet_utils [None req-049471b0-2a30-40f7-bae4-2fd46123aa71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88114) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:35:34.058853 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=100 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:35:34.143404 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 512/1025] 10.222.0.13 () {58 vars in 1024 bytes} [Wed Aug 31 09:35:33 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 09:35:34.506269 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0eb1fc0-3f8d-4ccd-ad7b-b7b25611b90e None None] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:35:34.631922 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0eb1fc0-3f8d-4ccd-ad7b-b7b25611b90e None None] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:35:34.672884 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32fae6ba-0386-47ac-b8c8-6fd6f73d3d33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:35:34.673578 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32fae6ba-0386-47ac-b8c8-6fd6f73d3d33 None admin] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:34.673851 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32fae6ba-0386-47ac-b8c8-6fd6f73d3d33 None admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:34.674116 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32fae6ba-0386-47ac-b8c8-6fd6f73d3d33 None admin] PATH_INFO: `/v3/domains/default` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:34.675423 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32fae6ba-0386-47ac-b8c8-6fd6f73d3d33 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:35:34.677038 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32fae6ba-0386-47ac-b8c8-6fd6f73d3d33 None admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:35:34.678960 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 514/1026] 10.222.0.13 () {56 vars in 1108 bytes} [Wed Aug 31 09:35:34 2022] GET /identity/v3/domains/default => generated 199 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 09:35:34.689196 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-049471b0-2a30-40f7-bae4-2fd46123aa71 None None] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:35:34.898046 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-049471b0-2a30-40f7-bae4-2fd46123aa71 None None] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:35:34.968989 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec55379-f954-44ed-b462-4bda63a05822 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:35:34.973891 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec55379-f954-44ed-b462-4bda63a05822 None admin] REQUEST_METHOD: `POST` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:34.973891 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec55379-f954-44ed-b462-4bda63a05822 None admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:34.973891 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec55379-f954-44ed-b462-4bda63a05822 None admin] PATH_INFO: `/v3/projects` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:34.973891 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ec55379-f954-44ed-b462-4bda63a05822 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:35:34.977422 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ec55379-f954-44ed-b462-4bda63a05822 None admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:35:34.986233 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c76eaa-2165-4818-9cd0-3c4aee9e2816 None None] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:34.986731 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c76eaa-2165-4818-9cd0-3c4aee9e2816 None None] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:34.987241 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c76eaa-2165-4818-9cd0-3c4aee9e2816 None None] PATH_INFO: `/` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:34.989799 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 515/1027] 10.222.0.13 () {54 vars in 927 bytes} [Wed Aug 31 09:35:34 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:35:34.996433 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4e0925-3877-4723-ae3d-f7f2d2351137 None None] REQUEST_METHOD: `POST` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:34.996733 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4e0925-3877-4723-ae3d-f7f2d2351137 None None] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:34.997080 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4e0925-3877-4723-ae3d-f7f2d2351137 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:35.022131 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.auth.core [None req-7e4e0925-3877-4723-ae3d-f7f2d2351137 None None] MFA Rules not processed for user `fd836cf590a145df9acf19cf3ae2ac56`. Rule list: `[]` (Enabled: `True`). {{(pid=88113) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:35:35.024244 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.fernet_utils [None req-7e4e0925-3877-4723-ae3d-f7f2d2351137 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:35:35.031971 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 513/1028] 10.222.0.13 () {60 vars in 1141 bytes} [Wed Aug 31 09:35:34 2022] POST /identity/v3/projects => generated 311 bytes in 348 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 09:35:35.083728 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=100 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:35:35.140334 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 516/1029] 10.222.0.13 () {58 vars in 1024 bytes} [Wed Aug 31 09:35:34 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 31 09:35:35.475152 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec55379-f954-44ed-b462-4bda63a05822 None admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:35:35.508143 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:35:35.656698 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec55379-f954-44ed-b462-4bda63a05822 None admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:35:35.729312 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90da5352-a3a2-4533-99d8-552979c8246e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:35:35.730441 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90da5352-a3a2-4533-99d8-552979c8246e None admin] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:35.730819 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90da5352-a3a2-4533-99d8-552979c8246e None admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:35.731925 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90da5352-a3a2-4533-99d8-552979c8246e None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:35.753729 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90da5352-a3a2-4533-99d8-552979c8246e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 09:35:35.762011 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application [None req-90da5352-a3a2-4533-99d8-552979c8246e None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 31 09:35:35.762011 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:35:35.762011 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:35:35.762011 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:35:35.762011 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:35:35.762011 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:35:35.762011 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:35:35.762011 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:35:35.762011 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:35:35.762011 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:35:35.762011 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:35:35.762011 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:35:35.762011 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 09:35:35.762011 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 09:35:35.762011 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 09:35:35.762011 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 09:35:35.763946 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 09:35:35.763946 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 09:35:35.763946 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 09:35:35.763946 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 09:35:35.763946 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:35:35.763946 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:35:35.763946 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:35:35.763946 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:35:35.763946 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:35:35.763946 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:35:35.763946 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:35:35.763946 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application with Lock( Aug 31 09:35:35.763946 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:35:35.763946 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:35:35.763946 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:35:35.763946 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:35:35.766208 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:35:35.766208 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:35:35.766208 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:35:35.766208 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:35:35.766208 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 09:35:35.766208 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 09:35:35.766208 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 09:35:35.766208 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 09:35:35.766208 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 09:35:35.766208 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 09:35:35.766208 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 31 09:35:35.766208 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application Aug 31 09:35:35.766208 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 514/1030] 10.222.0.13 () {56 vars in 1120 bytes} [Wed Aug 31 09:35:35 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 290 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 09:35:35.775646 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e4e0925-3877-4723-ae3d-f7f2d2351137 None None] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:35:35.841762 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc1cf00-3ba0-4a14-a1e3-9da5445508c7 None None] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:35.841762 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc1cf00-3ba0-4a14-a1e3-9da5445508c7 None None] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:35.841762 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc1cf00-3ba0-4a14-a1e3-9da5445508c7 None None] PATH_INFO: `/` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:35.843906 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 515/1031] 10.222.0.13 () {54 vars in 927 bytes} [Wed Aug 31 09:35:35 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 31 09:35:35.850258 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa51d73-46be-4f61-a1d4-eaabaa1896b8 None None] REQUEST_METHOD: `POST` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:35.850700 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa51d73-46be-4f61-a1d4-eaabaa1896b8 None None] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:35.851106 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa51d73-46be-4f61-a1d4-eaabaa1896b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:35.881483 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.auth.core [None req-6aa51d73-46be-4f61-a1d4-eaabaa1896b8 None None] MFA Rules not processed for user `fd836cf590a145df9acf19cf3ae2ac56`. Rule list: `[]` (Enabled: `True`). {{(pid=88114) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:35:35.883638 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.fernet_utils [None req-6aa51d73-46be-4f61-a1d4-eaabaa1896b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88114) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:35:35.945588 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e4e0925-3877-4723-ae3d-f7f2d2351137 None None] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:35:36.008126 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 516/1032] 10.222.0.13 () {58 vars in 1024 bytes} [Wed Aug 31 09:35:35 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 31 09:35:36.008533 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83073ec7-dd9a-4900-901a-69cb68de7c79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:35:36.009707 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83073ec7-dd9a-4900-901a-69cb68de7c79 None admin] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:36.010049 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83073ec7-dd9a-4900-901a-69cb68de7c79 None admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:36.010553 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83073ec7-dd9a-4900-901a-69cb68de7c79 None admin] PATH_INFO: `/v3/roles` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:36.011827 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83073ec7-dd9a-4900-901a-69cb68de7c79 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=88113) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:35:36.012381 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83073ec7-dd9a-4900-901a-69cb68de7c79 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:35:36.015008 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83073ec7-dd9a-4900-901a-69cb68de7c79 None admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:35:36.031457 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 517/1033] 10.222.0.13 () {56 vars in 1115 bytes} [Wed Aug 31 09:35:35 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:35:36.037436 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa51d73-46be-4f61-a1d4-eaabaa1896b8 None None] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:35:36.058697 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=100 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:35:36.235515 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa51d73-46be-4f61-a1d4-eaabaa1896b8 None None] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:35:36.296809 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a944fa-9be1-4faf-9e9e-affaf9c09d87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:35:36.297620 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a944fa-9be1-4faf-9e9e-affaf9c09d87 None admin] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:36.298024 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a944fa-9be1-4faf-9e9e-affaf9c09d87 None admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:36.298417 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a944fa-9be1-4faf-9e9e-affaf9c09d87 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:36.311205 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4a944fa-9be1-4faf-9e9e-affaf9c09d87 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:35:36.312998 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4a944fa-9be1-4faf-9e9e-affaf9c09d87 None admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:35:36.317030 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83073ec7-dd9a-4900-901a-69cb68de7c79 None admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:35:36.332473 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application [None req-a4a944fa-9be1-4faf-9e9e-affaf9c09d87 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 09:35:36.332473 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:35:36.332473 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:35:36.332473 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:35:36.332473 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:35:36.332473 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:35:36.332473 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:35:36.332473 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:35:36.332473 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:35:36.332473 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:35:36.332473 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:35:36.332473 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:35:36.332473 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 09:35:36.332473 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 09:35:36.332473 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 09:35:36.332473 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 09:35:36.334360 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:35:36.334360 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:35:36.334360 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:35:36.334360 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:35:36.334360 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:35:36.334360 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:35:36.334360 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:35:36.334360 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application with Lock( Aug 31 09:35:36.334360 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:35:36.334360 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:35:36.334360 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:35:36.334360 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:35:36.334360 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:35:36.334360 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:35:36.334360 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:35:36.334360 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:35:36.334360 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 09:35:36.342081 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 09:35:36.342081 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 09:35:36.342081 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application Aug 31 09:35:36.342081 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 517/1034] 10.222.0.13 () {56 vars in 1108 bytes} [Wed Aug 31 09:35:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 298 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:35:36.344835 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a944fa-9be1-4faf-9e9e-affaf9c09d87 None admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:35:36.360849 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=100 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:35:36.445604 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=100 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:35:36.466607 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83073ec7-dd9a-4900-901a-69cb68de7c79 None admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:35:36.510156 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a944fa-9be1-4faf-9e9e-affaf9c09d87 None admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:35:36.521005 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e44aa106-ac15-45df-b123-b00d6c52b183 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:35:36.522581 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44aa106-ac15-45df-b123-b00d6c52b183 None admin] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:36.523401 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44aa106-ac15-45df-b123-b00d6c52b183 None admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:36.523887 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44aa106-ac15-45df-b123-b00d6c52b183 None admin] PATH_INFO: `/v3/services/image` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:36.524901 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e44aa106-ac15-45df-b123-b00d6c52b183 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:35:36.527346 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e44aa106-ac15-45df-b123-b00d6c52b183 None admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:35:36.540889 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application [None req-e44aa106-ac15-45df-b123-b00d6c52b183 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Aug 31 09:35:36.540889 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:35:36.540889 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 31 09:35:36.540889 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 31 09:35:36.540889 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 31 09:35:36.540889 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 31 09:35:36.540889 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 31 09:35:36.540889 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 09:35:36.540889 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 31 09:35:36.540889 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application Aug 31 09:35:36.540889 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 31 09:35:36.540889 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application Aug 31 09:35:36.540889 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:35:36.540889 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:35:36.540889 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:35:36.540889 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:35:36.540889 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:35:36.542397 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:35:36.542397 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:35:36.542397 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:35:36.542397 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:35:36.542397 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:35:36.542397 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:35:36.542397 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 31 09:35:36.542397 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 31 09:35:36.542397 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 31 09:35:36.542397 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 31 09:35:36.542397 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:35:36.542397 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:35:36.542397 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:35:36.542397 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:35:36.542397 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:35:36.542397 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:35:36.543453 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:35:36.543453 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application with Lock( Aug 31 09:35:36.543453 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:35:36.543453 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:35:36.543453 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:35:36.543453 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:35:36.543453 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:35:36.543453 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:35:36.543453 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:35:36.543453 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:35:36.543453 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 31 09:35:36.543453 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 09:35:36.543453 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 31 09:35:36.543453 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application Aug 31 09:35:36.549525 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 518/1035] 10.222.0.13 () {56 vars in 1105 bytes} [Wed Aug 31 09:35:36 2022] GET /identity/v3/services/image => generated 86 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 09:35:36.549723 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08450d0a-620b-4fef-b6e8-c28833da74a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:35:36.549723 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08450d0a-620b-4fef-b6e8-c28833da74a9 None admin] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:36.549723 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08450d0a-620b-4fef-b6e8-c28833da74a9 None admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:36.549723 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08450d0a-620b-4fef-b6e8-c28833da74a9 None admin] PATH_INFO: `/v3/domains` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:36.550203 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08450d0a-620b-4fef-b6e8-c28833da74a9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88114) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:35:36.550203 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08450d0a-620b-4fef-b6e8-c28833da74a9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:35:36.551144 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08450d0a-620b-4fef-b6e8-c28833da74a9 None admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:35:36.555149 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e44aa106-ac15-45df-b123-b00d6c52b183 None admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:35:36.571691 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 518/1036] 10.222.0.13 () {56 vars in 1109 bytes} [Wed Aug 31 09:35:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:35:36.576191 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08450d0a-620b-4fef-b6e8-c28833da74a9 None admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:35:36.729311 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=100 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:35:36.734844 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e44aa106-ac15-45df-b123-b00d6c52b183 None admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:35:36.761501 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08450d0a-620b-4fef-b6e8-c28833da74a9 None admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:35:36.800766 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12adf36a-e80e-45f5-a822-993485b2f445 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:35:36.801772 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12adf36a-e80e-45f5-a822-993485b2f445 None admin] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:36.802244 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12adf36a-e80e-45f5-a822-993485b2f445 None admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:36.802720 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12adf36a-e80e-45f5-a822-993485b2f445 None admin] PATH_INFO: `/v3/services` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:36.803822 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12adf36a-e80e-45f5-a822-993485b2f445 None admin] RBAC: Adding query filter params (name=image) {{(pid=88113) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:35:36.804404 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12adf36a-e80e-45f5-a822-993485b2f445 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:35:36.806794 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=100 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:35:36.806952 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12adf36a-e80e-45f5-a822-993485b2f445 None admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:35:36.821444 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 519/1037] 10.222.0.13 () {56 vars in 1108 bytes} [Wed Aug 31 09:35:36 2022] GET /identity/v3/services?name=image => generated 122 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 09:35:36.824306 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12adf36a-e80e-45f5-a822-993485b2f445 None admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:35:36.831623 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3707b42d-b9f3-4e0c-b90d-6d08334fa56d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:35:36.832716 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3707b42d-b9f3-4e0c-b90d-6d08334fa56d None admin] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:36.833203 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3707b42d-b9f3-4e0c-b90d-6d08334fa56d None admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:36.833739 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3707b42d-b9f3-4e0c-b90d-6d08334fa56d None admin] PATH_INFO: `/v3/users/nova` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:36.848378 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3707b42d-b9f3-4e0c-b90d-6d08334fa56d None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:35:36.851093 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3707b42d-b9f3-4e0c-b90d-6d08334fa56d None admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:35:36.874691 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application [None req-3707b42d-b9f3-4e0c-b90d-6d08334fa56d None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 31 09:35:36.874691 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:35:36.874691 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:35:36.874691 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:35:36.874691 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:35:36.874691 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:35:36.874691 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:35:36.874691 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:35:36.874691 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:35:36.874691 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:35:36.874691 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:35:36.874691 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:35:36.874691 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 31 09:35:36.874691 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 31 09:35:36.874691 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 31 09:35:36.874691 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 31 09:35:36.875734 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:35:36.875734 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:35:36.875734 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 31 09:35:36.875734 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 09:35:36.875734 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 31 09:35:36.875734 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 09:35:36.875734 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:35:36.875734 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:35:36.875734 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:35:36.875734 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:35:36.875734 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:35:36.875734 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application with Lock( Aug 31 09:35:36.875734 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:35:36.875734 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:35:36.875734 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:35:36.875734 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:35:36.875734 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:35:36.875734 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:35:36.876741 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:35:36.876741 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:35:36.876741 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 31 09:35:36.876741 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 31 09:35:36.876741 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 31 09:35:36.876741 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 31 09:35:36.876741 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 31 09:35:36.876741 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 31 09:35:36.876741 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 31 09:35:36.876741 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application Aug 31 09:35:36.876741 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 519/1038] 10.222.0.13 () {56 vars in 1093 bytes} [Wed Aug 31 09:35:36 2022] GET /identity/v3/users/nova => generated 82 bytes in 302 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 09:35:36.887309 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3707b42d-b9f3-4e0c-b90d-6d08334fa56d None admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:35:36.983157 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12adf36a-e80e-45f5-a822-993485b2f445 None admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:35:37.046500 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d235619e-2c22-4b44-9917-e370e75d9bed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:35:37.047530 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d235619e-2c22-4b44-9917-e370e75d9bed None admin] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:37.048006 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d235619e-2c22-4b44-9917-e370e75d9bed None admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:37.048482 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d235619e-2c22-4b44-9917-e370e75d9bed None admin] PATH_INFO: `/v3/services` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:37.050398 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d235619e-2c22-4b44-9917-e370e75d9bed None admin] RBAC: Adding query filter params (type=image) {{(pid=88113) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:35:37.050998 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d235619e-2c22-4b44-9917-e370e75d9bed None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:35:37.053593 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d235619e-2c22-4b44-9917-e370e75d9bed None admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:35:37.067498 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 520/1039] 10.222.0.13 () {56 vars in 1108 bytes} [Wed Aug 31 09:35:36 2022] GET /identity/v3/services?type=image => generated 350 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 09:35:37.069615 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d235619e-2c22-4b44-9917-e370e75d9bed None admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:35:37.075707 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3707b42d-b9f3-4e0c-b90d-6d08334fa56d None admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:35:37.123517 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=100 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:35:37.138335 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0601766-08ed-4c68-b431-92431982324c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:35:37.139375 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0601766-08ed-4c68-b431-92431982324c None admin] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:37.139886 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0601766-08ed-4c68-b431-92431982324c None admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:37.140318 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0601766-08ed-4c68-b431-92431982324c None admin] PATH_INFO: `/v3/users` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:37.141689 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0601766-08ed-4c68-b431-92431982324c None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=88114) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:35:37.142226 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0601766-08ed-4c68-b431-92431982324c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:35:37.145188 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0601766-08ed-4c68-b431-92431982324c None admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:35:37.186140 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 520/1040] 10.222.0.13 () {56 vars in 1133 bytes} [Wed Aug 31 09:35:36 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 293 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:35:37.187523 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0601766-08ed-4c68-b431-92431982324c None admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:35:37.234293 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d235619e-2c22-4b44-9917-e370e75d9bed None admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:35:37.273909 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da688ab-3e02-447e-9367-8566f409bedf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:35:37.274533 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da688ab-3e02-447e-9367-8566f409bedf None admin] REQUEST_METHOD: `POST` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:37.274816 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da688ab-3e02-447e-9367-8566f409bedf None admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:37.275091 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da688ab-3e02-447e-9367-8566f409bedf None admin] PATH_INFO: `/v3/endpoints` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:37.275787 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5da688ab-3e02-447e-9367-8566f409bedf None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:35:37.277208 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5da688ab-3e02-447e-9367-8566f409bedf None admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:35:37.277614 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=100 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:35:37.300289 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 521/1041] 10.222.0.13 () {60 vars in 1144 bytes} [Wed Aug 31 09:35:37 2022] POST /identity/v3/endpoints => generated 327 bytes in 232 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 09:35:37.301492 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23551f8a-66bc-4e82-b6e4-5d2fa98ab0c0 None None] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:37.301724 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23551f8a-66bc-4e82-b6e4-5d2fa98ab0c0 None None] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:37.302004 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23551f8a-66bc-4e82-b6e4-5d2fa98ab0c0 None None] PATH_INFO: `/` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:37.303289 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 522/1042] 10.222.0.13 () {54 vars in 927 bytes} [Wed Aug 31 09:35:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:35:37.309152 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae0dbad-cfe8-4f0a-ad7a-22d71cc8e7b3 None None] REQUEST_METHOD: `POST` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:37.309508 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae0dbad-cfe8-4f0a-ad7a-22d71cc8e7b3 None None] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:37.309819 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae0dbad-cfe8-4f0a-ad7a-22d71cc8e7b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:37.332154 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.auth.core [None req-cae0dbad-cfe8-4f0a-ad7a-22d71cc8e7b3 None None] MFA Rules not processed for user `fd836cf590a145df9acf19cf3ae2ac56`. Rule list: `[]` (Enabled: `True`). {{(pid=88113) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:35:37.333450 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.fernet_utils [None req-cae0dbad-cfe8-4f0a-ad7a-22d71cc8e7b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:35:37.389644 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0601766-08ed-4c68-b431-92431982324c None admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:35:37.437903 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 523/1043] 10.222.0.13 () {58 vars in 1024 bytes} [Wed Aug 31 09:35:37 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 31 09:35:37.442946 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f76d38a-a1d6-44fd-b049-cc88c5441800 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:35:37.443764 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f76d38a-a1d6-44fd-b049-cc88c5441800 None admin] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:37.444388 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f76d38a-a1d6-44fd-b049-cc88c5441800 None admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:37.444654 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f76d38a-a1d6-44fd-b049-cc88c5441800 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:37.540244 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f76d38a-a1d6-44fd-b049-cc88c5441800 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:35:37.542737 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f76d38a-a1d6-44fd-b049-cc88c5441800 None admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:35:37.622516 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=100 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:35:37.714452 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae0dbad-cfe8-4f0a-ad7a-22d71cc8e7b3 None None] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:35:37.735674 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:35:37.745438 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 521/1044] 10.222.0.13 () {58 vars in 1282 bytes} [Wed Aug 31 09:35:37 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 558 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 31 09:35:37.753102 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f76d38a-a1d6-44fd-b049-cc88c5441800 None admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:35:37.894293 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae0dbad-cfe8-4f0a-ad7a-22d71cc8e7b3 None None] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:35:37.934874 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f76d38a-a1d6-44fd-b049-cc88c5441800 None admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:35:37.941575 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d3a919-7b83-43e9-b0d9-b2cff74cb6bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:35:37.942387 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d3a919-7b83-43e9-b0d9-b2cff74cb6bf None admin] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:37.942787 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d3a919-7b83-43e9-b0d9-b2cff74cb6bf None admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:37.943183 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d3a919-7b83-43e9-b0d9-b2cff74cb6bf None admin] PATH_INFO: `/v3/domains/default` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:37.944478 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45d3a919-7b83-43e9-b0d9-b2cff74cb6bf None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:35:37.946106 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45d3a919-7b83-43e9-b0d9-b2cff74cb6bf None admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:35:37.948164 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 524/1045] 10.222.0.13 () {56 vars in 1108 bytes} [Wed Aug 31 09:35:37 2022] GET /identity/v3/domains/default => generated 199 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 09:35:37.955491 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d3a919-7b83-43e9-b0d9-b2cff74cb6bf None admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:35:38.007776 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf213460-465c-4651-81f4-b4b97e6726e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:35:38.008739 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf213460-465c-4651-81f4-b4b97e6726e4 None admin] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:38.009161 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf213460-465c-4651-81f4-b4b97e6726e4 None admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:38.009639 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf213460-465c-4651-81f4-b4b97e6726e4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:38.027661 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=100 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:35:38.033218 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf213460-465c-4651-81f4-b4b97e6726e4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:35:38.037793 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf213460-465c-4651-81f4-b4b97e6726e4 None admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:35:38.055984 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application [None req-bf213460-465c-4651-81f4-b4b97e6726e4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 09:35:38.055984 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:35:38.055984 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:35:38.055984 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:35:38.055984 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:35:38.055984 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:35:38.055984 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:35:38.055984 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:35:38.055984 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:35:38.055984 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:35:38.055984 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:35:38.055984 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:35:38.055984 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 09:35:38.055984 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 09:35:38.055984 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 09:35:38.055984 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 09:35:38.057877 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:35:38.057877 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:35:38.057877 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:35:38.057877 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:35:38.057877 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:35:38.057877 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:35:38.057877 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:35:38.057877 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application with Lock( Aug 31 09:35:38.057877 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:35:38.057877 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:35:38.057877 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:35:38.057877 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:35:38.057877 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:35:38.057877 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:35:38.057877 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:35:38.057877 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:35:38.057877 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 09:35:38.059664 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 09:35:38.059664 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 09:35:38.059664 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application Aug 31 09:35:38.064354 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 522/1046] 10.222.0.13 () {56 vars in 1108 bytes} [Wed Aug 31 09:35:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 313 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 09:35:38.065529 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf213460-465c-4651-81f4-b4b97e6726e4 None admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:35:38.097749 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=100 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:35:38.135192 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d3a919-7b83-43e9-b0d9-b2cff74cb6bf None admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:35:38.193619 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb3f6c5-2ef9-4291-bbd1-a4e92a60c464 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:35:38.194572 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb3f6c5-2ef9-4291-bbd1-a4e92a60c464 None admin] REQUEST_METHOD: `POST` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:38.195050 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb3f6c5-2ef9-4291-bbd1-a4e92a60c464 None admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:38.195524 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb3f6c5-2ef9-4291-bbd1-a4e92a60c464 None admin] PATH_INFO: `/v3/users` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:38.196709 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bb3f6c5-2ef9-4291-bbd1-a4e92a60c464 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:35:38.199381 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bb3f6c5-2ef9-4291-bbd1-a4e92a60c464 None admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:35:38.205705 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf213460-465c-4651-81f4-b4b97e6726e4 None admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:35:38.232336 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 525/1047] 10.222.0.13 () {60 vars in 1132 bytes} [Wed Aug 31 09:35:37 2022] POST /identity/v3/users => generated 285 bytes in 277 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 09:35:38.281853 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c19f405-f6fa-43c8-a021-771a584c7176 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:35:38.283454 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c19f405-f6fa-43c8-a021-771a584c7176 None admin] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:38.284182 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c19f405-f6fa-43c8-a021-771a584c7176 None admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:38.284911 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c19f405-f6fa-43c8-a021-771a584c7176 None admin] PATH_INFO: `/v3/domains` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:38.286434 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c19f405-f6fa-43c8-a021-771a584c7176 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88114) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:35:38.287196 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c19f405-f6fa-43c8-a021-771a584c7176 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:35:38.290093 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c19f405-f6fa-43c8-a021-771a584c7176 None admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:35:38.315012 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 523/1048] 10.222.0.13 () {56 vars in 1109 bytes} [Wed Aug 31 09:35:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:35:38.321765 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb3f6c5-2ef9-4291-bbd1-a4e92a60c464 None admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:35:38.455151 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb3f6c5-2ef9-4291-bbd1-a4e92a60c464 None admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:35:38.502537 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ccf9122-7920-42e6-af6a-8922f8d327fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:35:38.503202 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ccf9122-7920-42e6-af6a-8922f8d327fb None admin] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:38.504430 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ccf9122-7920-42e6-af6a-8922f8d327fb None admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:38.504430 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ccf9122-7920-42e6-af6a-8922f8d327fb None admin] PATH_INFO: `/v3/projects/service` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:38.506574 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:35:38.513067 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ccf9122-7920-42e6-af6a-8922f8d327fb None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:35:38.514641 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ccf9122-7920-42e6-af6a-8922f8d327fb None admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:35:38.530624 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application [None req-2ccf9122-7920-42e6-af6a-8922f8d327fb None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 09:35:38.530624 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:35:38.530624 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:35:38.530624 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:35:38.530624 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:35:38.530624 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:35:38.530624 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:35:38.530624 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:35:38.530624 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:35:38.530624 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:35:38.530624 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:35:38.530624 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:35:38.530624 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 31 09:35:38.530624 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 31 09:35:38.530624 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 31 09:35:38.532446 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 31 09:35:38.532446 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:35:38.532446 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:35:38.532446 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:35:38.532446 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:35:38.532446 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:35:38.532446 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:35:38.532446 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:35:38.532446 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application with Lock( Aug 31 09:35:38.532446 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:35:38.532446 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:35:38.532446 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:35:38.532446 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:35:38.532446 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:35:38.532446 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:35:38.532446 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:35:38.532446 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:35:38.534304 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 31 09:35:38.534304 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 31 09:35:38.534304 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 31 09:35:38.534304 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 31 09:35:38.534304 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 31 09:35:38.534304 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 31 09:35:38.534304 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 09:35:38.534304 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application Aug 31 09:35:38.534304 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 526/1049] 10.222.0.13 () {56 vars in 1111 bytes} [Wed Aug 31 09:35:38 2022] GET /identity/v3/projects/service => generated 88 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:35:38.542454 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c19f405-f6fa-43c8-a021-771a584c7176 None admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:35:38.620503 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=100 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:35:38.679822 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c19f405-f6fa-43c8-a021-771a584c7176 None admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:35:38.722641 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c68a6a10-e19c-48db-9cac-ddc89813878b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:35:38.723237 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68a6a10-e19c-48db-9cac-ddc89813878b None admin] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:38.723500 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68a6a10-e19c-48db-9cac-ddc89813878b None admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:38.723752 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68a6a10-e19c-48db-9cac-ddc89813878b None admin] PATH_INFO: `/v3/projects` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:38.724409 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c68a6a10-e19c-48db-9cac-ddc89813878b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88114) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:35:38.724731 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c68a6a10-e19c-48db-9cac-ddc89813878b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:35:38.726235 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c68a6a10-e19c-48db-9cac-ddc89813878b None admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:35:38.740898 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 524/1050] 10.222.0.13 () {56 vars in 1148 bytes} [Wed Aug 31 09:35:38 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:35:38.747770 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ccf9122-7920-42e6-af6a-8922f8d327fb None admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:35:38.883352 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ccf9122-7920-42e6-af6a-8922f8d327fb None admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:35:38.925553 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3542e851-6a9a-4749-9290-9d155634cfb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:35:38.926264 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3542e851-6a9a-4749-9290-9d155634cfb9 None admin] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:38.926591 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3542e851-6a9a-4749-9290-9d155634cfb9 None admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:38.926902 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3542e851-6a9a-4749-9290-9d155634cfb9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:38.927706 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3542e851-6a9a-4749-9290-9d155634cfb9 None admin] RBAC: Adding query filter params (role.id=d41c162f8e1748989a09382ced5ef3a6, scope.project.id=49548eb44d62415582af820ab66decee, user.id=dc076493239d4ccdbdc37c0369c49529) {{(pid=88113) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:35:38.928052 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3542e851-6a9a-4749-9290-9d155634cfb9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:35:38.929651 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3542e851-6a9a-4749-9290-9d155634cfb9 None admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:35:38.936560 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 527/1051] 10.222.0.13 () {56 vars in 1374 bytes} [Wed Aug 31 09:35:38 2022] GET /identity/v3/role_assignments?user.id=dc076493239d4ccdbdc37c0369c49529&scope.project.id=49548eb44d62415582af820ab66decee&role.id=d41c162f8e1748989a09382ced5ef3a6 => generated 259 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 09:35:40.072269 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e12edb-6005-40be-a05a-2dc44fd42345 None None] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:40.073045 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e12edb-6005-40be-a05a-2dc44fd42345 None None] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:40.073375 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e12edb-6005-40be-a05a-2dc44fd42345 None None] PATH_INFO: `/` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:40.076902 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 525/1052] 10.222.0.13 () {54 vars in 927 bytes} [Wed Aug 31 09:35:40 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:35:40.085362 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd2cb94-902e-4866-a567-e318f5b52c26 None None] REQUEST_METHOD: `POST` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:40.086199 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd2cb94-902e-4866-a567-e318f5b52c26 None None] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:40.086348 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd2cb94-902e-4866-a567-e318f5b52c26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:40.118189 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.auth.core [None req-cfd2cb94-902e-4866-a567-e318f5b52c26 None None] MFA Rules not processed for user `fd836cf590a145df9acf19cf3ae2ac56`. Rule list: `[]` (Enabled: `True`). {{(pid=88113) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:35:40.120184 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.fernet_utils [None req-cfd2cb94-902e-4866-a567-e318f5b52c26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:35:40.227165 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=100 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:35:40.255343 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 528/1053] 10.222.0.13 () {58 vars in 1024 bytes} [Wed Aug 31 09:35:40 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 09:35:40.666235 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81fe849-d0d1-4a16-9e6e-e07238a3c8d8 None None] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:40.666687 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81fe849-d0d1-4a16-9e6e-e07238a3c8d8 None None] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:40.667036 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81fe849-d0d1-4a16-9e6e-e07238a3c8d8 None None] PATH_INFO: `/` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:40.670606 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 526/1054] 10.222.0.13 () {54 vars in 927 bytes} [Wed Aug 31 09:35:40 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:35:40.678767 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c410df-d29f-4f4b-aa08-fa62afade2e6 None None] REQUEST_METHOD: `POST` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:40.679248 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c410df-d29f-4f4b-aa08-fa62afade2e6 None None] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:40.679702 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c410df-d29f-4f4b-aa08-fa62afade2e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:40.708506 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.auth.core [None req-73c410df-d29f-4f4b-aa08-fa62afade2e6 None None] MFA Rules not processed for user `fd836cf590a145df9acf19cf3ae2ac56`. Rule list: `[]` (Enabled: `True`). {{(pid=88113) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:35:40.710465 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.fernet_utils [None req-73c410df-d29f-4f4b-aa08-fa62afade2e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:35:40.755673 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c81fe849-d0d1-4a16-9e6e-e07238a3c8d8 None None] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:35:40.823378 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 529/1055] 10.222.0.13 () {58 vars in 1024 bytes} [Wed Aug 31 09:35:40 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 09:35:40.967579 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c81fe849-d0d1-4a16-9e6e-e07238a3c8d8 None None] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:35:40.968941 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68adb8d1-8592-440d-9949-f4a719365d6a None None] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:40.973786 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68adb8d1-8592-440d-9949-f4a719365d6a None None] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:40.973786 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68adb8d1-8592-440d-9949-f4a719365d6a None None] PATH_INFO: `/` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:40.974177 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 530/1056] 10.222.0.13 () {54 vars in 927 bytes} [Wed Aug 31 09:35:40 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 31 09:35:40.983116 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3072b7a-0497-471d-a325-be25f3af6be1 None None] REQUEST_METHOD: `POST` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:40.984000 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3072b7a-0497-471d-a325-be25f3af6be1 None None] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:40.984000 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3072b7a-0497-471d-a325-be25f3af6be1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:40.988229 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=100 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:35:41.009076 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.auth.core [None req-a3072b7a-0497-471d-a325-be25f3af6be1 None None] MFA Rules not processed for user `fd836cf590a145df9acf19cf3ae2ac56`. Rule list: `[]` (Enabled: `True`). {{(pid=88113) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:35:41.010863 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.fernet_utils [None req-a3072b7a-0497-471d-a325-be25f3af6be1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:35:41.016037 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac61152b-7345-48d9-9b8b-7a7a05830fa1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:35:41.016675 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac61152b-7345-48d9-9b8b-7a7a05830fa1 None admin] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:41.017026 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac61152b-7345-48d9-9b8b-7a7a05830fa1 None admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:41.017294 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac61152b-7345-48d9-9b8b-7a7a05830fa1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:41.133743 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 531/1057] 10.222.0.13 () {58 vars in 1024 bytes} [Wed Aug 31 09:35:40 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 09:35:41.143328 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac61152b-7345-48d9-9b8b-7a7a05830fa1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:35:41.146098 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac61152b-7345-48d9-9b8b-7a7a05830fa1 None admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:35:41.163938 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3072b7a-0497-471d-a325-be25f3af6be1 None None] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:35:41.254040 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=100 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:35:41.297746 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 527/1058] 10.222.0.13 () {58 vars in 1282 bytes} [Wed Aug 31 09:35:40 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 543 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 31 09:35:41.303503 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac61152b-7345-48d9-9b8b-7a7a05830fa1 None admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:35:41.321306 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=100 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:35:41.354286 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3072b7a-0497-471d-a325-be25f3af6be1 None None] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:35:41.419777 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a2956a1-1bf6-41de-ad6d-6efbb04fe80d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:35:41.420848 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2956a1-1bf6-41de-ad6d-6efbb04fe80d None admin] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:41.421317 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2956a1-1bf6-41de-ad6d-6efbb04fe80d None admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:41.421808 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2956a1-1bf6-41de-ad6d-6efbb04fe80d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:41.439202 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac61152b-7345-48d9-9b8b-7a7a05830fa1 None admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:35:41.441220 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a2956a1-1bf6-41de-ad6d-6efbb04fe80d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 09:35:41.451228 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application [None req-9a2956a1-1bf6-41de-ad6d-6efbb04fe80d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 09:35:41.451228 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:35:41.451228 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:35:41.451228 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:35:41.451228 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:35:41.451228 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:35:41.451228 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:35:41.451228 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:35:41.451228 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:35:41.451228 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:35:41.451228 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:35:41.451228 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:35:41.451228 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 09:35:41.451228 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 09:35:41.451228 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 09:35:41.451228 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 09:35:41.452685 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 09:35:41.452685 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 09:35:41.452685 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 09:35:41.452685 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 09:35:41.452685 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:35:41.452685 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:35:41.452685 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:35:41.452685 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:35:41.452685 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:35:41.452685 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:35:41.452685 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:35:41.452685 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application with Lock( Aug 31 09:35:41.452685 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:35:41.452685 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:35:41.452685 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:35:41.452685 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:35:41.454055 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:35:41.454055 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:35:41.454055 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:35:41.454055 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:35:41.454055 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 09:35:41.454055 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 09:35:41.454055 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 09:35:41.454055 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 09:35:41.454055 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 09:35:41.454055 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 09:35:41.454055 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 09:35:41.454055 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application Aug 31 09:35:41.454055 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 532/1059] 10.222.0.13 () {56 vars in 1096 bytes} [Wed Aug 31 09:35:41 2022] GET /identity/v3/roles/admin => generated 83 bytes in 290 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 09:35:41.456479 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a2956a1-1bf6-41de-ad6d-6efbb04fe80d None admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:35:41.486689 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-774fe52e-e3bb-403e-9e31-a4a6322a2fc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:35:41.487418 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774fe52e-e3bb-403e-9e31-a4a6322a2fc7 None admin] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:41.487588 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774fe52e-e3bb-403e-9e31-a4a6322a2fc7 None admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:41.487844 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774fe52e-e3bb-403e-9e31-a4a6322a2fc7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:41.499149 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-774fe52e-e3bb-403e-9e31-a4a6322a2fc7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:35:41.500615 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-774fe52e-e3bb-403e-9e31-a4a6322a2fc7 None admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:35:41.521496 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application [None req-774fe52e-e3bb-403e-9e31-a4a6322a2fc7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 09:35:41.521496 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:35:41.521496 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:35:41.521496 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:35:41.521496 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:35:41.521496 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:35:41.521496 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:35:41.521496 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:35:41.521496 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:35:41.521496 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:35:41.521496 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:35:41.521496 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:35:41.521496 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 09:35:41.521496 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 09:35:41.521496 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 09:35:41.521496 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 09:35:41.522643 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:35:41.522643 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:35:41.522643 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:35:41.522643 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:35:41.522643 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:35:41.522643 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:35:41.522643 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:35:41.522643 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application with Lock( Aug 31 09:35:41.522643 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:35:41.522643 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:35:41.522643 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:35:41.522643 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:35:41.522643 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:35:41.522643 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:35:41.522643 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:35:41.522643 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:35:41.522643 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 09:35:41.523753 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 09:35:41.523753 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 09:35:41.523753 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application Aug 31 09:35:41.523753 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 528/1060] 10.222.0.13 () {56 vars in 1108 bytes} [Wed Aug 31 09:35:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 09:35:41.524946 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-774fe52e-e3bb-403e-9e31-a4a6322a2fc7 None admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:35:41.592079 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a2956a1-1bf6-41de-ad6d-6efbb04fe80d None admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:35:41.602990 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=100 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:35:41.614987 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=100 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:35:41.633938 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-379ff34d-ddec-4c42-898d-3ac0a3a3a9f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:35:41.634729 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379ff34d-ddec-4c42-898d-3ac0a3a3a9f8 None admin] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:41.635114 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379ff34d-ddec-4c42-898d-3ac0a3a3a9f8 None admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:41.635501 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379ff34d-ddec-4c42-898d-3ac0a3a3a9f8 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:41.648383 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-379ff34d-ddec-4c42-898d-3ac0a3a3a9f8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 09:35:41.654521 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application [None req-379ff34d-ddec-4c42-898d-3ac0a3a3a9f8 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 31 09:35:41.654521 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:35:41.654521 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:35:41.654521 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:35:41.654521 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:35:41.654521 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:35:41.654521 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:35:41.654521 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:35:41.654521 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:35:41.654521 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:35:41.654521 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:35:41.654521 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:35:41.654521 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 09:35:41.654521 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 09:35:41.654521 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 09:35:41.654521 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 09:35:41.656713 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 09:35:41.656713 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 09:35:41.656713 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 09:35:41.656713 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 09:35:41.656713 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:35:41.656713 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:35:41.656713 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:35:41.656713 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:35:41.656713 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:35:41.656713 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:35:41.656713 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:35:41.656713 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application with Lock( Aug 31 09:35:41.656713 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:35:41.656713 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:35:41.656713 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:35:41.656713 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:35:41.658569 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:35:41.658569 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:35:41.658569 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:35:41.658569 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:35:41.658569 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 09:35:41.658569 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 09:35:41.658569 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 09:35:41.658569 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 09:35:41.658569 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 09:35:41.658569 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 09:35:41.658569 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 31 09:35:41.658569 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application Aug 31 09:35:41.658569 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 533/1061] 10.222.0.13 () {56 vars in 1120 bytes} [Wed Aug 31 09:35:41 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:35:41.658569 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-379ff34d-ddec-4c42-898d-3ac0a3a3a9f8 None admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:35:41.667356 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-774fe52e-e3bb-403e-9e31-a4a6322a2fc7 None admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:35:41.724879 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d6f548-f6bc-4e12-9d98-f6e4606e4b21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:35:41.725890 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d6f548-f6bc-4e12-9d98-f6e4606e4b21 None admin] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:41.726343 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d6f548-f6bc-4e12-9d98-f6e4606e4b21 None admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:41.726788 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d6f548-f6bc-4e12-9d98-f6e4606e4b21 None admin] PATH_INFO: `/v3/roles` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:41.727813 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93d6f548-f6bc-4e12-9d98-f6e4606e4b21 None admin] RBAC: Adding query filter params (name=admin) {{(pid=88114) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:35:41.728275 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93d6f548-f6bc-4e12-9d98-f6e4606e4b21 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:35:41.730657 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93d6f548-f6bc-4e12-9d98-f6e4606e4b21 None admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:35:41.741358 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 529/1062] 10.222.0.13 () {56 vars in 1099 bytes} [Wed Aug 31 09:35:41 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:35:41.742248 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d6f548-f6bc-4e12-9d98-f6e4606e4b21 None admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:35:41.834676 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-379ff34d-ddec-4c42-898d-3ac0a3a3a9f8 None admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:35:41.869436 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d6f548-f6bc-4e12-9d98-f6e4606e4b21 None admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:35:41.890046 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a39b4760-c479-49f3-a679-2f73a57d2d97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:35:41.890677 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39b4760-c479-49f3-a679-2f73a57d2d97 None admin] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:41.890955 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39b4760-c479-49f3-a679-2f73a57d2d97 None admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:41.891219 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39b4760-c479-49f3-a679-2f73a57d2d97 None admin] PATH_INFO: `/v3/domains` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:41.891891 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a39b4760-c479-49f3-a679-2f73a57d2d97 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88113) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:35:41.892193 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a39b4760-c479-49f3-a679-2f73a57d2d97 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:35:41.893904 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a39b4760-c479-49f3-a679-2f73a57d2d97 None admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:35:41.900870 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=100 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:35:41.911862 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 534/1063] 10.222.0.13 () {56 vars in 1109 bytes} [Wed Aug 31 09:35:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:35:41.913698 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a39b4760-c479-49f3-a679-2f73a57d2d97 None admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:35:41.927272 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b598c436-0c9b-48df-89e0-5c179286fa00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:35:41.928139 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b598c436-0c9b-48df-89e0-5c179286fa00 None admin] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:41.928571 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b598c436-0c9b-48df-89e0-5c179286fa00 None admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:41.929720 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b598c436-0c9b-48df-89e0-5c179286fa00 None admin] PATH_INFO: `/v3/roles` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:41.930665 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b598c436-0c9b-48df-89e0-5c179286fa00 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=88114) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:35:41.931107 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b598c436-0c9b-48df-89e0-5c179286fa00 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:35:41.933373 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b598c436-0c9b-48df-89e0-5c179286fa00 None admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:35:41.949900 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 530/1064] 10.222.0.13 () {56 vars in 1115 bytes} [Wed Aug 31 09:35:41 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:35:41.953824 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b598c436-0c9b-48df-89e0-5c179286fa00 None admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:35:41.967781 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=100 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:35:42.076590 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a39b4760-c479-49f3-a679-2f73a57d2d97 None admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:35:42.123907 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f7d2548-49cb-4f29-a4d3-6128ea894238 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:35:42.124478 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7d2548-49cb-4f29-a4d3-6128ea894238 None admin] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:42.124757 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7d2548-49cb-4f29-a4d3-6128ea894238 None admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:42.125028 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7d2548-49cb-4f29-a4d3-6128ea894238 None admin] PATH_INFO: `/v3/users/b2548421988c4c268e5bf177b290c1bf` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:42.146093 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f7d2548-49cb-4f29-a4d3-6128ea894238 None admin] RBAC: Authorizing `identity:get_user(user_id=b2548421988c4c268e5bf177b290c1bf)` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:35:42.147946 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f7d2548-49cb-4f29-a4d3-6128ea894238 None admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:35:42.150059 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 535/1065] 10.222.0.13 () {56 vars in 1177 bytes} [Wed Aug 31 09:35:41 2022] GET /identity/v3/users/b2548421988c4c268e5bf177b290c1bf => generated 285 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:35:42.158856 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f7d2548-49cb-4f29-a4d3-6128ea894238 None admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:35:42.160317 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b598c436-0c9b-48df-89e0-5c179286fa00 None admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:35:42.217192 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5336c001-45f5-499d-a47a-90e0b7109d35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:35:42.217876 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5336c001-45f5-499d-a47a-90e0b7109d35 None admin] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:42.218154 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5336c001-45f5-499d-a47a-90e0b7109d35 None admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:42.218425 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5336c001-45f5-499d-a47a-90e0b7109d35 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:42.231495 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5336c001-45f5-499d-a47a-90e0b7109d35 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:35:42.233101 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5336c001-45f5-499d-a47a-90e0b7109d35 None admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:35:42.256437 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application [None req-5336c001-45f5-499d-a47a-90e0b7109d35 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 09:35:42.256437 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:35:42.256437 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:35:42.256437 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:35:42.256437 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:35:42.256437 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:35:42.256437 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:35:42.256437 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:35:42.256437 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:35:42.256437 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:35:42.256437 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:35:42.256437 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:35:42.256437 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 09:35:42.256437 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 09:35:42.256437 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 09:35:42.256437 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 09:35:42.257502 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:35:42.257502 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:35:42.257502 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:35:42.257502 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:35:42.257502 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:35:42.257502 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:35:42.257502 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:35:42.257502 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application with Lock( Aug 31 09:35:42.257502 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:35:42.257502 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:35:42.257502 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:35:42.257502 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:35:42.257502 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:35:42.257502 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:35:42.257502 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:35:42.257502 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:35:42.257502 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 09:35:42.258431 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 09:35:42.258431 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 09:35:42.258431 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application Aug 31 09:35:42.261734 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 531/1066] 10.222.0.13 () {56 vars in 1108 bytes} [Wed Aug 31 09:35:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 307 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:35:42.266086 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5336c001-45f5-499d-a47a-90e0b7109d35 None admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:35:42.319238 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=100 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:35:42.329787 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=100 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:35:42.339085 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f7d2548-49cb-4f29-a4d3-6128ea894238 None admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:35:42.383342 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-669b8510-4de6-42b6-8f77-0c0196b8071a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:35:42.384133 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669b8510-4de6-42b6-8f77-0c0196b8071a None admin] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:42.384503 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669b8510-4de6-42b6-8f77-0c0196b8071a None admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:42.384906 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669b8510-4de6-42b6-8f77-0c0196b8071a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:42.413076 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5336c001-45f5-499d-a47a-90e0b7109d35 None admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:35:42.455236 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e3a0789-4b81-4063-9887-1819a387eca7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:35:42.456277 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3a0789-4b81-4063-9887-1819a387eca7 None admin] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:42.456736 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3a0789-4b81-4063-9887-1819a387eca7 None admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:42.457236 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3a0789-4b81-4063-9887-1819a387eca7 None admin] PATH_INFO: `/v3/domains` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:42.458395 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e3a0789-4b81-4063-9887-1819a387eca7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88114) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:35:42.458916 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e3a0789-4b81-4063-9887-1819a387eca7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:35:42.461439 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e3a0789-4b81-4063-9887-1819a387eca7 None admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:35:42.478175 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 532/1067] 10.222.0.13 () {56 vars in 1109 bytes} [Wed Aug 31 09:35:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:35:42.478580 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-669b8510-4de6-42b6-8f77-0c0196b8071a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:35:42.482653 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-669b8510-4de6-42b6-8f77-0c0196b8071a None admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:35:42.484982 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e3a0789-4b81-4063-9887-1819a387eca7 None admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:35:42.595087 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=100 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:35:42.604878 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=100 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:35:42.611735 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e3a0789-4b81-4063-9887-1819a387eca7 None admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:35:42.615792 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 536/1068] 10.222.0.13 () {58 vars in 1282 bytes} [Wed Aug 31 09:35:42 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 459 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 31 09:35:42.622178 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-669b8510-4de6-42b6-8f77-0c0196b8071a None admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:35:42.671618 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9124be6-81e2-498e-af1c-f0cbaff9b93a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:35:42.672514 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9124be6-81e2-498e-af1c-f0cbaff9b93a None admin] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:42.672907 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9124be6-81e2-498e-af1c-f0cbaff9b93a None admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:42.673342 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9124be6-81e2-498e-af1c-f0cbaff9b93a None admin] PATH_INFO: `/v3/users/nova` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:42.686473 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9124be6-81e2-498e-af1c-f0cbaff9b93a None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:35:42.688747 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9124be6-81e2-498e-af1c-f0cbaff9b93a None admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:35:42.709454 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application [None req-a9124be6-81e2-498e-af1c-f0cbaff9b93a None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 31 09:35:42.709454 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:35:42.709454 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:35:42.709454 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:35:42.709454 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:35:42.709454 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:35:42.709454 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:35:42.709454 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:35:42.709454 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:35:42.709454 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:35:42.709454 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:35:42.709454 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:35:42.709454 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 31 09:35:42.709454 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 31 09:35:42.709454 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 31 09:35:42.709454 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 31 09:35:42.711374 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:35:42.711374 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:35:42.711374 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 31 09:35:42.711374 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 09:35:42.711374 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 31 09:35:42.711374 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 09:35:42.711374 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:35:42.711374 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:35:42.711374 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:35:42.711374 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:35:42.711374 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:35:42.711374 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application with Lock( Aug 31 09:35:42.711374 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:35:42.711374 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:35:42.711374 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:35:42.711374 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:35:42.711374 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:35:42.711374 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:35:42.712464 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:35:42.712464 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:35:42.712464 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 31 09:35:42.712464 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 31 09:35:42.712464 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 31 09:35:42.712464 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 31 09:35:42.712464 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 31 09:35:42.712464 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 31 09:35:42.712464 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 31 09:35:42.712464 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application Aug 31 09:35:42.712464 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 533/1069] 10.222.0.13 () {56 vars in 1093 bytes} [Wed Aug 31 09:35:42 2022] GET /identity/v3/users/nova => generated 82 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 09:35:42.717984 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9124be6-81e2-498e-af1c-f0cbaff9b93a None admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:35:42.755922 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-669b8510-4de6-42b6-8f77-0c0196b8071a None admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:35:42.799848 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ddff04-79cc-4cf6-bab1-6e6b4164f558 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:35:42.801412 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ddff04-79cc-4cf6-bab1-6e6b4164f558 None admin] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:42.801412 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ddff04-79cc-4cf6-bab1-6e6b4164f558 None admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:42.801922 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ddff04-79cc-4cf6-bab1-6e6b4164f558 None admin] PATH_INFO: `/v3/projects/0f66b24ed33240238e962a5335cdab1c` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:42.803933 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7ddff04-79cc-4cf6-bab1-6e6b4164f558 None admin] RBAC: Authorizing `identity:get_project(project_id=0f66b24ed33240238e962a5335cdab1c)` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:35:42.806659 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7ddff04-79cc-4cf6-bab1-6e6b4164f558 None admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:35:42.815799 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 537/1070] 10.222.0.13 () {56 vars in 1186 bytes} [Wed Aug 31 09:35:42 2022] GET /identity/v3/projects/0f66b24ed33240238e962a5335cdab1c => generated 311 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 09:35:42.817751 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ddff04-79cc-4cf6-bab1-6e6b4164f558 None admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:35:42.838936 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9124be6-81e2-498e-af1c-f0cbaff9b93a None admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:35:42.880966 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71cc7c14-132b-4647-b084-1ac535e6593b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:35:42.881633 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71cc7c14-132b-4647-b084-1ac535e6593b None admin] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:42.881910 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71cc7c14-132b-4647-b084-1ac535e6593b None admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:42.882183 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71cc7c14-132b-4647-b084-1ac535e6593b None admin] PATH_INFO: `/v3/users` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:42.882920 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71cc7c14-132b-4647-b084-1ac535e6593b None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=88114) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:35:42.883230 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71cc7c14-132b-4647-b084-1ac535e6593b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:35:42.884667 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71cc7c14-132b-4647-b084-1ac535e6593b None admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:35:42.904819 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 534/1071] 10.222.0.13 () {56 vars in 1133 bytes} [Wed Aug 31 09:35:42 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 09:35:42.910723 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71cc7c14-132b-4647-b084-1ac535e6593b None admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:35:42.915790 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=100 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:35:42.932712 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=100 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:35:43.000965 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ddff04-79cc-4cf6-bab1-6e6b4164f558 None admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:35:43.027256 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71cc7c14-132b-4647-b084-1ac535e6593b None admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:35:43.044262 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e7b0d57-ce6d-4297-90c9-8972bfe1b5cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:35:43.045006 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7b0d57-ce6d-4297-90c9-8972bfe1b5cc None admin] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:43.045497 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7b0d57-ce6d-4297-90c9-8972bfe1b5cc None admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:43.045913 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7b0d57-ce6d-4297-90c9-8972bfe1b5cc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:43.046739 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e7b0d57-ce6d-4297-90c9-8972bfe1b5cc None admin] RBAC: Adding query filter params (role.id=5d4ee63a3add47fc9a6b8b16d39e0a1f, scope.project.id=0f66b24ed33240238e962a5335cdab1c, user.id=b2548421988c4c268e5bf177b290c1bf) {{(pid=88113) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:35:43.047124 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e7b0d57-ce6d-4297-90c9-8972bfe1b5cc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:35:43.048692 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e7b0d57-ce6d-4297-90c9-8972bfe1b5cc None admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:35:43.055580 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 538/1072] 10.222.0.13 () {56 vars in 1374 bytes} [Wed Aug 31 09:35:42 2022] GET /identity/v3/role_assignments?user.id=b2548421988c4c268e5bf177b290c1bf&scope.project.id=0f66b24ed33240238e962a5335cdab1c&role.id=5d4ee63a3add47fc9a6b8b16d39e0a1f => generated 259 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:35:43.070064 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c3aa86c-f0a4-439b-b0b3-7dd66bc0aad6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:35:43.070729 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3aa86c-f0a4-439b-b0b3-7dd66bc0aad6 None admin] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:43.071007 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3aa86c-f0a4-439b-b0b3-7dd66bc0aad6 None admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:43.071278 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3aa86c-f0a4-439b-b0b3-7dd66bc0aad6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:43.085053 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c3aa86c-f0a4-439b-b0b3-7dd66bc0aad6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:35:43.086633 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c3aa86c-f0a4-439b-b0b3-7dd66bc0aad6 None admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:35:43.105287 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application [None req-4c3aa86c-f0a4-439b-b0b3-7dd66bc0aad6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 09:35:43.105287 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:35:43.105287 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:35:43.105287 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:35:43.105287 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:35:43.105287 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:35:43.105287 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:35:43.105287 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:35:43.105287 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:35:43.105287 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:35:43.105287 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:35:43.105287 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:35:43.105287 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 09:35:43.105287 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 09:35:43.105287 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 09:35:43.105287 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 09:35:43.106420 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:35:43.106420 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:35:43.106420 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:35:43.106420 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:35:43.106420 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:35:43.106420 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:35:43.106420 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:35:43.106420 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application with Lock( Aug 31 09:35:43.106420 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:35:43.106420 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:35:43.106420 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:35:43.106420 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:35:43.106420 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:35:43.106420 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:35:43.106420 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:35:43.106420 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:35:43.106420 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 09:35:43.107344 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 09:35:43.107344 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 09:35:43.107344 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application Aug 31 09:35:43.107344 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 535/1073] 10.222.0.13 () {56 vars in 1108 bytes} [Wed Aug 31 09:35:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 09:35:43.113738 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e7b0d57-ce6d-4297-90c9-8972bfe1b5cc None admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:35:43.245757 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e7b0d57-ce6d-4297-90c9-8972bfe1b5cc None admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:35:43.266339 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=100 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:35:43.287055 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b6418ea-9f8c-4654-856d-7ccb8f364f58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:35:43.287699 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6418ea-9f8c-4654-856d-7ccb8f364f58 None admin] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:43.287974 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6418ea-9f8c-4654-856d-7ccb8f364f58 None admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:43.288244 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6418ea-9f8c-4654-856d-7ccb8f364f58 None admin] PATH_INFO: `/v3/domains` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:43.288909 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b6418ea-9f8c-4654-856d-7ccb8f364f58 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88113) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:35:43.289209 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b6418ea-9f8c-4654-856d-7ccb8f364f58 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:35:43.290712 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b6418ea-9f8c-4654-856d-7ccb8f364f58 None admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:35:43.305279 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 539/1074] 10.222.0.13 () {56 vars in 1109 bytes} [Wed Aug 31 09:35:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 09:35:43.312072 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c3aa86c-f0a4-439b-b0b3-7dd66bc0aad6 None admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:35:43.445484 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=100 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:35:43.483479 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c3aa86c-f0a4-439b-b0b3-7dd66bc0aad6 None admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:35:43.522549 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c086486f-7af1-4aea-a1a0-fcf8babaa79d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:35:43.523097 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c086486f-7af1-4aea-a1a0-fcf8babaa79d None admin] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:43.523368 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c086486f-7af1-4aea-a1a0-fcf8babaa79d None admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:43.523637 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c086486f-7af1-4aea-a1a0-fcf8babaa79d None admin] PATH_INFO: `/v3/projects/service` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:43.532074 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c086486f-7af1-4aea-a1a0-fcf8babaa79d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:35:43.533570 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c086486f-7af1-4aea-a1a0-fcf8babaa79d None admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:35:43.546179 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application [None req-c086486f-7af1-4aea-a1a0-fcf8babaa79d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 09:35:43.546179 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:35:43.546179 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:35:43.546179 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:35:43.546179 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:35:43.546179 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:35:43.546179 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:35:43.546179 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:35:43.546179 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:35:43.546179 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:35:43.546179 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:35:43.546179 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:35:43.546179 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 31 09:35:43.546179 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 31 09:35:43.546179 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 31 09:35:43.547997 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 31 09:35:43.547997 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:35:43.547997 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:35:43.547997 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:35:43.547997 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:35:43.547997 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:35:43.547997 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:35:43.547997 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:35:43.547997 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application with Lock( Aug 31 09:35:43.547997 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:35:43.547997 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:35:43.547997 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:35:43.547997 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:35:43.547997 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:35:43.547997 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:35:43.547997 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:35:43.547997 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:35:43.549225 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 31 09:35:43.549225 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 31 09:35:43.549225 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 31 09:35:43.549225 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 31 09:35:43.549225 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 31 09:35:43.549225 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 31 09:35:43.549225 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 09:35:43.549225 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application Aug 31 09:35:43.549225 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 536/1075] 10.222.0.13 () {56 vars in 1111 bytes} [Wed Aug 31 09:35:43 2022] GET /identity/v3/projects/service => generated 88 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:35:43.553611 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b6418ea-9f8c-4654-856d-7ccb8f364f58 None admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:35:43.667167 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b6418ea-9f8c-4654-856d-7ccb8f364f58 None admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:35:43.706641 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a3bec2b-b308-4571-b542-922f48a9d59f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:35:43.707252 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3bec2b-b308-4571-b542-922f48a9d59f None admin] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:43.707523 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3bec2b-b308-4571-b542-922f48a9d59f None admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:43.707788 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3bec2b-b308-4571-b542-922f48a9d59f None admin] PATH_INFO: `/v3/projects` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:43.708485 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a3bec2b-b308-4571-b542-922f48a9d59f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88113) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:35:43.708783 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a3bec2b-b308-4571-b542-922f48a9d59f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:35:43.710251 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a3bec2b-b308-4571-b542-922f48a9d59f None admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:35:43.724625 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 540/1076] 10.222.0.13 () {56 vars in 1148 bytes} [Wed Aug 31 09:35:43 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 09:35:43.731764 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c086486f-7af1-4aea-a1a0-fcf8babaa79d None admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:35:43.897648 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c086486f-7af1-4aea-a1a0-fcf8babaa79d None admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:35:43.932542 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=100 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:35:43.940817 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f63c7f2-9b9e-4544-8093-e535b1472193 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:35:43.941580 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f63c7f2-9b9e-4544-8093-e535b1472193 None admin] REQUEST_METHOD: `PUT` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:43.941878 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f63c7f2-9b9e-4544-8093-e535b1472193 None admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:43.942167 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f63c7f2-9b9e-4544-8093-e535b1472193 None admin] PATH_INFO: `/v3/projects/49548eb44d62415582af820ab66decee/users/dc076493239d4ccdbdc37c0369c49529/roles/d41c162f8e1748989a09382ced5ef3a6` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:43.944533 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f63c7f2-9b9e-4544-8093-e535b1472193 None admin] RBAC: Authorizing `identity:create_grant(project_id=49548eb44d62415582af820ab66decee, user_id=dc076493239d4ccdbdc37c0369c49529, role_id=d41c162f8e1748989a09382ced5ef3a6)` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:35:43.946242 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f63c7f2-9b9e-4544-8093-e535b1472193 None admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:35:43.963405 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 537/1077] 10.222.0.13 () {58 vars in 1439 bytes} [Wed Aug 31 09:35:43 2022] PUT /identity/v3/projects/49548eb44d62415582af820ab66decee/users/dc076493239d4ccdbdc37c0369c49529/roles/d41c162f8e1748989a09382ced5ef3a6 => generated 0 bytes in 233 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:35:45.263330 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21101162-c1a5-4e33-8ff6-efcde9481310 None None] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:45.264110 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21101162-c1a5-4e33-8ff6-efcde9481310 None None] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:45.264466 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21101162-c1a5-4e33-8ff6-efcde9481310 None None] PATH_INFO: `/` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:45.267241 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 541/1078] 10.222.0.13 () {54 vars in 927 bytes} [Wed Aug 31 09:35:45 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:35:45.276392 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837fcfc5-8676-4067-a36f-1dbac40fa315 None None] REQUEST_METHOD: `POST` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:45.276962 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837fcfc5-8676-4067-a36f-1dbac40fa315 None None] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:45.277263 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837fcfc5-8676-4067-a36f-1dbac40fa315 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:45.308888 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.auth.core [None req-837fcfc5-8676-4067-a36f-1dbac40fa315 None None] MFA Rules not processed for user `fd836cf590a145df9acf19cf3ae2ac56`. Rule list: `[]` (Enabled: `True`). {{(pid=88114) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:35:45.310956 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.fernet_utils [None req-837fcfc5-8676-4067-a36f-1dbac40fa315 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88114) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:35:45.436069 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 538/1079] 10.222.0.13 () {58 vars in 1024 bytes} [Wed Aug 31 09:35:45 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 09:35:45.708018 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21101162-c1a5-4e33-8ff6-efcde9481310 None None] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:35:45.781560 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=100 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:35:45.863228 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21101162-c1a5-4e33-8ff6-efcde9481310 None None] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:35:45.904005 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc87bbf7-c366-4782-9954-4f47021159b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:35:45.904659 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc87bbf7-c366-4782-9954-4f47021159b9 None admin] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:45.904935 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc87bbf7-c366-4782-9954-4f47021159b9 None admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:45.905205 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc87bbf7-c366-4782-9954-4f47021159b9 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:45.917246 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc87bbf7-c366-4782-9954-4f47021159b9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 09:35:45.922382 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application [None req-cc87bbf7-c366-4782-9954-4f47021159b9 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 09:35:45.922382 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:35:45.922382 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:35:45.922382 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:35:45.922382 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:35:45.922382 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:35:45.922382 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:35:45.922382 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:35:45.922382 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:35:45.922382 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:35:45.922382 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:35:45.922382 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:35:45.922382 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 09:35:45.922382 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 09:35:45.922382 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 09:35:45.922382 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 09:35:45.924386 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 09:35:45.924386 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 09:35:45.924386 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 09:35:45.924386 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 09:35:45.924386 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:35:45.924386 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:35:45.924386 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:35:45.924386 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:35:45.924386 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:35:45.924386 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:35:45.924386 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:35:45.924386 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application with Lock( Aug 31 09:35:45.924386 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:35:45.924386 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:35:45.924386 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:35:45.924386 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:35:45.926072 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:35:45.926072 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:35:45.926072 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:35:45.926072 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:35:45.926072 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 09:35:45.926072 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 09:35:45.926072 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 09:35:45.926072 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 09:35:45.926072 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 09:35:45.926072 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 09:35:45.926072 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 09:35:45.926072 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application Aug 31 09:35:45.928675 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 542/1080] 10.222.0.13 () {56 vars in 1096 bytes} [Wed Aug 31 09:35:45 2022] GET /identity/v3/roles/admin => generated 83 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 09:35:45.929753 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-837fcfc5-8676-4067-a36f-1dbac40fa315 None None] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:35:46.045931 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca89f3f4-ddc8-43d6-a3bd-126c2cc7818f None None] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:46.046200 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca89f3f4-ddc8-43d6-a3bd-126c2cc7818f None None] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:46.046469 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca89f3f4-ddc8-43d6-a3bd-126c2cc7818f None None] PATH_INFO: `/` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:46.048407 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 543/1081] 10.222.0.13 () {54 vars in 927 bytes} [Wed Aug 31 09:35:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 31 09:35:46.054351 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c21db93-bcac-42cb-8c3a-37553df56a2b None None] REQUEST_METHOD: `POST` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:46.054647 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c21db93-bcac-42cb-8c3a-37553df56a2b None None] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:46.054921 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c21db93-bcac-42cb-8c3a-37553df56a2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:46.076833 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.auth.core [None req-4c21db93-bcac-42cb-8c3a-37553df56a2b None None] MFA Rules not processed for user `fd836cf590a145df9acf19cf3ae2ac56`. Rule list: `[]` (Enabled: `True`). {{(pid=88113) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:35:46.078088 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.fernet_utils [None req-4c21db93-bcac-42cb-8c3a-37553df56a2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:35:46.095409 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-837fcfc5-8676-4067-a36f-1dbac40fa315 None None] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:35:46.121901 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:35:46.150527 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1622873-14d0-4731-a63c-fc0332563d71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:35:46.151567 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1622873-14d0-4731-a63c-fc0332563d71 None admin] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:46.152060 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1622873-14d0-4731-a63c-fc0332563d71 None admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:46.152546 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1622873-14d0-4731-a63c-fc0332563d71 None admin] PATH_INFO: `/v3/roles` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:46.154247 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 544/1082] 10.222.0.13 () {58 vars in 1024 bytes} [Wed Aug 31 09:35:46 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 31 09:35:46.154433 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1622873-14d0-4731-a63c-fc0332563d71 None admin] RBAC: Adding query filter params (name=admin) {{(pid=88114) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:35:46.155015 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1622873-14d0-4731-a63c-fc0332563d71 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:35:46.157707 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1622873-14d0-4731-a63c-fc0332563d71 None admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:35:46.177934 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 539/1083] 10.222.0.13 () {56 vars in 1099 bytes} [Wed Aug 31 09:35:45 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 09:35:46.179236 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c21db93-bcac-42cb-8c3a-37553df56a2b None None] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:35:46.212231 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=100 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:35:46.297820 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c21db93-bcac-42cb-8c3a-37553df56a2b None None] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:35:46.352507 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a393a1b4-f842-492c-82cf-f55030a4fd03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:35:46.353638 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a393a1b4-f842-492c-82cf-f55030a4fd03 None admin] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:46.354139 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a393a1b4-f842-492c-82cf-f55030a4fd03 None admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:46.354623 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a393a1b4-f842-492c-82cf-f55030a4fd03 None admin] PATH_INFO: `/v3/users/b2548421988c4c268e5bf177b290c1bf` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:46.358246 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a393a1b4-f842-492c-82cf-f55030a4fd03 None admin] RBAC: Authorizing `identity:get_user(user_id=b2548421988c4c268e5bf177b290c1bf)` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:35:46.360979 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a393a1b4-f842-492c-82cf-f55030a4fd03 None admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:35:46.364085 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 545/1084] 10.222.0.13 () {56 vars in 1177 bytes} [Wed Aug 31 09:35:46 2022] GET /identity/v3/users/b2548421988c4c268e5bf177b290c1bf => generated 285 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 09:35:46.370400 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1622873-14d0-4731-a63c-fc0332563d71 None admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:35:46.441898 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a393a1b4-f842-492c-82cf-f55030a4fd03 None admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:35:46.534977 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1622873-14d0-4731-a63c-fc0332563d71 None admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:35:46.570304 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=100 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:35:46.577718 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68587a24-e7a0-4b5c-87d3-c712d29d8aa9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:35:46.578362 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68587a24-e7a0-4b5c-87d3-c712d29d8aa9 None admin] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:46.578641 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68587a24-e7a0-4b5c-87d3-c712d29d8aa9 None admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:46.578914 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68587a24-e7a0-4b5c-87d3-c712d29d8aa9 None admin] PATH_INFO: `/v3/projects/0f66b24ed33240238e962a5335cdab1c` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:46.580069 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68587a24-e7a0-4b5c-87d3-c712d29d8aa9 None admin] RBAC: Authorizing `identity:get_project(project_id=0f66b24ed33240238e962a5335cdab1c)` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:35:46.581540 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68587a24-e7a0-4b5c-87d3-c712d29d8aa9 None admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:35:46.581874 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a393a1b4-f842-492c-82cf-f55030a4fd03 None admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:35:46.583207 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 540/1085] 10.222.0.13 () {56 vars in 1186 bytes} [Wed Aug 31 09:35:46 2022] GET /identity/v3/projects/0f66b24ed33240238e962a5335cdab1c => generated 311 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:35:46.589275 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68587a24-e7a0-4b5c-87d3-c712d29d8aa9 None admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:35:46.620474 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c1f0cc-d68d-4eb2-93ff-1c0d29c89145 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:35:46.621117 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c1f0cc-d68d-4eb2-93ff-1c0d29c89145 None admin] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:46.621412 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c1f0cc-d68d-4eb2-93ff-1c0d29c89145 None admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:46.622397 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c1f0cc-d68d-4eb2-93ff-1c0d29c89145 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:46.633903 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3c1f0cc-d68d-4eb2-93ff-1c0d29c89145 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 09:35:46.639764 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application [None req-f3c1f0cc-d68d-4eb2-93ff-1c0d29c89145 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 31 09:35:46.639764 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:35:46.639764 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:35:46.639764 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:35:46.639764 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:35:46.639764 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:35:46.639764 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:35:46.639764 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:35:46.639764 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:35:46.639764 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:35:46.639764 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:35:46.639764 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:35:46.639764 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 09:35:46.639764 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 09:35:46.639764 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 09:35:46.639764 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 09:35:46.641893 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 09:35:46.641893 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 09:35:46.641893 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 09:35:46.641893 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 09:35:46.641893 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:35:46.641893 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:35:46.641893 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:35:46.641893 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:35:46.641893 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:35:46.641893 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:35:46.641893 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:35:46.641893 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application with Lock( Aug 31 09:35:46.641893 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:35:46.641893 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:35:46.641893 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:35:46.641893 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:35:46.644081 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:35:46.644081 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:35:46.644081 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:35:46.644081 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:35:46.644081 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 09:35:46.644081 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 09:35:46.644081 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 09:35:46.644081 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 09:35:46.644081 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 09:35:46.644081 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 09:35:46.644081 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 31 09:35:46.644081 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application Aug 31 09:35:46.644081 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 546/1086] 10.222.0.13 () {56 vars in 1120 bytes} [Wed Aug 31 09:35:46 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 09:35:46.647399 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=100 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:35:46.649374 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c1f0cc-d68d-4eb2-93ff-1c0d29c89145 None admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:35:46.720404 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68587a24-e7a0-4b5c-87d3-c712d29d8aa9 None admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:35:46.758633 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-906b2909-ca93-4197-9db1-27484c51d666 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:35:46.759473 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906b2909-ca93-4197-9db1-27484c51d666 None admin] REQUEST_METHOD: `PUT` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:46.759816 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906b2909-ca93-4197-9db1-27484c51d666 None admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:46.760200 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906b2909-ca93-4197-9db1-27484c51d666 None admin] PATH_INFO: `/v3/projects/0f66b24ed33240238e962a5335cdab1c/users/b2548421988c4c268e5bf177b290c1bf/roles/5d4ee63a3add47fc9a6b8b16d39e0a1f` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:46.763189 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-906b2909-ca93-4197-9db1-27484c51d666 None admin] RBAC: Authorizing `identity:create_grant(project_id=0f66b24ed33240238e962a5335cdab1c, user_id=b2548421988c4c268e5bf177b290c1bf, role_id=5d4ee63a3add47fc9a6b8b16d39e0a1f)` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:35:46.764896 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-906b2909-ca93-4197-9db1-27484c51d666 None admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:35:46.774454 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c1f0cc-d68d-4eb2-93ff-1c0d29c89145 None admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:35:46.783442 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 541/1087] 10.222.0.13 () {58 vars in 1439 bytes} [Wed Aug 31 09:35:46 2022] PUT /identity/v3/projects/0f66b24ed33240238e962a5335cdab1c/users/b2548421988c4c268e5bf177b290c1bf/roles/5d4ee63a3add47fc9a6b8b16d39e0a1f => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 09:35:46.817500 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a72b3776-35a7-437a-a5b4-3f825451ce89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:35:46.818140 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72b3776-35a7-437a-a5b4-3f825451ce89 None admin] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:46.818420 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72b3776-35a7-437a-a5b4-3f825451ce89 None admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:46.818714 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72b3776-35a7-437a-a5b4-3f825451ce89 None admin] PATH_INFO: `/v3/roles` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:46.819388 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a72b3776-35a7-437a-a5b4-3f825451ce89 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=88113) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:35:46.819694 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a72b3776-35a7-437a-a5b4-3f825451ce89 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:35:46.821206 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a72b3776-35a7-437a-a5b4-3f825451ce89 None admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:35:46.839147 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 547/1088] 10.222.0.13 () {56 vars in 1115 bytes} [Wed Aug 31 09:35:46 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 09:35:46.842045 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-906b2909-ca93-4197-9db1-27484c51d666 None admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:35:47.006594 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-906b2909-ca93-4197-9db1-27484c51d666 None admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:35:47.029948 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:35:47.079370 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed637a7-4b39-4328-af8a-afe0fe89af38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:35:47.080331 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed637a7-4b39-4328-af8a-afe0fe89af38 None admin] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:47.080810 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed637a7-4b39-4328-af8a-afe0fe89af38 None admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:47.081286 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed637a7-4b39-4328-af8a-afe0fe89af38 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:47.101096 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ed637a7-4b39-4328-af8a-afe0fe89af38 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:35:47.103812 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ed637a7-4b39-4328-af8a-afe0fe89af38 None admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:35:47.128992 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application [None req-0ed637a7-4b39-4328-af8a-afe0fe89af38 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 09:35:47.128992 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:35:47.128992 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:35:47.128992 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:35:47.128992 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:35:47.128992 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:35:47.128992 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:35:47.128992 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:35:47.128992 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:35:47.128992 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:35:47.128992 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:35:47.128992 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:35:47.128992 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 09:35:47.128992 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 09:35:47.128992 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 09:35:47.128992 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 09:35:47.130984 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:35:47.130984 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:35:47.130984 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:35:47.130984 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:35:47.130984 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:35:47.130984 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:35:47.130984 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:35:47.130984 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application with Lock( Aug 31 09:35:47.130984 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:35:47.130984 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:35:47.130984 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:35:47.130984 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:35:47.130984 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:35:47.130984 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:35:47.130984 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:35:47.130984 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:35:47.130984 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 09:35:47.132767 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 09:35:47.132767 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 09:35:47.132767 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application Aug 31 09:35:47.132767 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 542/1089] 10.222.0.13 () {56 vars in 1108 bytes} [Wed Aug 31 09:35:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 290 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:35:47.144817 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a72b3776-35a7-437a-a5b4-3f825451ce89 None admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:35:47.168479 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=100 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:35:47.266929 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a72b3776-35a7-437a-a5b4-3f825451ce89 None admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:35:47.305851 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-206ad9f6-f7bd-4fcb-80e3-e80c03268bec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:35:47.306867 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206ad9f6-f7bd-4fcb-80e3-e80c03268bec None admin] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:47.306867 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206ad9f6-f7bd-4fcb-80e3-e80c03268bec None admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:47.307187 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206ad9f6-f7bd-4fcb-80e3-e80c03268bec None admin] PATH_INFO: `/v3/domains` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:47.307643 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-206ad9f6-f7bd-4fcb-80e3-e80c03268bec None admin] RBAC: Adding query filter params (name=Default) {{(pid=88113) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:35:47.307948 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-206ad9f6-f7bd-4fcb-80e3-e80c03268bec None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:35:47.310049 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-206ad9f6-f7bd-4fcb-80e3-e80c03268bec None admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:35:47.323666 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 548/1090] 10.222.0.13 () {56 vars in 1109 bytes} [Wed Aug 31 09:35:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:35:47.330680 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed637a7-4b39-4328-af8a-afe0fe89af38 None admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:35:47.491298 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed637a7-4b39-4328-af8a-afe0fe89af38 None admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:35:47.532571 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b2a0e7f-850a-4284-81e2-b26c4d024b3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:35:47.533350 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2a0e7f-850a-4284-81e2-b26c4d024b3c None admin] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:47.533767 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2a0e7f-850a-4284-81e2-b26c4d024b3c None admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:47.534356 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2a0e7f-850a-4284-81e2-b26c4d024b3c None admin] PATH_INFO: `/v3/users/nova` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:47.543435 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b2a0e7f-850a-4284-81e2-b26c4d024b3c None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:35:47.545046 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b2a0e7f-850a-4284-81e2-b26c4d024b3c None admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:35:47.558337 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application [None req-5b2a0e7f-850a-4284-81e2-b26c4d024b3c None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 31 09:35:47.558337 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:35:47.558337 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:35:47.558337 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:35:47.558337 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:35:47.558337 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:35:47.558337 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:35:47.558337 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:35:47.558337 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:35:47.558337 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:35:47.558337 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:35:47.558337 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:35:47.558337 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 31 09:35:47.558337 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 31 09:35:47.558337 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 31 09:35:47.558337 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 31 09:35:47.560624 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:35:47.560624 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:35:47.560624 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 31 09:35:47.560624 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 09:35:47.560624 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 31 09:35:47.560624 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 09:35:47.560624 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:35:47.560624 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:35:47.560624 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:35:47.560624 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:35:47.560624 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:35:47.560624 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application with Lock( Aug 31 09:35:47.560624 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:35:47.560624 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:35:47.560624 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:35:47.560624 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:35:47.560624 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:35:47.560624 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:35:47.562652 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:35:47.562652 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:35:47.562652 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 31 09:35:47.562652 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 31 09:35:47.562652 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 31 09:35:47.562652 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 31 09:35:47.562652 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 31 09:35:47.562652 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 31 09:35:47.562652 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 31 09:35:47.562652 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application Aug 31 09:35:47.562652 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 543/1091] 10.222.0.13 () {56 vars in 1093 bytes} [Wed Aug 31 09:35:47 2022] GET /identity/v3/users/nova => generated 82 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:35:47.568880 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-206ad9f6-f7bd-4fcb-80e3-e80c03268bec None admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:35:47.666860 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=100 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:35:47.694921 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-206ad9f6-f7bd-4fcb-80e3-e80c03268bec None admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:35:47.738954 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52928325-e860-4c3b-a230-8479fd450e85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:35:47.739603 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52928325-e860-4c3b-a230-8479fd450e85 None admin] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:47.739880 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52928325-e860-4c3b-a230-8479fd450e85 None admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:47.740149 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52928325-e860-4c3b-a230-8479fd450e85 None admin] PATH_INFO: `/v3/users` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:47.740913 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52928325-e860-4c3b-a230-8479fd450e85 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=88113) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:35:47.741214 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52928325-e860-4c3b-a230-8479fd450e85 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:35:47.742734 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52928325-e860-4c3b-a230-8479fd450e85 None admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:35:47.763890 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 549/1092] 10.222.0.13 () {56 vars in 1133 bytes} [Wed Aug 31 09:35:47 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 09:35:47.770238 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b2a0e7f-850a-4284-81e2-b26c4d024b3c None admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:35:47.799054 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=100 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:35:47.909596 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b2a0e7f-850a-4284-81e2-b26c4d024b3c None admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:35:47.953142 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e93d8e-242d-485d-8c23-a9601f120ab2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:35:47.953867 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e93d8e-242d-485d-8c23-a9601f120ab2 None admin] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:47.954214 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e93d8e-242d-485d-8c23-a9601f120ab2 None admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:47.954553 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e93d8e-242d-485d-8c23-a9601f120ab2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:48.051518 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63e93d8e-242d-485d-8c23-a9601f120ab2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:35:48.054060 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63e93d8e-242d-485d-8c23-a9601f120ab2 None admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:35:48.096595 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=100 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:35:48.190558 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 544/1093] 10.222.0.13 () {58 vars in 1282 bytes} [Wed Aug 31 09:35:47 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 421 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 31 09:35:48.197212 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52928325-e860-4c3b-a230-8479fd450e85 None admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:35:48.316238 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52928325-e860-4c3b-a230-8479fd450e85 None admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:35:48.358022 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a7db07-8446-429c-a5d6-5a75a096bd01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:35:48.358742 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a7db07-8446-429c-a5d6-5a75a096bd01 None admin] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:48.359083 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a7db07-8446-429c-a5d6-5a75a096bd01 None admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:48.359435 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a7db07-8446-429c-a5d6-5a75a096bd01 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:48.362728 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=100 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:35:48.373880 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19a7db07-8446-429c-a5d6-5a75a096bd01 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:35:48.375577 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19a7db07-8446-429c-a5d6-5a75a096bd01 None admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:35:48.393879 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application [None req-19a7db07-8446-429c-a5d6-5a75a096bd01 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 09:35:48.393879 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:35:48.393879 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:35:48.393879 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:35:48.393879 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:35:48.393879 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:35:48.393879 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:35:48.393879 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:35:48.393879 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:35:48.393879 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:35:48.393879 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:35:48.393879 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:35:48.393879 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 09:35:48.393879 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 09:35:48.393879 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 09:35:48.393879 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 09:35:48.395747 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:35:48.395747 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:35:48.395747 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:35:48.395747 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:35:48.395747 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:35:48.395747 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:35:48.395747 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:35:48.395747 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application with Lock( Aug 31 09:35:48.395747 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:35:48.395747 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:35:48.395747 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:35:48.395747 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:35:48.395747 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:35:48.395747 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:35:48.395747 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:35:48.395747 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:35:48.395747 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 09:35:48.397484 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 09:35:48.397484 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 09:35:48.397484 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application Aug 31 09:35:48.397484 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 550/1094] 10.222.0.13 () {56 vars in 1108 bytes} [Wed Aug 31 09:35:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:35:48.404126 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e93d8e-242d-485d-8c23-a9601f120ab2 None admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:35:48.590616 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e93d8e-242d-485d-8c23-a9601f120ab2 None admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:35:48.632322 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=100 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:35:48.652281 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a44cb6-ae64-449d-b14e-54ba050ce3bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:35:48.653317 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a44cb6-ae64-449d-b14e-54ba050ce3bb None admin] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:48.653864 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a44cb6-ae64-449d-b14e-54ba050ce3bb None admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:48.654341 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a44cb6-ae64-449d-b14e-54ba050ce3bb None admin] PATH_INFO: `/v3/domains` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:48.655503 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71a44cb6-ae64-449d-b14e-54ba050ce3bb None admin] RBAC: Adding query filter params (name=Default) {{(pid=88114) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:35:48.656051 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71a44cb6-ae64-449d-b14e-54ba050ce3bb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:35:48.658636 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71a44cb6-ae64-449d-b14e-54ba050ce3bb None admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:35:48.681703 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 545/1095] 10.222.0.13 () {56 vars in 1109 bytes} [Wed Aug 31 09:35:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 279 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:35:48.692557 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a7db07-8446-429c-a5d6-5a75a096bd01 None admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:35:48.799660 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de3b9f5-a070-4c46-a4b5-6f8643ae056c None None] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:48.800189 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de3b9f5-a070-4c46-a4b5-6f8643ae056c None None] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:48.800623 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de3b9f5-a070-4c46-a4b5-6f8643ae056c None None] PATH_INFO: `/` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:48.802970 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 546/1096] 10.222.0.13 () {54 vars in 927 bytes} [Wed Aug 31 09:35:48 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 31 09:35:48.810044 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5edda8-137e-4055-bebe-ea7b0a0f2391 None None] REQUEST_METHOD: `POST` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:48.810488 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5edda8-137e-4055-bebe-ea7b0a0f2391 None None] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:48.810940 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5edda8-137e-4055-bebe-ea7b0a0f2391 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:48.820153 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a7db07-8446-429c-a5d6-5a75a096bd01 None admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:35:48.838968 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.auth.core [None req-6e5edda8-137e-4055-bebe-ea7b0a0f2391 None None] MFA Rules not processed for user `fd836cf590a145df9acf19cf3ae2ac56`. Rule list: `[]` (Enabled: `True`). {{(pid=88114) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:35:48.841743 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.fernet_utils [None req-6e5edda8-137e-4055-bebe-ea7b0a0f2391 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88114) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:35:48.878299 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b82341a-1dd0-477e-bb71-dd1bfd3e0c2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:35:48.879283 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b82341a-1dd0-477e-bb71-dd1bfd3e0c2a None admin] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:48.879656 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b82341a-1dd0-477e-bb71-dd1bfd3e0c2a None admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:48.880029 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b82341a-1dd0-477e-bb71-dd1bfd3e0c2a None admin] PATH_INFO: `/v3/projects/service` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:48.888653 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b82341a-1dd0-477e-bb71-dd1bfd3e0c2a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:35:48.890242 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b82341a-1dd0-477e-bb71-dd1bfd3e0c2a None admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:35:48.904989 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application [None req-0b82341a-1dd0-477e-bb71-dd1bfd3e0c2a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 09:35:48.904989 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:35:48.904989 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:35:48.904989 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:35:48.904989 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:35:48.904989 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:35:48.904989 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:35:48.904989 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:35:48.904989 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:35:48.904989 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:35:48.904989 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:35:48.904989 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:35:48.904989 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 31 09:35:48.904989 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 31 09:35:48.904989 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 31 09:35:48.907304 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 31 09:35:48.907304 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:35:48.907304 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:35:48.907304 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:35:48.907304 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:35:48.907304 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:35:48.907304 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:35:48.907304 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:35:48.907304 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application with Lock( Aug 31 09:35:48.907304 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:35:48.907304 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:35:48.907304 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:35:48.907304 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:35:48.907304 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:35:48.907304 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:35:48.907304 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:35:48.907304 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:35:48.909667 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 31 09:35:48.909667 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 31 09:35:48.909667 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 31 09:35:48.909667 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 31 09:35:48.909667 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 31 09:35:48.909667 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 31 09:35:48.909667 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 09:35:48.909667 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application Aug 31 09:35:48.909667 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 551/1097] 10.222.0.13 () {56 vars in 1111 bytes} [Wed Aug 31 09:35:48 2022] GET /identity/v3/projects/service => generated 88 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:35:48.913709 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b82341a-1dd0-477e-bb71-dd1bfd3e0c2a None admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:35:48.941767 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 547/1098] 10.222.0.13 () {58 vars in 1024 bytes} [Wed Aug 31 09:35:48 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 09:35:48.978433 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=100 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:35:49.060857 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b82341a-1dd0-477e-bb71-dd1bfd3e0c2a None admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:35:49.120756 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db7e437-b630-4bce-a176-e4abdbda078a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:35:49.121671 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db7e437-b630-4bce-a176-e4abdbda078a None admin] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:49.122170 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db7e437-b630-4bce-a176-e4abdbda078a None admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:49.122674 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db7e437-b630-4bce-a176-e4abdbda078a None admin] PATH_INFO: `/v3/projects` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:49.123876 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8db7e437-b630-4bce-a176-e4abdbda078a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88113) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:35:49.124377 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8db7e437-b630-4bce-a176-e4abdbda078a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:35:49.126966 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8db7e437-b630-4bce-a176-e4abdbda078a None admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:35:49.149059 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 552/1099] 10.222.0.13 () {56 vars in 1148 bytes} [Wed Aug 31 09:35:48 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 09:35:49.156392 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e5edda8-137e-4055-bebe-ea7b0a0f2391 None None] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:35:49.215641 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db7e437-b630-4bce-a176-e4abdbda078a None admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:35:49.275551 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e5edda8-137e-4055-bebe-ea7b0a0f2391 None None] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:35:49.291615 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=100 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:35:49.314679 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-929ad095-83c4-4f61-bc11-70f4594a5cbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:35:49.315296 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929ad095-83c4-4f61-bc11-70f4594a5cbe None admin] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:49.315577 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929ad095-83c4-4f61-bc11-70f4594a5cbe None admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:49.315842 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929ad095-83c4-4f61-bc11-70f4594a5cbe None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:49.316586 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-929ad095-83c4-4f61-bc11-70f4594a5cbe None admin] RBAC: Adding query filter params (role.id=d41c162f8e1748989a09382ced5ef3a6, scope.project.id=49548eb44d62415582af820ab66decee, user.id=dc076493239d4ccdbdc37c0369c49529) {{(pid=88114) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:35:49.316879 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-929ad095-83c4-4f61-bc11-70f4594a5cbe None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:35:49.318357 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-929ad095-83c4-4f61-bc11-70f4594a5cbe None admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:35:49.325357 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 548/1100] 10.222.0.13 () {56 vars in 1374 bytes} [Wed Aug 31 09:35:49 2022] GET /identity/v3/role_assignments?user.id=dc076493239d4ccdbdc37c0369c49529&scope.project.id=49548eb44d62415582af820ab66decee&role.id=d41c162f8e1748989a09382ced5ef3a6 => generated 608 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:35:49.352540 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db7e437-b630-4bce-a176-e4abdbda078a None admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:35:49.365047 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=100 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:35:49.397543 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c85c546-211d-4c20-92c8-d49042e2d7da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:35:49.398280 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c85c546-211d-4c20-92c8-d49042e2d7da None admin] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:49.398659 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c85c546-211d-4c20-92c8-d49042e2d7da None admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:49.399028 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c85c546-211d-4c20-92c8-d49042e2d7da None admin] PATH_INFO: `/v3/roles/admin` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:49.413226 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c85c546-211d-4c20-92c8-d49042e2d7da None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 09:35:49.418757 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application [None req-1c85c546-211d-4c20-92c8-d49042e2d7da None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 09:35:49.418757 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:35:49.418757 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:35:49.418757 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:35:49.418757 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:35:49.418757 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:35:49.418757 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:35:49.418757 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:35:49.418757 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:35:49.418757 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:35:49.418757 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:35:49.418757 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:35:49.418757 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 09:35:49.418757 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 09:35:49.418757 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 09:35:49.418757 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 09:35:49.420709 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 09:35:49.420709 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 09:35:49.420709 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 09:35:49.420709 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 09:35:49.420709 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:35:49.420709 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:35:49.420709 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:35:49.420709 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:35:49.420709 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:35:49.420709 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:35:49.420709 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:35:49.420709 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application with Lock( Aug 31 09:35:49.420709 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:35:49.420709 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:35:49.420709 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:35:49.420709 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:35:49.422363 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:35:49.422363 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:35:49.422363 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:35:49.422363 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:35:49.422363 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 09:35:49.422363 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 09:35:49.422363 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 09:35:49.422363 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 09:35:49.422363 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 09:35:49.422363 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 09:35:49.422363 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 09:35:49.422363 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application Aug 31 09:35:49.422363 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 553/1101] 10.222.0.13 () {56 vars in 1096 bytes} [Wed Aug 31 09:35:49 2022] GET /identity/v3/roles/admin => generated 83 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:35:49.428694 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-929ad095-83c4-4f61-bc11-70f4594a5cbe None admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:35:49.580697 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-929ad095-83c4-4f61-bc11-70f4594a5cbe None admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:35:49.627127 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d5e9a5-618c-4e09-a851-e99ec20f6363 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:35:49.627793 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d5e9a5-618c-4e09-a851-e99ec20f6363 None admin] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:49.628030 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d5e9a5-618c-4e09-a851-e99ec20f6363 None admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:49.628305 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d5e9a5-618c-4e09-a851-e99ec20f6363 None admin] PATH_INFO: `/v3/roles` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:49.628994 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3d5e9a5-618c-4e09-a851-e99ec20f6363 None admin] RBAC: Adding query filter params (name=admin) {{(pid=88114) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:35:49.629298 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3d5e9a5-618c-4e09-a851-e99ec20f6363 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:35:49.630747 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3d5e9a5-618c-4e09-a851-e99ec20f6363 None admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:35:49.642339 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 549/1102] 10.222.0.13 () {56 vars in 1099 bytes} [Wed Aug 31 09:35:49 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:35:49.650737 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c85c546-211d-4c20-92c8-d49042e2d7da None admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:35:49.784777 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c85c546-211d-4c20-92c8-d49042e2d7da None admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:35:49.837891 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e010cda5-73b1-4a7d-a21d-c23e6861302b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:35:49.838722 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e010cda5-73b1-4a7d-a21d-c23e6861302b None admin] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:49.839148 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e010cda5-73b1-4a7d-a21d-c23e6861302b None admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:49.839505 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e010cda5-73b1-4a7d-a21d-c23e6861302b None admin] PATH_INFO: `/v3/users/b2548421988c4c268e5bf177b290c1bf` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:49.842589 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e010cda5-73b1-4a7d-a21d-c23e6861302b None admin] RBAC: Authorizing `identity:get_user(user_id=b2548421988c4c268e5bf177b290c1bf)` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:35:49.844327 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e010cda5-73b1-4a7d-a21d-c23e6861302b None admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:35:49.847354 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 554/1103] 10.222.0.13 () {56 vars in 1177 bytes} [Wed Aug 31 09:35:49 2022] GET /identity/v3/users/b2548421988c4c268e5bf177b290c1bf => generated 285 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:35:49.857870 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d5e9a5-618c-4e09-a851-e99ec20f6363 None admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:35:49.926473 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=100 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:35:50.017015 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d5e9a5-618c-4e09-a851-e99ec20f6363 None admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:35:50.068274 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f92d19e-7f07-473d-a578-4733fdd42e1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:35:50.068903 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f92d19e-7f07-473d-a578-4733fdd42e1d None admin] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:50.069189 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f92d19e-7f07-473d-a578-4733fdd42e1d None admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:50.069519 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f92d19e-7f07-473d-a578-4733fdd42e1d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:50.168600 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f92d19e-7f07-473d-a578-4733fdd42e1d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:35:50.171133 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f92d19e-7f07-473d-a578-4733fdd42e1d None admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:35:50.243510 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=100 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:35:50.362200 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 550/1104] 10.222.0.13 () {58 vars in 1282 bytes} [Wed Aug 31 09:35:49 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 505 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 31 09:35:50.369581 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e010cda5-73b1-4a7d-a21d-c23e6861302b None admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:35:50.419482 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=100 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:35:50.493299 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e010cda5-73b1-4a7d-a21d-c23e6861302b None admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:35:50.532209 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e36c00-4096-42fc-aa1d-25289fb76b39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:35:50.532823 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e36c00-4096-42fc-aa1d-25289fb76b39 None admin] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:50.533147 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e36c00-4096-42fc-aa1d-25289fb76b39 None admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:50.533492 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e36c00-4096-42fc-aa1d-25289fb76b39 None admin] PATH_INFO: `/v3/projects/0f66b24ed33240238e962a5335cdab1c` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:50.534697 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0e36c00-4096-42fc-aa1d-25289fb76b39 None admin] RBAC: Authorizing `identity:get_project(project_id=0f66b24ed33240238e962a5335cdab1c)` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:35:50.536193 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0e36c00-4096-42fc-aa1d-25289fb76b39 None admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:35:50.538204 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 555/1105] 10.222.0.13 () {56 vars in 1186 bytes} [Wed Aug 31 09:35:50 2022] GET /identity/v3/projects/0f66b24ed33240238e962a5335cdab1c => generated 311 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:35:50.545500 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f92d19e-7f07-473d-a578-4733fdd42e1d None admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:35:50.658212 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f92d19e-7f07-473d-a578-4733fdd42e1d None admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:35:50.697605 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddfe8a79-0c0d-49b5-b2e8-ab5f9a2e6de2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:35:50.698269 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddfe8a79-0c0d-49b5-b2e8-ab5f9a2e6de2 None admin] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:50.698589 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddfe8a79-0c0d-49b5-b2e8-ab5f9a2e6de2 None admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:50.698913 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddfe8a79-0c0d-49b5-b2e8-ab5f9a2e6de2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:50.701030 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddfe8a79-0c0d-49b5-b2e8-ab5f9a2e6de2 None admin] RBAC: Adding query filter params (role.id=5d4ee63a3add47fc9a6b8b16d39e0a1f, scope.project.id=0f66b24ed33240238e962a5335cdab1c, user.id=b2548421988c4c268e5bf177b290c1bf) {{(pid=88114) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:35:50.701423 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddfe8a79-0c0d-49b5-b2e8-ab5f9a2e6de2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:35:50.702928 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddfe8a79-0c0d-49b5-b2e8-ab5f9a2e6de2 None admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:35:50.709530 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 551/1106] 10.222.0.13 () {56 vars in 1374 bytes} [Wed Aug 31 09:35:50 2022] GET /identity/v3/role_assignments?user.id=b2548421988c4c268e5bf177b290c1bf&scope.project.id=0f66b24ed33240238e962a5335cdab1c&role.id=5d4ee63a3add47fc9a6b8b16d39e0a1f => generated 608 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:35:52.763312 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9093462e-5c31-41a6-80bb-717fdf26587b None None] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:52.764174 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9093462e-5c31-41a6-80bb-717fdf26587b None None] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:52.764174 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9093462e-5c31-41a6-80bb-717fdf26587b None None] PATH_INFO: `/` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:52.765268 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 556/1107] 10.222.0.13 () {54 vars in 927 bytes} [Wed Aug 31 09:35:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 31 09:35:52.771647 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c40d19-6c25-419e-b240-7928c1b3e1e2 None None] REQUEST_METHOD: `POST` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:52.772234 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c40d19-6c25-419e-b240-7928c1b3e1e2 None None] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:52.772234 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c40d19-6c25-419e-b240-7928c1b3e1e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:52.779716 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=100 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:35:52.794072 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.auth.core [None req-f0c40d19-6c25-419e-b240-7928c1b3e1e2 None None] MFA Rules not processed for user `fd836cf590a145df9acf19cf3ae2ac56`. Rule list: `[]` (Enabled: `True`). {{(pid=88114) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:35:52.795583 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.fernet_utils [None req-f0c40d19-6c25-419e-b240-7928c1b3e1e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88114) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:35:52.875865 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 552/1108] 10.222.0.13 () {58 vars in 1024 bytes} [Wed Aug 31 09:35:52 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 31 09:35:53.156159 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9093462e-5c31-41a6-80bb-717fdf26587b None None] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:35:53.276002 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9093462e-5c31-41a6-80bb-717fdf26587b None None] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:35:53.280071 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=100 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:35:53.318711 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-276f7719-7978-47e6-be39-93f87c3e58ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:35:53.319421 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276f7719-7978-47e6-be39-93f87c3e58ec None admin] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:53.319746 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276f7719-7978-47e6-be39-93f87c3e58ec None admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:53.320064 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276f7719-7978-47e6-be39-93f87c3e58ec None admin] PATH_INFO: `/v3/domains/default` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:53.321500 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-276f7719-7978-47e6-be39-93f87c3e58ec None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:35:53.323041 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-276f7719-7978-47e6-be39-93f87c3e58ec None admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:35:53.325240 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 557/1109] 10.222.0.13 () {56 vars in 1108 bytes} [Wed Aug 31 09:35:53 2022] GET /identity/v3/domains/default => generated 199 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 09:35:53.331190 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c40d19-6c25-419e-b240-7928c1b3e1e2 None None] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:35:53.443828 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c40d19-6c25-419e-b240-7928c1b3e1e2 None None] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:35:53.478171 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=100 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:35:53.484996 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c05a159c-1733-4c97-b0e9-ec44b9051484 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:35:53.485644 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05a159c-1733-4c97-b0e9-ec44b9051484 None admin] REQUEST_METHOD: `POST` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:53.485921 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05a159c-1733-4c97-b0e9-ec44b9051484 None admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:53.486190 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05a159c-1733-4c97-b0e9-ec44b9051484 None admin] PATH_INFO: `/v3/users` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:53.486854 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c05a159c-1733-4c97-b0e9-ec44b9051484 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:35:53.488289 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c05a159c-1733-4c97-b0e9-ec44b9051484 None admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:35:53.511718 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 553/1110] 10.222.0.13 () {60 vars in 1132 bytes} [Wed Aug 31 09:35:53 2022] POST /identity/v3/users => generated 286 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 09:35:55.595550 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af46d605-96bd-4ab9-b3b0-269d3a7787f5 None None] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:55.596127 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af46d605-96bd-4ab9-b3b0-269d3a7787f5 None None] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:55.596127 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af46d605-96bd-4ab9-b3b0-269d3a7787f5 None None] PATH_INFO: `/` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:55.597467 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 558/1111] 10.222.0.13 () {54 vars in 927 bytes} [Wed Aug 31 09:35:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:35:55.603637 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c0b9c1-307e-4506-8f5c-7dd16c570d5a None None] REQUEST_METHOD: `POST` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:55.604148 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c0b9c1-307e-4506-8f5c-7dd16c570d5a None None] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:55.604448 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c0b9c1-307e-4506-8f5c-7dd16c570d5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:55.628457 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.auth.core [None req-02c0b9c1-307e-4506-8f5c-7dd16c570d5a None None] MFA Rules not processed for user `fd836cf590a145df9acf19cf3ae2ac56`. Rule list: `[]` (Enabled: `True`). {{(pid=88114) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:35:55.630524 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.fernet_utils [None req-02c0b9c1-307e-4506-8f5c-7dd16c570d5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88114) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:35:55.745650 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 554/1112] 10.222.0.13 () {58 vars in 1024 bytes} [Wed Aug 31 09:35:55 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 09:35:56.031357 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af46d605-96bd-4ab9-b3b0-269d3a7787f5 None None] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:35:56.151560 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af46d605-96bd-4ab9-b3b0-269d3a7787f5 None None] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:35:56.194544 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-386ecf30-409f-40db-a252-57d15ddb018a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:35:56.195194 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386ecf30-409f-40db-a252-57d15ddb018a None admin] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:56.195475 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386ecf30-409f-40db-a252-57d15ddb018a None admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:56.195746 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386ecf30-409f-40db-a252-57d15ddb018a None admin] PATH_INFO: `/v3/roles/889857b056624d47b572d4a80ecc3f8e` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:56.197326 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-386ecf30-409f-40db-a252-57d15ddb018a None admin] RBAC: Authorizing `identity:get_role(role_id=889857b056624d47b572d4a80ecc3f8e)` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:35:56.198816 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-386ecf30-409f-40db-a252-57d15ddb018a None admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:35:56.201471 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 559/1113] 10.222.0.13 () {56 vars in 1177 bytes} [Wed Aug 31 09:35:56 2022] GET /identity/v3/roles/889857b056624d47b572d4a80ecc3f8e => generated 223 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 09:35:56.207722 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c0b9c1-307e-4506-8f5c-7dd16c570d5a None None] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:35:56.346595 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c0b9c1-307e-4506-8f5c-7dd16c570d5a None None] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:35:56.365995 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:35:56.386377 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8130020-daed-40f0-9f75-b1df18c6b6df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:35:56.387605 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8130020-daed-40f0-9f75-b1df18c6b6df None admin] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:56.387890 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8130020-daed-40f0-9f75-b1df18c6b6df None admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:56.388165 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8130020-daed-40f0-9f75-b1df18c6b6df None admin] PATH_INFO: `/v3/users/737c82143b18442aa70aa210ac6b1162` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:56.403252 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8130020-daed-40f0-9f75-b1df18c6b6df None admin] RBAC: Authorizing `identity:get_user(user_id=737c82143b18442aa70aa210ac6b1162)` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:35:56.404709 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8130020-daed-40f0-9f75-b1df18c6b6df None admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:35:56.406462 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 555/1114] 10.222.0.13 () {56 vars in 1177 bytes} [Wed Aug 31 09:35:56 2022] GET /identity/v3/users/737c82143b18442aa70aa210ac6b1162 => generated 286 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:35:56.412645 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-386ecf30-409f-40db-a252-57d15ddb018a None admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:35:56.459303 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=100 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:35:56.524332 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-386ecf30-409f-40db-a252-57d15ddb018a None admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:35:56.565935 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2295fb0d-2814-424e-912b-4ec20eb4ebcf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:35:56.566619 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2295fb0d-2814-424e-912b-4ec20eb4ebcf None admin] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:56.566937 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2295fb0d-2814-424e-912b-4ec20eb4ebcf None admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:56.567270 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2295fb0d-2814-424e-912b-4ec20eb4ebcf None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:56.646119 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2295fb0d-2814-424e-912b-4ec20eb4ebcf None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:35:56.647558 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2295fb0d-2814-424e-912b-4ec20eb4ebcf None admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:35:56.694104 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=100 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:35:56.766969 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 560/1115] 10.222.0.13 () {58 vars in 1282 bytes} [Wed Aug 31 09:35:56 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 355 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 31 09:35:56.773793 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8130020-daed-40f0-9f75-b1df18c6b6df None admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:35:56.913783 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8130020-daed-40f0-9f75-b1df18c6b6df None admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:35:56.951322 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ed16ab-c19c-4deb-8866-a8585ddf3865 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:35:56.951968 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ed16ab-c19c-4deb-8866-a8585ddf3865 None admin] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:56.952248 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ed16ab-c19c-4deb-8866-a8585ddf3865 None admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:56.952519 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ed16ab-c19c-4deb-8866-a8585ddf3865 None admin] PATH_INFO: `/v3/projects/0f66b24ed33240238e962a5335cdab1c` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:56.953778 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82ed16ab-c19c-4deb-8866-a8585ddf3865 None admin] RBAC: Authorizing `identity:get_project(project_id=0f66b24ed33240238e962a5335cdab1c)` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:35:56.955212 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82ed16ab-c19c-4deb-8866-a8585ddf3865 None admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:35:56.956878 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 556/1116] 10.222.0.13 () {56 vars in 1186 bytes} [Wed Aug 31 09:35:56 2022] GET /identity/v3/projects/0f66b24ed33240238e962a5335cdab1c => generated 311 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:35:56.966932 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2295fb0d-2814-424e-912b-4ec20eb4ebcf None admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:35:57.071567 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2295fb0d-2814-424e-912b-4ec20eb4ebcf None admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:35:57.109871 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe4d60d4-fb9d-4973-bc8e-e21c5e7f64ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:35:57.110482 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4d60d4-fb9d-4973-bc8e-e21c5e7f64ab None admin] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:57.110761 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4d60d4-fb9d-4973-bc8e-e21c5e7f64ab None admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:57.111033 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4d60d4-fb9d-4973-bc8e-e21c5e7f64ab None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:57.111770 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe4d60d4-fb9d-4973-bc8e-e21c5e7f64ab None admin] RBAC: Adding query filter params (role.id=889857b056624d47b572d4a80ecc3f8e, scope.project.id=0f66b24ed33240238e962a5335cdab1c, user.id=737c82143b18442aa70aa210ac6b1162) {{(pid=88113) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:35:57.112079 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe4d60d4-fb9d-4973-bc8e-e21c5e7f64ab None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:35:57.113546 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe4d60d4-fb9d-4973-bc8e-e21c5e7f64ab None admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:35:57.119547 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 561/1117] 10.222.0.13 () {56 vars in 1374 bytes} [Wed Aug 31 09:35:56 2022] GET /identity/v3/role_assignments?user.id=737c82143b18442aa70aa210ac6b1162&scope.project.id=0f66b24ed33240238e962a5335cdab1c&role.id=889857b056624d47b572d4a80ecc3f8e => generated 259 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:35:59.167481 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7e11b6-242c-401d-8925-47205303e9f8 None None] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:59.168401 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7e11b6-242c-401d-8925-47205303e9f8 None None] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:59.168401 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7e11b6-242c-401d-8925-47205303e9f8 None None] PATH_INFO: `/` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:59.170007 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 557/1118] 10.222.0.13 () {54 vars in 927 bytes} [Wed Aug 31 09:35:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:35:59.175665 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e092bb78-61ea-4253-a100-572c423daadb None None] REQUEST_METHOD: `POST` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:59.176191 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e092bb78-61ea-4253-a100-572c423daadb None None] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:59.176191 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e092bb78-61ea-4253-a100-572c423daadb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:59.183644 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=100 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:35:59.196690 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.auth.core [None req-e092bb78-61ea-4253-a100-572c423daadb None None] MFA Rules not processed for user `fd836cf590a145df9acf19cf3ae2ac56`. Rule list: `[]` (Enabled: `True`). {{(pid=88113) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:35:59.198009 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.fernet_utils [None req-e092bb78-61ea-4253-a100-572c423daadb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:35:59.273578 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 562/1119] 10.222.0.13 () {58 vars in 1024 bytes} [Wed Aug 31 09:35:59 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 09:35:59.556807 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c7e11b6-242c-401d-8925-47205303e9f8 None None] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:35:59.610052 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=100 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:35:59.672826 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c7e11b6-242c-401d-8925-47205303e9f8 None None] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:35:59.711569 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fce37fe-fd87-4c62-b512-2b984945a978 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:35:59.712278 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fce37fe-fd87-4c62-b512-2b984945a978 None admin] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:59.712598 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fce37fe-fd87-4c62-b512-2b984945a978 None admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:59.712920 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fce37fe-fd87-4c62-b512-2b984945a978 None admin] PATH_INFO: `/v3/roles/889857b056624d47b572d4a80ecc3f8e` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:59.714456 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fce37fe-fd87-4c62-b512-2b984945a978 None admin] RBAC: Authorizing `identity:get_role(role_id=889857b056624d47b572d4a80ecc3f8e)` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:35:59.715974 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fce37fe-fd87-4c62-b512-2b984945a978 None admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:35:59.717348 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 558/1120] 10.222.0.13 () {56 vars in 1177 bytes} [Wed Aug 31 09:35:59 2022] GET /identity/v3/roles/889857b056624d47b572d4a80ecc3f8e => generated 223 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:35:59.723641 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e092bb78-61ea-4253-a100-572c423daadb None None] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:35:59.836332 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e092bb78-61ea-4253-a100-572c423daadb None None] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:35:59.877795 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb9bfc0d-610a-4aad-809a-0d5fd6d7e098 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:35:59.878510 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9bfc0d-610a-4aad-809a-0d5fd6d7e098 None admin] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:35:59.878836 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9bfc0d-610a-4aad-809a-0d5fd6d7e098 None admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:35:59.879156 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9bfc0d-610a-4aad-809a-0d5fd6d7e098 None admin] PATH_INFO: `/v3/users/737c82143b18442aa70aa210ac6b1162` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:35:59.880402 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb9bfc0d-610a-4aad-809a-0d5fd6d7e098 None admin] RBAC: Authorizing `identity:get_user(user_id=737c82143b18442aa70aa210ac6b1162)` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:35:59.881954 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb9bfc0d-610a-4aad-809a-0d5fd6d7e098 None admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:35:59.883693 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 563/1121] 10.222.0.13 () {56 vars in 1177 bytes} [Wed Aug 31 09:35:59 2022] GET /identity/v3/users/737c82143b18442aa70aa210ac6b1162 => generated 286 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:35:59.890346 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fce37fe-fd87-4c62-b512-2b984945a978 None admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:36:00.012435 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fce37fe-fd87-4c62-b512-2b984945a978 None admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:36:00.028187 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=100 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:36:00.051696 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bddcd60-1cd2-442c-b2a7-86c5faacb8ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:36:00.052342 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bddcd60-1cd2-442c-b2a7-86c5faacb8ee None admin] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:36:00.052619 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bddcd60-1cd2-442c-b2a7-86c5faacb8ee None admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:36:00.052891 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bddcd60-1cd2-442c-b2a7-86c5faacb8ee None admin] PATH_INFO: `/v3/projects/0f66b24ed33240238e962a5335cdab1c` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:36:00.054088 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bddcd60-1cd2-442c-b2a7-86c5faacb8ee None admin] RBAC: Authorizing `identity:get_project(project_id=0f66b24ed33240238e962a5335cdab1c)` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:36:00.055525 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bddcd60-1cd2-442c-b2a7-86c5faacb8ee None admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:36:00.057208 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 559/1122] 10.222.0.13 () {56 vars in 1186 bytes} [Wed Aug 31 09:35:59 2022] GET /identity/v3/projects/0f66b24ed33240238e962a5335cdab1c => generated 311 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:36:00.064830 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb9bfc0d-610a-4aad-809a-0d5fd6d7e098 None admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:36:00.067970 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=100 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:36:00.182897 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb9bfc0d-610a-4aad-809a-0d5fd6d7e098 None admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:36:00.222641 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ac744d-b87a-4898-b9a3-f72dadf47c3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:36:00.223324 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ac744d-b87a-4898-b9a3-f72dadf47c3b None admin] REQUEST_METHOD: `PUT` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:36:00.223598 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ac744d-b87a-4898-b9a3-f72dadf47c3b None admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:36:00.223868 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ac744d-b87a-4898-b9a3-f72dadf47c3b None admin] PATH_INFO: `/v3/projects/0f66b24ed33240238e962a5335cdab1c/users/737c82143b18442aa70aa210ac6b1162/roles/889857b056624d47b572d4a80ecc3f8e` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:36:00.226049 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46ac744d-b87a-4898-b9a3-f72dadf47c3b None admin] RBAC: Authorizing `identity:create_grant(project_id=0f66b24ed33240238e962a5335cdab1c, user_id=737c82143b18442aa70aa210ac6b1162, role_id=889857b056624d47b572d4a80ecc3f8e)` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:36:00.227508 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46ac744d-b87a-4898-b9a3-f72dadf47c3b None admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:36:00.243188 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 564/1123] 10.222.0.13 () {58 vars in 1439 bytes} [Wed Aug 31 09:36:00 2022] PUT /identity/v3/projects/0f66b24ed33240238e962a5335cdab1c/users/737c82143b18442aa70aa210ac6b1162/roles/889857b056624d47b572d4a80ecc3f8e => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:36:02.196168 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c279ccf-445a-42e5-b652-07990aa4cc76 None None] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:36:02.196776 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c279ccf-445a-42e5-b652-07990aa4cc76 None None] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:36:02.197150 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c279ccf-445a-42e5-b652-07990aa4cc76 None None] PATH_INFO: `/` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:36:02.200625 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 560/1124] 10.222.0.13 () {54 vars in 927 bytes} [Wed Aug 31 09:36:02 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:36:02.208758 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef1bcbb-d1ec-4af3-b93b-b91c12192cd4 None None] REQUEST_METHOD: `POST` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:36:02.209630 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef1bcbb-d1ec-4af3-b93b-b91c12192cd4 None None] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:36:02.209803 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef1bcbb-d1ec-4af3-b93b-b91c12192cd4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:36:02.230874 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.auth.core [None req-bef1bcbb-d1ec-4af3-b93b-b91c12192cd4 None None] MFA Rules not processed for user `fd836cf590a145df9acf19cf3ae2ac56`. Rule list: `[]` (Enabled: `True`). {{(pid=88113) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:36:02.232197 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.fernet_utils [None req-bef1bcbb-d1ec-4af3-b93b-b91c12192cd4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:36:02.299626 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:36:02.312620 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 565/1125] 10.222.0.13 () {58 vars in 1024 bytes} [Wed Aug 31 09:36:02 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 09:36:02.584514 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c279ccf-445a-42e5-b652-07990aa4cc76 None None] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:36:02.696934 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c279ccf-445a-42e5-b652-07990aa4cc76 None None] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:36:02.735986 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f5fdbdd-4af9-4b5d-88c5-4ae7eaf355a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:36:02.736630 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5fdbdd-4af9-4b5d-88c5-4ae7eaf355a0 None admin] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:36:02.736909 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5fdbdd-4af9-4b5d-88c5-4ae7eaf355a0 None admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:36:02.737183 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5fdbdd-4af9-4b5d-88c5-4ae7eaf355a0 None admin] PATH_INFO: `/v3/roles/889857b056624d47b572d4a80ecc3f8e` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:36:02.738664 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f5fdbdd-4af9-4b5d-88c5-4ae7eaf355a0 None admin] RBAC: Authorizing `identity:get_role(role_id=889857b056624d47b572d4a80ecc3f8e)` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:36:02.740036 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f5fdbdd-4af9-4b5d-88c5-4ae7eaf355a0 None admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:36:02.741439 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 561/1126] 10.222.0.13 () {56 vars in 1177 bytes} [Wed Aug 31 09:36:02 2022] GET /identity/v3/roles/889857b056624d47b572d4a80ecc3f8e => generated 223 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 09:36:02.748076 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef1bcbb-d1ec-4af3-b93b-b91c12192cd4 None None] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:36:02.857756 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef1bcbb-d1ec-4af3-b93b-b91c12192cd4 None None] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:36:02.898491 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38875977-4005-4894-bcda-580ad4f6f5bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:36:02.899161 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38875977-4005-4894-bcda-580ad4f6f5bb None admin] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:36:02.899438 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38875977-4005-4894-bcda-580ad4f6f5bb None admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:36:02.899708 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38875977-4005-4894-bcda-580ad4f6f5bb None admin] PATH_INFO: `/v3/users/737c82143b18442aa70aa210ac6b1162` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:36:02.901003 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38875977-4005-4894-bcda-580ad4f6f5bb None admin] RBAC: Authorizing `identity:get_user(user_id=737c82143b18442aa70aa210ac6b1162)` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:36:02.902954 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38875977-4005-4894-bcda-580ad4f6f5bb None admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:36:02.904387 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 566/1127] 10.222.0.13 () {56 vars in 1177 bytes} [Wed Aug 31 09:36:02 2022] GET /identity/v3/users/737c82143b18442aa70aa210ac6b1162 => generated 286 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:36:02.910906 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f5fdbdd-4af9-4b5d-88c5-4ae7eaf355a0 None admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:36:02.975352 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=100 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:36:03.038975 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f5fdbdd-4af9-4b5d-88c5-4ae7eaf355a0 None admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:36:03.080343 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5491f1d6-d40f-4c32-9e1e-330fc6be70ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:36:03.081026 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5491f1d6-d40f-4c32-9e1e-330fc6be70ff None admin] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:36:03.081351 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5491f1d6-d40f-4c32-9e1e-330fc6be70ff None admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:36:03.081705 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5491f1d6-d40f-4c32-9e1e-330fc6be70ff None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:36:03.159268 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5491f1d6-d40f-4c32-9e1e-330fc6be70ff None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:36:03.160636 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5491f1d6-d40f-4c32-9e1e-330fc6be70ff None admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:36:03.213736 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=100 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:36:03.271240 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 562/1128] 10.222.0.13 () {58 vars in 1282 bytes} [Wed Aug 31 09:36:02 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 361 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 31 09:36:03.278102 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38875977-4005-4894-bcda-580ad4f6f5bb None admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:36:03.342401 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=100 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:36:03.388497 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38875977-4005-4894-bcda-580ad4f6f5bb None admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:36:03.426901 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb3d6b40-d72e-42a0-bd2c-e0f7e9ea6d65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:36:03.427458 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3d6b40-d72e-42a0-bd2c-e0f7e9ea6d65 None admin] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:36:03.427734 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3d6b40-d72e-42a0-bd2c-e0f7e9ea6d65 None admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:36:03.428004 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3d6b40-d72e-42a0-bd2c-e0f7e9ea6d65 None admin] PATH_INFO: `/v3/projects/0f66b24ed33240238e962a5335cdab1c` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:36:03.429165 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb3d6b40-d72e-42a0-bd2c-e0f7e9ea6d65 None admin] RBAC: Authorizing `identity:get_project(project_id=0f66b24ed33240238e962a5335cdab1c)` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:36:03.430642 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb3d6b40-d72e-42a0-bd2c-e0f7e9ea6d65 None admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:36:03.432318 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 567/1129] 10.222.0.13 () {56 vars in 1186 bytes} [Wed Aug 31 09:36:03 2022] GET /identity/v3/projects/0f66b24ed33240238e962a5335cdab1c => generated 311 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 09:36:03.438975 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5491f1d6-d40f-4c32-9e1e-330fc6be70ff None admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:36:03.550745 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5491f1d6-d40f-4c32-9e1e-330fc6be70ff None admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:36:03.610867 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b14a082e-cee8-4e81-89dd-4fa47a382b62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:36:03.611895 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14a082e-cee8-4e81-89dd-4fa47a382b62 None admin] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:36:03.612381 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14a082e-cee8-4e81-89dd-4fa47a382b62 None admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:36:03.612863 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14a082e-cee8-4e81-89dd-4fa47a382b62 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:36:03.614189 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b14a082e-cee8-4e81-89dd-4fa47a382b62 None admin] RBAC: Adding query filter params (role.id=889857b056624d47b572d4a80ecc3f8e, scope.project.id=0f66b24ed33240238e962a5335cdab1c, user.id=737c82143b18442aa70aa210ac6b1162) {{(pid=88114) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:36:03.614693 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b14a082e-cee8-4e81-89dd-4fa47a382b62 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:36:03.617119 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b14a082e-cee8-4e81-89dd-4fa47a382b62 None admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:36:03.628744 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 563/1130] 10.222.0.13 () {56 vars in 1374 bytes} [Wed Aug 31 09:36:03 2022] GET /identity/v3/role_assignments?user.id=737c82143b18442aa70aa210ac6b1162&scope.project.id=0f66b24ed33240238e962a5335cdab1c&role.id=889857b056624d47b572d4a80ecc3f8e => generated 608 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:36:05.615358 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5a6be3-9a70-475f-bd85-0be2f1d50b85 None None] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:36:05.615925 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5a6be3-9a70-475f-bd85-0be2f1d50b85 None None] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:36:05.615925 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5a6be3-9a70-475f-bd85-0be2f1d50b85 None None] PATH_INFO: `/` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:36:05.617309 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 568/1131] 10.222.0.13 () {54 vars in 927 bytes} [Wed Aug 31 09:36:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:36:05.623696 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed2c54e-ccd7-435d-801a-26f6a7b37eda None None] REQUEST_METHOD: `POST` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:36:05.624284 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed2c54e-ccd7-435d-801a-26f6a7b37eda None None] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:36:05.624284 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed2c54e-ccd7-435d-801a-26f6a7b37eda None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:36:05.644246 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.auth.core [None req-2ed2c54e-ccd7-435d-801a-26f6a7b37eda None None] MFA Rules not processed for user `fd836cf590a145df9acf19cf3ae2ac56`. Rule list: `[]` (Enabled: `True`). {{(pid=88114) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:36:05.645507 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.fernet_utils [None req-2ed2c54e-ccd7-435d-801a-26f6a7b37eda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88114) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:36:05.655650 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=100 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:36:05.729162 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 564/1132] 10.222.0.13 () {58 vars in 1024 bytes} [Wed Aug 31 09:36:05 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 09:36:06.079344 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d5a6be3-9a70-475f-bd85-0be2f1d50b85 None None] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:36:06.192744 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d5a6be3-9a70-475f-bd85-0be2f1d50b85 None None] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:36:06.222782 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=100 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:36:06.232492 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c59f9b6b-36a1-4776-b866-e32a9720dbc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:36:06.233158 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59f9b6b-36a1-4776-b866-e32a9720dbc7 None admin] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:36:06.233459 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59f9b6b-36a1-4776-b866-e32a9720dbc7 None admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:36:06.233812 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59f9b6b-36a1-4776-b866-e32a9720dbc7 None admin] PATH_INFO: `/v3/domains/default` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:36:06.235013 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c59f9b6b-36a1-4776-b866-e32a9720dbc7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:36:06.236825 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c59f9b6b-36a1-4776-b866-e32a9720dbc7 None admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:36:06.238487 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 569/1133] 10.222.0.13 () {56 vars in 1108 bytes} [Wed Aug 31 09:36:06 2022] GET /identity/v3/domains/default => generated 199 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:36:06.248325 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed2c54e-ccd7-435d-801a-26f6a7b37eda None None] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:36:06.369135 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed2c54e-ccd7-435d-801a-26f6a7b37eda None None] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:36:06.409829 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3582411-a5d6-4a80-b176-ebe36d89ca93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:36:06.410454 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3582411-a5d6-4a80-b176-ebe36d89ca93 None admin] REQUEST_METHOD: `POST` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:36:06.410730 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3582411-a5d6-4a80-b176-ebe36d89ca93 None admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:36:06.410999 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3582411-a5d6-4a80-b176-ebe36d89ca93 None admin] PATH_INFO: `/v3/projects` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:36:06.411705 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3582411-a5d6-4a80-b176-ebe36d89ca93 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:36:06.413157 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3582411-a5d6-4a80-b176-ebe36d89ca93 None admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:36:06.429845 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=100 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:36:06.444165 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 565/1134] 10.222.0.13 () {60 vars in 1141 bytes} [Wed Aug 31 09:36:06 2022] POST /identity/v3/projects => generated 311 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 09:36:08.468690 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b3afda-c955-468d-9c38-e7e6e832e856 None None] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:36:08.469301 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b3afda-c955-468d-9c38-e7e6e832e856 None None] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:36:08.469301 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b3afda-c955-468d-9c38-e7e6e832e856 None None] PATH_INFO: `/` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:36:08.470855 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 570/1135] 10.222.0.13 () {54 vars in 927 bytes} [Wed Aug 31 09:36:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 31 09:36:08.476878 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf03e708-e39e-4366-9df0-672bcddd99eb None None] REQUEST_METHOD: `POST` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:36:08.477676 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf03e708-e39e-4366-9df0-672bcddd99eb None None] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:36:08.477676 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf03e708-e39e-4366-9df0-672bcddd99eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:36:08.498236 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.auth.core [None req-cf03e708-e39e-4366-9df0-672bcddd99eb None None] MFA Rules not processed for user `fd836cf590a145df9acf19cf3ae2ac56`. Rule list: `[]` (Enabled: `True`). {{(pid=88114) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:36:08.499461 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.fernet_utils [None req-cf03e708-e39e-4366-9df0-672bcddd99eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88114) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:36:08.576234 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 566/1136] 10.222.0.13 () {58 vars in 1024 bytes} [Wed Aug 31 09:36:08 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 31 09:36:08.861460 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b3afda-c955-468d-9c38-e7e6e832e856 None None] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:36:08.973356 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b3afda-c955-468d-9c38-e7e6e832e856 None None] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:36:09.010771 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d668985d-4706-4557-825a-95f6b6ee3b47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:36:09.011605 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d668985d-4706-4557-825a-95f6b6ee3b47 None admin] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:36:09.012018 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d668985d-4706-4557-825a-95f6b6ee3b47 None admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:36:09.012293 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d668985d-4706-4557-825a-95f6b6ee3b47 None admin] PATH_INFO: `/v3/domains/default` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:36:09.013515 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d668985d-4706-4557-825a-95f6b6ee3b47 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:36:09.014971 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d668985d-4706-4557-825a-95f6b6ee3b47 None admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:36:09.016632 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 571/1137] 10.222.0.13 () {56 vars in 1108 bytes} [Wed Aug 31 09:36:08 2022] GET /identity/v3/domains/default => generated 199 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 09:36:09.025719 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf03e708-e39e-4366-9df0-672bcddd99eb None None] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:36:09.137373 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf03e708-e39e-4366-9df0-672bcddd99eb None None] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:36:09.163580 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:36:09.179831 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee193d82-3267-4976-84b4-f9a9a63be5a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:36:09.180444 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee193d82-3267-4976-84b4-f9a9a63be5a8 None admin] REQUEST_METHOD: `POST` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:36:09.180721 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee193d82-3267-4976-84b4-f9a9a63be5a8 None admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:36:09.180988 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee193d82-3267-4976-84b4-f9a9a63be5a8 None admin] PATH_INFO: `/v3/users` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:36:09.181686 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee193d82-3267-4976-84b4-f9a9a63be5a8 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:36:09.183174 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee193d82-3267-4976-84b4-f9a9a63be5a8 None admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:36:09.207535 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 567/1138] 10.222.0.13 () {60 vars in 1132 bytes} [Wed Aug 31 09:36:09 2022] POST /identity/v3/users => generated 286 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 09:36:11.224076 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f807dd3-fdef-4a56-be1e-2cb1bd0feb90 None None] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:36:11.224612 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f807dd3-fdef-4a56-be1e-2cb1bd0feb90 None None] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:36:11.224612 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f807dd3-fdef-4a56-be1e-2cb1bd0feb90 None None] PATH_INFO: `/` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:36:11.225945 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 572/1139] 10.222.0.13 () {54 vars in 927 bytes} [Wed Aug 31 09:36:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 31 09:36:11.231755 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d13a57b-c2c8-4c3a-bd76-4d265084c573 None None] REQUEST_METHOD: `POST` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:36:11.232218 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d13a57b-c2c8-4c3a-bd76-4d265084c573 None None] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:36:11.232337 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d13a57b-c2c8-4c3a-bd76-4d265084c573 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:36:11.251972 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.auth.core [None req-3d13a57b-c2c8-4c3a-bd76-4d265084c573 None None] MFA Rules not processed for user `fd836cf590a145df9acf19cf3ae2ac56`. Rule list: `[]` (Enabled: `True`). {{(pid=88114) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:36:11.253199 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.fernet_utils [None req-3d13a57b-c2c8-4c3a-bd76-4d265084c573 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88114) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:36:11.326962 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 568/1140] 10.222.0.13 () {58 vars in 1024 bytes} [Wed Aug 31 09:36:11 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 31 09:36:11.601814 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f807dd3-fdef-4a56-be1e-2cb1bd0feb90 None None] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:36:11.670860 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=100 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:36:11.712830 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f807dd3-fdef-4a56-be1e-2cb1bd0feb90 None None] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:36:11.754374 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6603f1c-e0a8-4408-8350-0d4f1bb2058a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:36:11.755021 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6603f1c-e0a8-4408-8350-0d4f1bb2058a None admin] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:36:11.755303 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6603f1c-e0a8-4408-8350-0d4f1bb2058a None admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:36:11.755576 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6603f1c-e0a8-4408-8350-0d4f1bb2058a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:36:11.767726 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6603f1c-e0a8-4408-8350-0d4f1bb2058a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 09:36:11.772625 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application [None req-c6603f1c-e0a8-4408-8350-0d4f1bb2058a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 09:36:11.772625 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:36:11.772625 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:36:11.772625 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:36:11.772625 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:36:11.772625 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:36:11.772625 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:36:11.772625 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:36:11.772625 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:36:11.772625 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:36:11.772625 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:36:11.772625 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:36:11.772625 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 09:36:11.772625 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 09:36:11.772625 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 09:36:11.772625 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 09:36:11.774403 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 09:36:11.774403 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 09:36:11.774403 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 09:36:11.774403 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 09:36:11.774403 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:36:11.774403 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:36:11.774403 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:36:11.774403 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:36:11.774403 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:36:11.774403 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:36:11.774403 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:36:11.774403 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application with Lock( Aug 31 09:36:11.774403 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:36:11.774403 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:36:11.774403 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:36:11.774403 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:36:11.776350 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:36:11.776350 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:36:11.776350 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:36:11.776350 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:36:11.776350 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 09:36:11.776350 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 09:36:11.776350 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 09:36:11.776350 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 09:36:11.776350 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 09:36:11.776350 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 09:36:11.776350 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 09:36:11.776350 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application Aug 31 09:36:11.776350 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 573/1141] 10.222.0.13 () {56 vars in 1096 bytes} [Wed Aug 31 09:36:11 2022] GET /identity/v3/roles/admin => generated 83 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 09:36:11.781585 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d13a57b-c2c8-4c3a-bd76-4d265084c573 None None] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:36:11.904226 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d13a57b-c2c8-4c3a-bd76-4d265084c573 None None] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:36:11.919812 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:36:11.949748 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b0a29f-4694-46e8-be9e-296d3541767b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:36:11.950385 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b0a29f-4694-46e8-be9e-296d3541767b None admin] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:36:11.950665 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b0a29f-4694-46e8-be9e-296d3541767b None admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:36:11.950938 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b0a29f-4694-46e8-be9e-296d3541767b None admin] PATH_INFO: `/v3/roles` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:36:11.951638 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5b0a29f-4694-46e8-be9e-296d3541767b None admin] RBAC: Adding query filter params (name=admin) {{(pid=88114) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:36:11.951945 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5b0a29f-4694-46e8-be9e-296d3541767b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:36:11.953358 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5b0a29f-4694-46e8-be9e-296d3541767b None admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:36:11.962986 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 569/1142] 10.222.0.13 () {56 vars in 1099 bytes} [Wed Aug 31 09:36:11 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:36:11.972453 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6603f1c-e0a8-4408-8350-0d4f1bb2058a None admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:36:12.079016 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6603f1c-e0a8-4408-8350-0d4f1bb2058a None admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:36:12.103347 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=100 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:36:12.115625 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba23be9-9e13-42ec-8cb7-495e33a4459d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:36:12.116255 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba23be9-9e13-42ec-8cb7-495e33a4459d None admin] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:36:12.116528 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba23be9-9e13-42ec-8cb7-495e33a4459d None admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:36:12.116794 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba23be9-9e13-42ec-8cb7-495e33a4459d None admin] PATH_INFO: `/v3/users/3981f3f67ae7467f8d159e7cf6d730fd` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:36:12.130837 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ba23be9-9e13-42ec-8cb7-495e33a4459d None admin] RBAC: Authorizing `identity:get_user(user_id=3981f3f67ae7467f8d159e7cf6d730fd)` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:36:12.132303 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ba23be9-9e13-42ec-8cb7-495e33a4459d None admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:36:12.134021 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 574/1143] 10.222.0.13 () {56 vars in 1177 bytes} [Wed Aug 31 09:36:11 2022] GET /identity/v3/users/3981f3f67ae7467f8d159e7cf6d730fd => generated 286 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:36:12.141277 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b0a29f-4694-46e8-be9e-296d3541767b None admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:36:12.245893 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b0a29f-4694-46e8-be9e-296d3541767b None admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:36:12.281941 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-012023fe-d2ca-4cf1-b743-a0bf288a66f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:36:12.282553 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012023fe-d2ca-4cf1-b743-a0bf288a66f7 None admin] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:36:12.282956 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012023fe-d2ca-4cf1-b743-a0bf288a66f7 None admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:36:12.283450 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012023fe-d2ca-4cf1-b743-a0bf288a66f7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:36:12.329006 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=100 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:36:12.357938 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-012023fe-d2ca-4cf1-b743-a0bf288a66f7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:36:12.359432 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-012023fe-d2ca-4cf1-b743-a0bf288a66f7 None admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:36:12.476290 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 570/1144] 10.222.0.13 () {58 vars in 1282 bytes} [Wed Aug 31 09:36:12 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 335 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 31 09:36:12.482884 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba23be9-9e13-42ec-8cb7-495e33a4459d None admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:36:12.594839 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba23be9-9e13-42ec-8cb7-495e33a4459d None admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:36:12.628907 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d32093-6953-4a26-8f2f-ee8eea7defc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:36:12.629566 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d32093-6953-4a26-8f2f-ee8eea7defc9 None admin] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:36:12.629847 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d32093-6953-4a26-8f2f-ee8eea7defc9 None admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:36:12.630121 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d32093-6953-4a26-8f2f-ee8eea7defc9 None admin] PATH_INFO: `/v3/projects/b39f712d51cc4ea6af519827d37767b7` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:36:12.631242 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36d32093-6953-4a26-8f2f-ee8eea7defc9 None admin] RBAC: Authorizing `identity:get_project(project_id=b39f712d51cc4ea6af519827d37767b7)` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:36:12.632682 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36d32093-6953-4a26-8f2f-ee8eea7defc9 None admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:36:12.634369 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 575/1145] 10.222.0.13 () {56 vars in 1186 bytes} [Wed Aug 31 09:36:12 2022] GET /identity/v3/projects/b39f712d51cc4ea6af519827d37767b7 => generated 311 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:36:12.641749 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-012023fe-d2ca-4cf1-b743-a0bf288a66f7 None admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:36:12.713165 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=100 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:36:12.747318 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-012023fe-d2ca-4cf1-b743-a0bf288a66f7 None admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:36:12.785524 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b82f3298-9563-4736-9bbd-4447cc8af041 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:36:12.786131 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82f3298-9563-4736-9bbd-4447cc8af041 None admin] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:36:12.786403 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82f3298-9563-4736-9bbd-4447cc8af041 None admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:36:12.786668 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82f3298-9563-4736-9bbd-4447cc8af041 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:36:12.787389 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b82f3298-9563-4736-9bbd-4447cc8af041 None admin] RBAC: Adding query filter params (role.id=5d4ee63a3add47fc9a6b8b16d39e0a1f, scope.project.id=b39f712d51cc4ea6af519827d37767b7, user.id=3981f3f67ae7467f8d159e7cf6d730fd) {{(pid=88114) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:36:12.787689 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b82f3298-9563-4736-9bbd-4447cc8af041 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:36:12.789113 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b82f3298-9563-4736-9bbd-4447cc8af041 None admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:36:12.796068 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 571/1146] 10.222.0.13 () {56 vars in 1374 bytes} [Wed Aug 31 09:36:12 2022] GET /identity/v3/role_assignments?user.id=3981f3f67ae7467f8d159e7cf6d730fd&scope.project.id=b39f712d51cc4ea6af519827d37767b7&role.id=5d4ee63a3add47fc9a6b8b16d39e0a1f => generated 259 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:36:14.728344 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb1b34f-4f2f-46d4-ad92-068053c8811b None None] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:36:14.728906 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb1b34f-4f2f-46d4-ad92-068053c8811b None None] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:36:14.728906 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb1b34f-4f2f-46d4-ad92-068053c8811b None None] PATH_INFO: `/` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:36:14.730278 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 576/1147] 10.222.0.13 () {54 vars in 927 bytes} [Wed Aug 31 09:36:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:36:14.735897 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6422ba-6648-4a04-ba76-242b322e7263 None None] REQUEST_METHOD: `POST` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:36:14.736330 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6422ba-6648-4a04-ba76-242b322e7263 None None] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:36:14.736445 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6422ba-6648-4a04-ba76-242b322e7263 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:36:14.754995 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.auth.core [None req-1f6422ba-6648-4a04-ba76-242b322e7263 None None] MFA Rules not processed for user `fd836cf590a145df9acf19cf3ae2ac56`. Rule list: `[]` (Enabled: `True`). {{(pid=88114) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:36:14.756207 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.fernet_utils [None req-1f6422ba-6648-4a04-ba76-242b322e7263 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88114) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:36:14.860437 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 572/1148] 10.222.0.13 () {58 vars in 1024 bytes} [Wed Aug 31 09:36:14 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 09:36:15.148796 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb1b34f-4f2f-46d4-ad92-068053c8811b None None] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:36:15.204923 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=100 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:36:15.260886 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb1b34f-4f2f-46d4-ad92-068053c8811b None None] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:36:15.298746 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d6fef5d-97f4-4968-a0b2-87e677252771 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:36:15.299377 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6fef5d-97f4-4968-a0b2-87e677252771 None admin] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:36:15.299654 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6fef5d-97f4-4968-a0b2-87e677252771 None admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:36:15.299925 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6fef5d-97f4-4968-a0b2-87e677252771 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:36:15.310832 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d6fef5d-97f4-4968-a0b2-87e677252771 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 09:36:15.315696 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application [None req-6d6fef5d-97f4-4968-a0b2-87e677252771 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 09:36:15.315696 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:36:15.315696 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:36:15.315696 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:36:15.315696 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:36:15.315696 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:36:15.315696 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:36:15.315696 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:36:15.315696 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:36:15.315696 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:36:15.315696 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:36:15.315696 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:36:15.315696 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 09:36:15.315696 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 09:36:15.315696 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 09:36:15.315696 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 09:36:15.317665 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 09:36:15.317665 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 09:36:15.317665 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 09:36:15.317665 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 09:36:15.317665 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:36:15.317665 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:36:15.317665 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:36:15.317665 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:36:15.317665 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:36:15.317665 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:36:15.317665 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:36:15.317665 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application with Lock( Aug 31 09:36:15.317665 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:36:15.317665 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:36:15.317665 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:36:15.317665 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:36:15.319577 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:36:15.319577 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:36:15.319577 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:36:15.319577 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:36:15.319577 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 09:36:15.319577 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 09:36:15.319577 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 09:36:15.319577 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 09:36:15.319577 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 09:36:15.319577 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 09:36:15.319577 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 09:36:15.319577 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application Aug 31 09:36:15.319577 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 577/1149] 10.222.0.13 () {56 vars in 1096 bytes} [Wed Aug 31 09:36:15 2022] GET /identity/v3/roles/admin => generated 83 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:36:15.325106 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f6422ba-6648-4a04-ba76-242b322e7263 None None] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:36:15.396463 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=100 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:36:15.433018 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f6422ba-6648-4a04-ba76-242b322e7263 None None] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:36:15.471942 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d7f62d4-0b8e-4adf-bfcc-3e20faae5a19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:36:15.472556 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7f62d4-0b8e-4adf-bfcc-3e20faae5a19 None admin] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:36:15.472914 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7f62d4-0b8e-4adf-bfcc-3e20faae5a19 None admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:36:15.473247 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7f62d4-0b8e-4adf-bfcc-3e20faae5a19 None admin] PATH_INFO: `/v3/roles` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:36:15.474179 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d7f62d4-0b8e-4adf-bfcc-3e20faae5a19 None admin] RBAC: Adding query filter params (name=admin) {{(pid=88114) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:36:15.474512 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d7f62d4-0b8e-4adf-bfcc-3e20faae5a19 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:36:15.476039 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d7f62d4-0b8e-4adf-bfcc-3e20faae5a19 None admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:36:15.485245 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 573/1150] 10.222.0.13 () {56 vars in 1099 bytes} [Wed Aug 31 09:36:15 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:36:15.491217 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d6fef5d-97f4-4968-a0b2-87e677252771 None admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:36:15.601805 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d6fef5d-97f4-4968-a0b2-87e677252771 None admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:36:15.615102 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=100 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:36:15.643228 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ba129f-947d-4f89-9b4c-e11f5afb6afc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:36:15.643888 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ba129f-947d-4f89-9b4c-e11f5afb6afc None admin] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:36:15.644160 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ba129f-947d-4f89-9b4c-e11f5afb6afc None admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:36:15.644433 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ba129f-947d-4f89-9b4c-e11f5afb6afc None admin] PATH_INFO: `/v3/users/3981f3f67ae7467f8d159e7cf6d730fd` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:36:15.645691 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05ba129f-947d-4f89-9b4c-e11f5afb6afc None admin] RBAC: Authorizing `identity:get_user(user_id=3981f3f67ae7467f8d159e7cf6d730fd)` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:36:15.647171 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05ba129f-947d-4f89-9b4c-e11f5afb6afc None admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:36:15.648883 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 578/1151] 10.222.0.13 () {56 vars in 1177 bytes} [Wed Aug 31 09:36:15 2022] GET /identity/v3/users/3981f3f67ae7467f8d159e7cf6d730fd => generated 286 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:36:15.656340 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d7f62d4-0b8e-4adf-bfcc-3e20faae5a19 None admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:36:15.772181 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d7f62d4-0b8e-4adf-bfcc-3e20faae5a19 None admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:36:15.799204 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=100 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:36:15.807145 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc63d63-7b44-42c1-973f-c1d6729965a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:36:15.807779 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc63d63-7b44-42c1-973f-c1d6729965a0 None admin] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:36:15.808685 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc63d63-7b44-42c1-973f-c1d6729965a0 None admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:36:15.809405 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc63d63-7b44-42c1-973f-c1d6729965a0 None admin] PATH_INFO: `/v3/projects/b39f712d51cc4ea6af519827d37767b7` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:36:15.810548 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bc63d63-7b44-42c1-973f-c1d6729965a0 None admin] RBAC: Authorizing `identity:get_project(project_id=b39f712d51cc4ea6af519827d37767b7)` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:36:15.811966 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bc63d63-7b44-42c1-973f-c1d6729965a0 None admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:36:15.813630 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 574/1152] 10.222.0.13 () {56 vars in 1186 bytes} [Wed Aug 31 09:36:15 2022] GET /identity/v3/projects/b39f712d51cc4ea6af519827d37767b7 => generated 311 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:36:15.823276 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ba129f-947d-4f89-9b4c-e11f5afb6afc None admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:36:15.994712 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ba129f-947d-4f89-9b4c-e11f5afb6afc None admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:36:16.056484 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f8a7ca-c027-4abe-aefd-f49c5d2609a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:36:16.057674 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f8a7ca-c027-4abe-aefd-f49c5d2609a7 None admin] REQUEST_METHOD: `PUT` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:36:16.058263 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f8a7ca-c027-4abe-aefd-f49c5d2609a7 None admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:36:16.058746 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f8a7ca-c027-4abe-aefd-f49c5d2609a7 None admin] PATH_INFO: `/v3/projects/b39f712d51cc4ea6af519827d37767b7/users/3981f3f67ae7467f8d159e7cf6d730fd/roles/5d4ee63a3add47fc9a6b8b16d39e0a1f` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:36:16.062080 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2f8a7ca-c027-4abe-aefd-f49c5d2609a7 None admin] RBAC: Authorizing `identity:create_grant(project_id=b39f712d51cc4ea6af519827d37767b7, user_id=3981f3f67ae7467f8d159e7cf6d730fd, role_id=5d4ee63a3add47fc9a6b8b16d39e0a1f)` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:36:16.064583 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2f8a7ca-c027-4abe-aefd-f49c5d2609a7 None admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:36:16.082506 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 579/1153] 10.222.0.13 () {58 vars in 1439 bytes} [Wed Aug 31 09:36:15 2022] PUT /identity/v3/projects/b39f712d51cc4ea6af519827d37767b7/users/3981f3f67ae7467f8d159e7cf6d730fd/roles/5d4ee63a3add47fc9a6b8b16d39e0a1f => generated 0 bytes in 260 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:36:18.047685 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccdb9658-8456-472d-a1fe-61c7fb6bb347 None None] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:36:18.048219 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccdb9658-8456-472d-a1fe-61c7fb6bb347 None None] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:36:18.048219 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccdb9658-8456-472d-a1fe-61c7fb6bb347 None None] PATH_INFO: `/` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:36:18.049730 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 575/1154] 10.222.0.13 () {54 vars in 927 bytes} [Wed Aug 31 09:36:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:36:18.055178 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29419bff-2b0a-423a-8a22-74541046ca51 None None] REQUEST_METHOD: `POST` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:36:18.055604 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29419bff-2b0a-423a-8a22-74541046ca51 None None] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:36:18.055703 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29419bff-2b0a-423a-8a22-74541046ca51 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:36:18.075483 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.auth.core [None req-29419bff-2b0a-423a-8a22-74541046ca51 None None] MFA Rules not processed for user `fd836cf590a145df9acf19cf3ae2ac56`. Rule list: `[]` (Enabled: `True`). {{(pid=88113) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:36:18.076638 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.fernet_utils [None req-29419bff-2b0a-423a-8a22-74541046ca51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:36:18.108840 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:36:18.149137 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 580/1155] 10.222.0.13 () {58 vars in 1024 bytes} [Wed Aug 31 09:36:18 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 09:36:18.420232 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccdb9658-8456-472d-a1fe-61c7fb6bb347 None None] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:36:18.543546 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccdb9658-8456-472d-a1fe-61c7fb6bb347 None None] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:36:18.591675 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-397e03bb-5ce0-4a38-adcc-9089616340d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:36:18.592339 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397e03bb-5ce0-4a38-adcc-9089616340d1 None admin] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:36:18.592619 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397e03bb-5ce0-4a38-adcc-9089616340d1 None admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:36:18.592890 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397e03bb-5ce0-4a38-adcc-9089616340d1 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:36:18.604766 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-397e03bb-5ce0-4a38-adcc-9089616340d1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 09:36:18.610915 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application [None req-397e03bb-5ce0-4a38-adcc-9089616340d1 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 09:36:18.610915 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:36:18.610915 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:36:18.610915 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:36:18.610915 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:36:18.610915 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:36:18.610915 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:36:18.610915 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:36:18.610915 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:36:18.610915 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:36:18.610915 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:36:18.610915 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:36:18.610915 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 09:36:18.610915 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 09:36:18.610915 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 09:36:18.610915 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 09:36:18.612175 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 09:36:18.612175 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 09:36:18.612175 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 09:36:18.612175 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 09:36:18.612175 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:36:18.612175 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:36:18.612175 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:36:18.612175 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:36:18.612175 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:36:18.612175 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:36:18.612175 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:36:18.612175 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application with Lock( Aug 31 09:36:18.612175 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:36:18.612175 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:36:18.612175 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:36:18.612175 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:36:18.613059 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:36:18.613059 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:36:18.613059 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:36:18.613059 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:36:18.613059 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 09:36:18.613059 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 09:36:18.613059 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 09:36:18.613059 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 09:36:18.613059 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 09:36:18.613059 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 09:36:18.613059 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 09:36:18.613059 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application Aug 31 09:36:18.613059 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 576/1156] 10.222.0.13 () {56 vars in 1096 bytes} [Wed Aug 31 09:36:18 2022] GET /identity/v3/roles/admin => generated 83 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:36:18.621280 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29419bff-2b0a-423a-8a22-74541046ca51 None None] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:36:18.759803 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29419bff-2b0a-423a-8a22-74541046ca51 None None] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:36:18.824126 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c3b4a09-dde6-464e-a434-2c2846bfaff1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:36:18.825131 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3b4a09-dde6-464e-a434-2c2846bfaff1 None admin] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:36:18.825476 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3b4a09-dde6-464e-a434-2c2846bfaff1 None admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:36:18.825757 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3b4a09-dde6-464e-a434-2c2846bfaff1 None admin] PATH_INFO: `/v3/roles` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:36:18.826485 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c3b4a09-dde6-464e-a434-2c2846bfaff1 None admin] RBAC: Adding query filter params (name=admin) {{(pid=88113) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:36:18.826794 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c3b4a09-dde6-464e-a434-2c2846bfaff1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:36:18.828297 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c3b4a09-dde6-464e-a434-2c2846bfaff1 None admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:36:18.839013 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 581/1157] 10.222.0.13 () {56 vars in 1099 bytes} [Wed Aug 31 09:36:18 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:36:18.846954 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-397e03bb-5ce0-4a38-adcc-9089616340d1 None admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:36:18.941589 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=100 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:36:18.998669 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-397e03bb-5ce0-4a38-adcc-9089616340d1 None admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:36:19.045481 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7cf3b6e-e589-4831-9f5b-7ee3160d1e4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:36:19.046003 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7cf3b6e-e589-4831-9f5b-7ee3160d1e4b None admin] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:36:19.046279 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7cf3b6e-e589-4831-9f5b-7ee3160d1e4b None admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:36:19.046546 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7cf3b6e-e589-4831-9f5b-7ee3160d1e4b None admin] PATH_INFO: `/v3/users/3981f3f67ae7467f8d159e7cf6d730fd` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:36:19.047784 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7cf3b6e-e589-4831-9f5b-7ee3160d1e4b None admin] RBAC: Authorizing `identity:get_user(user_id=3981f3f67ae7467f8d159e7cf6d730fd)` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:36:19.049237 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7cf3b6e-e589-4831-9f5b-7ee3160d1e4b None admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:36:19.050923 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 577/1158] 10.222.0.13 () {56 vars in 1177 bytes} [Wed Aug 31 09:36:18 2022] GET /identity/v3/users/3981f3f67ae7467f8d159e7cf6d730fd => generated 286 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:36:19.058404 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c3b4a09-dde6-464e-a434-2c2846bfaff1 None admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:36:19.095327 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=100 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:36:19.174450 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c3b4a09-dde6-464e-a434-2c2846bfaff1 None admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:36:19.210662 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b91d314-8e16-4ae7-9310-98268a9c5c49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:36:19.211393 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b91d314-8e16-4ae7-9310-98268a9c5c49 None admin] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:36:19.211755 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b91d314-8e16-4ae7-9310-98268a9c5c49 None admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:36:19.212109 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b91d314-8e16-4ae7-9310-98268a9c5c49 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:36:19.290964 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b91d314-8e16-4ae7-9310-98268a9c5c49 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:36:19.292682 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b91d314-8e16-4ae7-9310-98268a9c5c49 None admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:36:19.324748 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=100 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:36:19.404518 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 582/1159] 10.222.0.13 () {58 vars in 1282 bytes} [Wed Aug 31 09:36:19 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 347 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 31 09:36:19.410772 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7cf3b6e-e589-4831-9f5b-7ee3160d1e4b None admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:36:19.520640 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7cf3b6e-e589-4831-9f5b-7ee3160d1e4b None admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:36:19.550286 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=100 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:36:19.563457 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4cdc451-4c75-470f-b86c-3daf63775420 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:36:19.564112 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4cdc451-4c75-470f-b86c-3daf63775420 None admin] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:36:19.564385 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4cdc451-4c75-470f-b86c-3daf63775420 None admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:36:19.564658 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4cdc451-4c75-470f-b86c-3daf63775420 None admin] PATH_INFO: `/v3/projects/b39f712d51cc4ea6af519827d37767b7` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:36:19.565876 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4cdc451-4c75-470f-b86c-3daf63775420 None admin] RBAC: Authorizing `identity:get_project(project_id=b39f712d51cc4ea6af519827d37767b7)` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:36:19.567361 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4cdc451-4c75-470f-b86c-3daf63775420 None admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:36:19.569104 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 578/1160] 10.222.0.13 () {56 vars in 1186 bytes} [Wed Aug 31 09:36:19 2022] GET /identity/v3/projects/b39f712d51cc4ea6af519827d37767b7 => generated 311 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:36:19.575796 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b91d314-8e16-4ae7-9310-98268a9c5c49 None admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:36:19.708290 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b91d314-8e16-4ae7-9310-98268a9c5c49 None admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:36:19.742755 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=100 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:36:19.749907 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-534a866e-b7b5-44d0-a7cc-cd04cf4b338c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:36:19.750536 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534a866e-b7b5-44d0-a7cc-cd04cf4b338c None admin] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:36:19.750819 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534a866e-b7b5-44d0-a7cc-cd04cf4b338c None admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:36:19.751086 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534a866e-b7b5-44d0-a7cc-cd04cf4b338c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:36:19.751827 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-534a866e-b7b5-44d0-a7cc-cd04cf4b338c None admin] RBAC: Adding query filter params (role.id=5d4ee63a3add47fc9a6b8b16d39e0a1f, scope.project.id=b39f712d51cc4ea6af519827d37767b7, user.id=3981f3f67ae7467f8d159e7cf6d730fd) {{(pid=88113) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:36:19.752134 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-534a866e-b7b5-44d0-a7cc-cd04cf4b338c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:36:19.753637 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-534a866e-b7b5-44d0-a7cc-cd04cf4b338c None admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:36:19.760374 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 583/1161] 10.222.0.13 () {56 vars in 1374 bytes} [Wed Aug 31 09:36:19 2022] GET /identity/v3/role_assignments?user.id=3981f3f67ae7467f8d159e7cf6d730fd&scope.project.id=b39f712d51cc4ea6af519827d37767b7&role.id=5d4ee63a3add47fc9a6b8b16d39e0a1f => generated 608 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:36:21.741796 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f46276-f681-4082-8407-428606abb363 None None] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:36:21.742458 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f46276-f681-4082-8407-428606abb363 None None] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:36:21.742458 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f46276-f681-4082-8407-428606abb363 None None] PATH_INFO: `/` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:36:21.743652 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 579/1162] 10.222.0.13 () {54 vars in 927 bytes} [Wed Aug 31 09:36:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:36:21.749141 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55642a48-73f0-4f8b-8f2f-7036f264a343 None None] REQUEST_METHOD: `POST` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:36:21.749686 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55642a48-73f0-4f8b-8f2f-7036f264a343 None None] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:36:21.749686 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55642a48-73f0-4f8b-8f2f-7036f264a343 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:36:21.771274 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.auth.core [None req-55642a48-73f0-4f8b-8f2f-7036f264a343 None None] MFA Rules not processed for user `fd836cf590a145df9acf19cf3ae2ac56`. Rule list: `[]` (Enabled: `True`). {{(pid=88113) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:36:21.772468 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.fernet_utils [None req-55642a48-73f0-4f8b-8f2f-7036f264a343 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:36:21.843676 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 584/1163] 10.222.0.13 () {58 vars in 1024 bytes} [Wed Aug 31 09:36:21 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 09:36:22.118666 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f46276-f681-4082-8407-428606abb363 None None] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:36:22.227923 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f46276-f681-4082-8407-428606abb363 None None] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:36:22.263525 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c975bec-0229-4074-9e9e-20d831dbaa3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:36:22.264136 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c975bec-0229-4074-9e9e-20d831dbaa3b None admin] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:36:22.264409 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c975bec-0229-4074-9e9e-20d831dbaa3b None admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:36:22.264679 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c975bec-0229-4074-9e9e-20d831dbaa3b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:36:22.327360 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=100 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:36:22.341706 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c975bec-0229-4074-9e9e-20d831dbaa3b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:36:22.343176 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c975bec-0229-4074-9e9e-20d831dbaa3b None admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:36:22.452851 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 580/1164] 10.222.0.13 () {58 vars in 1282 bytes} [Wed Aug 31 09:36:22 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 335 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 31 09:36:22.459231 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55642a48-73f0-4f8b-8f2f-7036f264a343 None None] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:36:22.578476 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55642a48-73f0-4f8b-8f2f-7036f264a343 None None] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:36:22.610894 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=100 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:36:22.623952 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aeb35d0-d655-4e1e-ab36-8855363de502 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:36:22.624620 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aeb35d0-d655-4e1e-ab36-8855363de502 None admin] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:36:22.624897 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aeb35d0-d655-4e1e-ab36-8855363de502 None admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:36:22.625167 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aeb35d0-d655-4e1e-ab36-8855363de502 None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:36:22.633289 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aeb35d0-d655-4e1e-ab36-8855363de502 None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:36:22.634835 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aeb35d0-d655-4e1e-ab36-8855363de502 None admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:36:22.649120 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application [None req-9aeb35d0-d655-4e1e-ab36-8855363de502 None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. Aug 31 09:36:22.649120 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:36:22.649120 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Aug 31 09:36:22.649120 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Aug 31 09:36:22.649120 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 31 09:36:22.649120 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 31 09:36:22.649120 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 31 09:36:22.649120 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 31 09:36:22.649120 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. Aug 31 09:36:22.649120 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application Aug 31 09:36:22.649120 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 31 09:36:22.649120 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application Aug 31 09:36:22.649120 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:36:22.649120 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:36:22.649120 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:36:22.649120 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:36:22.649120 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:36:22.650346 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:36:22.650346 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:36:22.650346 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:36:22.650346 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:36:22.650346 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:36:22.650346 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:36:22.650346 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 09:36:22.650346 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 09:36:22.650346 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 09:36:22.650346 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 09:36:22.650346 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:36:22.650346 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:36:22.650346 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:36:22.650346 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:36:22.650346 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:36:22.650346 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:36:22.651955 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:36:22.651955 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application with Lock( Aug 31 09:36:22.651955 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:36:22.651955 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:36:22.651955 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:36:22.651955 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:36:22.651955 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:36:22.651955 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:36:22.651955 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:36:22.651955 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:36:22.651955 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Aug 31 09:36:22.651955 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 09:36:22.651955 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. Aug 31 09:36:22.651955 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application Aug 31 09:36:22.651955 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 585/1165] 10.222.0.13 () {56 vars in 1117 bytes} [Wed Aug 31 09:36:22 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:36:22.658606 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c975bec-0229-4074-9e9e-20d831dbaa3b None admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:36:22.758409 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=100 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:36:22.768488 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c975bec-0229-4074-9e9e-20d831dbaa3b None admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:36:22.805468 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-451c7e8a-45d8-4fe3-96c3-88516de028d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:36:22.806082 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451c7e8a-45d8-4fe3-96c3-88516de028d6 None admin] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:36:22.806362 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451c7e8a-45d8-4fe3-96c3-88516de028d6 None admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:36:22.806630 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451c7e8a-45d8-4fe3-96c3-88516de028d6 None admin] PATH_INFO: `/v3/domains` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:36:22.807393 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-451c7e8a-45d8-4fe3-96c3-88516de028d6 None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=88114) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:36:22.807732 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-451c7e8a-45d8-4fe3-96c3-88516de028d6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:36:22.809111 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-451c7e8a-45d8-4fe3-96c3-88516de028d6 None admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:36:22.816770 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 581/1166] 10.222.0.13 () {56 vars in 1115 bytes} [Wed Aug 31 09:36:22 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:36:24.771359 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049dd818-6b95-4209-bc35-1ed1dd93963b None None] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:36:24.771932 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049dd818-6b95-4209-bc35-1ed1dd93963b None None] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:36:24.771932 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049dd818-6b95-4209-bc35-1ed1dd93963b None None] PATH_INFO: `/` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:36:24.773219 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 586/1167] 10.222.0.13 () {54 vars in 927 bytes} [Wed Aug 31 09:36:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:36:24.779457 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f483a8-5c7d-4ec6-8e26-dc3995d3b542 None None] REQUEST_METHOD: `POST` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:36:24.780171 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f483a8-5c7d-4ec6-8e26-dc3995d3b542 None None] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:36:24.780361 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f483a8-5c7d-4ec6-8e26-dc3995d3b542 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:36:24.799094 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.auth.core [None req-f1f483a8-5c7d-4ec6-8e26-dc3995d3b542 None None] MFA Rules not processed for user `fd836cf590a145df9acf19cf3ae2ac56`. Rule list: `[]` (Enabled: `True`). {{(pid=88114) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:36:24.800295 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.fernet_utils [None req-f1f483a8-5c7d-4ec6-8e26-dc3995d3b542 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88114) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:36:24.868640 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 582/1168] 10.222.0.13 () {58 vars in 1024 bytes} [Wed Aug 31 09:36:24 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 09:36:25.139728 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-049dd818-6b95-4209-bc35-1ed1dd93963b None None] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:36:25.266504 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-049dd818-6b95-4209-bc35-1ed1dd93963b None None] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:36:25.315057 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aea2444-5b1c-406d-b190-59d7a6dd2d90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:36:25.316104 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aea2444-5b1c-406d-b190-59d7a6dd2d90 None admin] REQUEST_METHOD: `POST` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:36:25.316608 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aea2444-5b1c-406d-b190-59d7a6dd2d90 None admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:36:25.317171 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aea2444-5b1c-406d-b190-59d7a6dd2d90 None admin] PATH_INFO: `/v3/domains` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:36:25.318762 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aea2444-5b1c-406d-b190-59d7a6dd2d90 None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:36:25.321550 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aea2444-5b1c-406d-b190-59d7a6dd2d90 None admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:36:25.344021 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 587/1169] 10.222.0.13 () {60 vars in 1138 bytes} [Wed Aug 31 09:36:25 2022] POST /identity/v3/domains => generated 267 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 09:36:27.380385 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3719468f-ee7a-4834-a3e7-4fd40e439f78 None None] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:36:27.380891 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3719468f-ee7a-4834-a3e7-4fd40e439f78 None None] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:36:27.380891 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3719468f-ee7a-4834-a3e7-4fd40e439f78 None None] PATH_INFO: `/` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:36:27.382230 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 583/1170] 10.222.0.13 () {54 vars in 927 bytes} [Wed Aug 31 09:36:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:36:27.388782 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44e74a5-7337-4dee-8ccd-ed11112e1f2e None None] REQUEST_METHOD: `POST` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:36:27.389232 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44e74a5-7337-4dee-8ccd-ed11112e1f2e None None] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:36:27.389341 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44e74a5-7337-4dee-8ccd-ed11112e1f2e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:36:27.408794 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.auth.core [None req-e44e74a5-7337-4dee-8ccd-ed11112e1f2e None None] MFA Rules not processed for user `fd836cf590a145df9acf19cf3ae2ac56`. Rule list: `[]` (Enabled: `True`). {{(pid=88113) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:36:27.410072 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.fernet_utils [None req-e44e74a5-7337-4dee-8ccd-ed11112e1f2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:36:27.452679 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:36:27.481254 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 588/1171] 10.222.0.13 () {58 vars in 1024 bytes} [Wed Aug 31 09:36:27 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 09:36:27.760869 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3719468f-ee7a-4834-a3e7-4fd40e439f78 None None] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:36:27.852029 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=100 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:36:27.868754 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3719468f-ee7a-4834-a3e7-4fd40e439f78 None None] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:36:27.907656 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-381db5c3-4062-45c1-b647-db9b42c02738 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:36:27.908295 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381db5c3-4062-45c1-b647-db9b42c02738 None admin] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:36:27.908572 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381db5c3-4062-45c1-b647-db9b42c02738 None admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:36:27.908842 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381db5c3-4062-45c1-b647-db9b42c02738 None admin] PATH_INFO: `/v3/domains/d9782f071887440784a1917aa8501dfa` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:36:27.921245 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-381db5c3-4062-45c1-b647-db9b42c02738 None admin] RBAC: Authorizing `identity:get_domain(domain_id=d9782f071887440784a1917aa8501dfa)` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:36:27.922736 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-381db5c3-4062-45c1-b647-db9b42c02738 None admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:36:27.924429 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 584/1172] 10.222.0.13 () {56 vars in 1183 bytes} [Wed Aug 31 09:36:27 2022] GET /identity/v3/domains/d9782f071887440784a1917aa8501dfa => generated 267 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:36:27.930362 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e44e74a5-7337-4dee-8ccd-ed11112e1f2e None None] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:36:28.031921 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e44e74a5-7337-4dee-8ccd-ed11112e1f2e None None] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:36:28.066860 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fb0dbd4-ea13-40b8-bc07-2898baf430fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:36:28.067478 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb0dbd4-ea13-40b8-bc07-2898baf430fc None admin] REQUEST_METHOD: `POST` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:36:28.067756 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb0dbd4-ea13-40b8-bc07-2898baf430fc None admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:36:28.068029 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb0dbd4-ea13-40b8-bc07-2898baf430fc None admin] PATH_INFO: `/v3/projects` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:36:28.068685 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fb0dbd4-ea13-40b8-bc07-2898baf430fc None admin] RBAC: Authorizing `identity:create_project()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:36:28.070183 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fb0dbd4-ea13-40b8-bc07-2898baf430fc None admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:36:28.101183 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 589/1173] 10.222.0.13 () {60 vars in 1141 bytes} [Wed Aug 31 09:36:27 2022] POST /identity/v3/projects => generated 361 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 09:36:30.072557 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0536fb1c-27a3-4060-910e-75f7ce865301 None None] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:36:30.073205 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0536fb1c-27a3-4060-910e-75f7ce865301 None None] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:36:30.073299 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0536fb1c-27a3-4060-910e-75f7ce865301 None None] PATH_INFO: `/` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:36:30.075271 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 585/1174] 10.222.0.13 () {54 vars in 927 bytes} [Wed Aug 31 09:36:30 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:36:30.080998 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad51afc-4366-49a2-bb7f-5d3e67d5ebfe None None] REQUEST_METHOD: `POST` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:36:30.081524 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad51afc-4366-49a2-bb7f-5d3e67d5ebfe None None] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:36:30.081649 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad51afc-4366-49a2-bb7f-5d3e67d5ebfe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:36:30.100984 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.auth.core [None req-3ad51afc-4366-49a2-bb7f-5d3e67d5ebfe None None] MFA Rules not processed for user `fd836cf590a145df9acf19cf3ae2ac56`. Rule list: `[]` (Enabled: `True`). {{(pid=88113) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:36:30.102204 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.fernet_utils [None req-3ad51afc-4366-49a2-bb7f-5d3e67d5ebfe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:36:30.123833 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:36:30.174957 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 590/1175] 10.222.0.13 () {58 vars in 1024 bytes} [Wed Aug 31 09:36:30 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 31 09:36:30.444457 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0536fb1c-27a3-4060-910e-75f7ce865301 None None] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:36:30.548550 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0536fb1c-27a3-4060-910e-75f7ce865301 None None] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:36:30.584980 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d432ec2f-40b2-4924-b6e5-9e5d6f2010fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:36:30.585630 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d432ec2f-40b2-4924-b6e5-9e5d6f2010fc None admin] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:36:30.585905 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d432ec2f-40b2-4924-b6e5-9e5d6f2010fc None admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:36:30.586174 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d432ec2f-40b2-4924-b6e5-9e5d6f2010fc None admin] PATH_INFO: `/v3/domains/d9782f071887440784a1917aa8501dfa` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:36:30.587278 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d432ec2f-40b2-4924-b6e5-9e5d6f2010fc None admin] RBAC: Authorizing `identity:get_domain(domain_id=d9782f071887440784a1917aa8501dfa)` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:36:30.589362 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d432ec2f-40b2-4924-b6e5-9e5d6f2010fc None admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:36:30.590958 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 586/1176] 10.222.0.13 () {56 vars in 1183 bytes} [Wed Aug 31 09:36:30 2022] GET /identity/v3/domains/d9782f071887440784a1917aa8501dfa => generated 267 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:36:30.596855 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad51afc-4366-49a2-bb7f-5d3e67d5ebfe None None] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:36:30.702534 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad51afc-4366-49a2-bb7f-5d3e67d5ebfe None None] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:36:30.738350 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d87ab10d-5d98-431a-bb76-10507dd554b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:36:30.738957 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87ab10d-5d98-431a-bb76-10507dd554b5 None admin] REQUEST_METHOD: `POST` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:36:30.739226 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87ab10d-5d98-431a-bb76-10507dd554b5 None admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:36:30.739491 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87ab10d-5d98-431a-bb76-10507dd554b5 None admin] PATH_INFO: `/v3/users` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:36:30.740133 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d87ab10d-5d98-431a-bb76-10507dd554b5 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:36:30.741593 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d87ab10d-5d98-431a-bb76-10507dd554b5 None admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:36:30.764722 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 591/1177] 10.222.0.13 () {60 vars in 1132 bytes} [Wed Aug 31 09:36:30 2022] POST /identity/v3/users => generated 311 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 09:36:32.759214 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c92502-7c7a-4714-bd8b-a58f9c98df68 None None] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:36:32.759823 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c92502-7c7a-4714-bd8b-a58f9c98df68 None None] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:36:32.759823 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c92502-7c7a-4714-bd8b-a58f9c98df68 None None] PATH_INFO: `/` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:36:32.761166 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 587/1178] 10.222.0.13 () {54 vars in 927 bytes} [Wed Aug 31 09:36:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:36:32.767638 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c45014-0494-4627-b0d5-14d482b8fd6e None None] REQUEST_METHOD: `POST` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:36:32.768226 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c45014-0494-4627-b0d5-14d482b8fd6e None None] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:36:32.768226 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c45014-0494-4627-b0d5-14d482b8fd6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:36:32.787543 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.auth.core [None req-a3c45014-0494-4627-b0d5-14d482b8fd6e None None] MFA Rules not processed for user `fd836cf590a145df9acf19cf3ae2ac56`. Rule list: `[]` (Enabled: `True`). {{(pid=88113) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:36:32.788761 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.fernet_utils [None req-a3c45014-0494-4627-b0d5-14d482b8fd6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:36:32.798545 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:36:32.868005 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 592/1179] 10.222.0.13 () {58 vars in 1024 bytes} [Wed Aug 31 09:36:32 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 09:36:33.137792 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c92502-7c7a-4714-bd8b-a58f9c98df68 None None] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:36:33.154611 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=100 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:36:33.244346 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c92502-7c7a-4714-bd8b-a58f9c98df68 None None] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:36:33.282729 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed929cf-3e7c-43a2-81d4-6469c30b3b32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:36:33.283367 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed929cf-3e7c-43a2-81d4-6469c30b3b32 None admin] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:36:33.283646 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed929cf-3e7c-43a2-81d4-6469c30b3b32 None admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:36:33.283913 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed929cf-3e7c-43a2-81d4-6469c30b3b32 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:36:33.296642 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ed929cf-3e7c-43a2-81d4-6469c30b3b32 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 09:36:33.301937 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application [None req-8ed929cf-3e7c-43a2-81d4-6469c30b3b32 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 09:36:33.301937 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:36:33.301937 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:36:33.301937 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:36:33.301937 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:36:33.301937 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:36:33.301937 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:36:33.301937 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:36:33.301937 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:36:33.301937 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:36:33.301937 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:36:33.301937 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:36:33.301937 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 09:36:33.301937 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 09:36:33.301937 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 09:36:33.301937 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 09:36:33.303052 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 09:36:33.303052 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 09:36:33.303052 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 09:36:33.303052 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 09:36:33.303052 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:36:33.303052 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:36:33.303052 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:36:33.303052 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:36:33.303052 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:36:33.303052 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:36:33.303052 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:36:33.303052 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application with Lock( Aug 31 09:36:33.303052 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:36:33.303052 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:36:33.303052 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:36:33.303052 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:36:33.304235 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:36:33.304235 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:36:33.304235 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:36:33.304235 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:36:33.304235 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 09:36:33.304235 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 09:36:33.304235 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 09:36:33.304235 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 09:36:33.304235 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 09:36:33.304235 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 09:36:33.304235 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 09:36:33.304235 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application Aug 31 09:36:33.304235 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 588/1180] 10.222.0.13 () {56 vars in 1096 bytes} [Wed Aug 31 09:36:33 2022] GET /identity/v3/roles/admin => generated 83 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:36:33.309433 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c45014-0494-4627-b0d5-14d482b8fd6e None None] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:36:33.410732 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c45014-0494-4627-b0d5-14d482b8fd6e None None] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:36:33.443967 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb26617-c341-48d1-8e96-0fd033e0830e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:36:33.444564 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb26617-c341-48d1-8e96-0fd033e0830e None admin] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:36:33.444837 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb26617-c341-48d1-8e96-0fd033e0830e None admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:36:33.445104 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb26617-c341-48d1-8e96-0fd033e0830e None admin] PATH_INFO: `/v3/roles` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:36:33.445792 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eb26617-c341-48d1-8e96-0fd033e0830e None admin] RBAC: Adding query filter params (name=admin) {{(pid=88113) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:36:33.446088 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eb26617-c341-48d1-8e96-0fd033e0830e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:36:33.447439 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eb26617-c341-48d1-8e96-0fd033e0830e None admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:36:33.455949 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 593/1181] 10.222.0.13 () {56 vars in 1099 bytes} [Wed Aug 31 09:36:33 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:36:33.461909 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed929cf-3e7c-43a2-81d4-6469c30b3b32 None admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:36:33.538282 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=100 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:36:33.564258 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed929cf-3e7c-43a2-81d4-6469c30b3b32 None admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:36:33.600074 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c58080-0fac-4a8b-b543-b13394bcb577 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:36:33.600703 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c58080-0fac-4a8b-b543-b13394bcb577 None admin] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:36:33.600982 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c58080-0fac-4a8b-b543-b13394bcb577 None admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:36:33.601252 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c58080-0fac-4a8b-b543-b13394bcb577 None admin] PATH_INFO: `/v3/users/1400b17335734644b81746547847b228` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:36:33.614710 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5c58080-0fac-4a8b-b543-b13394bcb577 None admin] RBAC: Authorizing `identity:get_user(user_id=1400b17335734644b81746547847b228)` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:36:33.616153 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5c58080-0fac-4a8b-b543-b13394bcb577 None admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:36:33.617814 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 589/1182] 10.222.0.13 () {56 vars in 1177 bytes} [Wed Aug 31 09:36:33 2022] GET /identity/v3/users/1400b17335734644b81746547847b228 => generated 311 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:36:33.624140 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb26617-c341-48d1-8e96-0fd033e0830e None admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:36:33.627961 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=100 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:36:33.727332 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb26617-c341-48d1-8e96-0fd033e0830e None admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:36:33.760076 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b337cba-3d1f-4043-9796-8c9b9d10f27a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:36:33.760750 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b337cba-3d1f-4043-9796-8c9b9d10f27a None admin] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:36:33.761021 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b337cba-3d1f-4043-9796-8c9b9d10f27a None admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:36:33.761284 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b337cba-3d1f-4043-9796-8c9b9d10f27a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:36:33.827662 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b337cba-3d1f-4043-9796-8c9b9d10f27a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:36:33.829144 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b337cba-3d1f-4043-9796-8c9b9d10f27a None admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:36:33.832267 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=100 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:36:33.930109 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 594/1183] 10.222.0.13 () {58 vars in 1282 bytes} [Wed Aug 31 09:36:33 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 31 09:36:33.936320 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c58080-0fac-4a8b-b543-b13394bcb577 None admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:36:34.035416 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c58080-0fac-4a8b-b543-b13394bcb577 None admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:36:34.064717 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=100 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:36:34.070134 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c373d0-f193-4c5c-a0c4-fda387e4b7bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:36:34.070761 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c373d0-f193-4c5c-a0c4-fda387e4b7bd None admin] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:36:34.071034 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c373d0-f193-4c5c-a0c4-fda387e4b7bd None admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:36:34.071302 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c373d0-f193-4c5c-a0c4-fda387e4b7bd None admin] PATH_INFO: `/v3/projects/bb9b6d7419c446ffa65390a4a9246b83` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:36:34.072408 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53c373d0-f193-4c5c-a0c4-fda387e4b7bd None admin] RBAC: Authorizing `identity:get_project(project_id=bb9b6d7419c446ffa65390a4a9246b83)` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:36:34.074452 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53c373d0-f193-4c5c-a0c4-fda387e4b7bd None admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:36:34.076094 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 590/1184] 10.222.0.13 () {56 vars in 1186 bytes} [Wed Aug 31 09:36:33 2022] GET /identity/v3/projects/bb9b6d7419c446ffa65390a4a9246b83 => generated 361 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:36:34.082304 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b337cba-3d1f-4043-9796-8c9b9d10f27a None admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:36:34.180451 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b337cba-3d1f-4043-9796-8c9b9d10f27a None admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:36:34.183006 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=100 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:36:34.216173 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-878e006b-4723-4c29-8ff6-4de2afccab47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:36:34.216782 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878e006b-4723-4c29-8ff6-4de2afccab47 None admin] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:36:34.217062 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878e006b-4723-4c29-8ff6-4de2afccab47 None admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:36:34.217335 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878e006b-4723-4c29-8ff6-4de2afccab47 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:36:34.218091 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-878e006b-4723-4c29-8ff6-4de2afccab47 None admin] RBAC: Adding query filter params (role.id=5d4ee63a3add47fc9a6b8b16d39e0a1f, scope.project.id=bb9b6d7419c446ffa65390a4a9246b83, user.id=1400b17335734644b81746547847b228) {{(pid=88113) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:36:34.218400 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-878e006b-4723-4c29-8ff6-4de2afccab47 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:36:34.219851 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-878e006b-4723-4c29-8ff6-4de2afccab47 None admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:36:34.225200 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 595/1185] 10.222.0.13 () {56 vars in 1374 bytes} [Wed Aug 31 09:36:34 2022] GET /identity/v3/role_assignments?user.id=1400b17335734644b81746547847b228&scope.project.id=bb9b6d7419c446ffa65390a4a9246b83&role.id=5d4ee63a3add47fc9a6b8b16d39e0a1f => generated 259 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:36:36.126795 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1306599a-15fa-44cc-98fa-fe0fdfd36197 None None] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:36:36.127338 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1306599a-15fa-44cc-98fa-fe0fdfd36197 None None] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:36:36.127338 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1306599a-15fa-44cc-98fa-fe0fdfd36197 None None] PATH_INFO: `/` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:36:36.128656 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 591/1186] 10.222.0.13 () {54 vars in 927 bytes} [Wed Aug 31 09:36:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:36:36.133753 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5f6f4f-89a2-48c0-9f0c-a1fdcdb0f98b None None] REQUEST_METHOD: `POST` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:36:36.134199 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5f6f4f-89a2-48c0-9f0c-a1fdcdb0f98b None None] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:36:36.134307 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5f6f4f-89a2-48c0-9f0c-a1fdcdb0f98b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:36:36.153686 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.auth.core [None req-9f5f6f4f-89a2-48c0-9f0c-a1fdcdb0f98b None None] MFA Rules not processed for user `fd836cf590a145df9acf19cf3ae2ac56`. Rule list: `[]` (Enabled: `True`). {{(pid=88113) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:36:36.154862 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.fernet_utils [None req-9f5f6f4f-89a2-48c0-9f0c-a1fdcdb0f98b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:36:36.220897 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 596/1187] 10.222.0.13 () {58 vars in 1024 bytes} [Wed Aug 31 09:36:36 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 09:36:36.490451 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1306599a-15fa-44cc-98fa-fe0fdfd36197 None None] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:36:36.591908 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1306599a-15fa-44cc-98fa-fe0fdfd36197 None None] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:36:36.626215 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc723cc0-def2-4f53-ac24-e894b7f1694d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:36:36.626833 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc723cc0-def2-4f53-ac24-e894b7f1694d None admin] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:36:36.627109 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc723cc0-def2-4f53-ac24-e894b7f1694d None admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:36:36.627378 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc723cc0-def2-4f53-ac24-e894b7f1694d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:36:36.637144 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc723cc0-def2-4f53-ac24-e894b7f1694d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 09:36:36.642150 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application [None req-fc723cc0-def2-4f53-ac24-e894b7f1694d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 09:36:36.642150 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:36:36.642150 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:36:36.642150 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:36:36.642150 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:36:36.642150 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:36:36.642150 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:36:36.642150 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:36:36.642150 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:36:36.642150 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:36:36.642150 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:36:36.642150 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:36:36.642150 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 09:36:36.642150 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 09:36:36.642150 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 09:36:36.642150 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 09:36:36.643259 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 09:36:36.643259 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 09:36:36.643259 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 09:36:36.643259 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 09:36:36.643259 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:36:36.643259 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:36:36.643259 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:36:36.643259 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:36:36.643259 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:36:36.643259 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:36:36.643259 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:36:36.643259 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application with Lock( Aug 31 09:36:36.643259 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:36:36.643259 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:36:36.643259 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:36:36.643259 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:36:36.644442 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:36:36.644442 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:36:36.644442 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:36:36.644442 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:36:36.644442 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 09:36:36.644442 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 09:36:36.644442 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 09:36:36.644442 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 09:36:36.644442 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 09:36:36.644442 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 09:36:36.644442 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 09:36:36.644442 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application Aug 31 09:36:36.644442 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 592/1188] 10.222.0.13 () {56 vars in 1096 bytes} [Wed Aug 31 09:36:36 2022] GET /identity/v3/roles/admin => generated 83 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:36:36.649299 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f5f6f4f-89a2-48c0-9f0c-a1fdcdb0f98b None None] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:36:36.742216 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=100 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:36:36.746554 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f5f6f4f-89a2-48c0-9f0c-a1fdcdb0f98b None None] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:36:36.779974 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f55e2319-a740-44ab-9dd9-59de00aedd63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:36:36.780572 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55e2319-a740-44ab-9dd9-59de00aedd63 None admin] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:36:36.780846 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55e2319-a740-44ab-9dd9-59de00aedd63 None admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:36:36.781111 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55e2319-a740-44ab-9dd9-59de00aedd63 None admin] PATH_INFO: `/v3/roles` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:36:36.781800 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f55e2319-a740-44ab-9dd9-59de00aedd63 None admin] RBAC: Adding query filter params (name=admin) {{(pid=88113) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:36:36.782098 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f55e2319-a740-44ab-9dd9-59de00aedd63 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:36:36.783450 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f55e2319-a740-44ab-9dd9-59de00aedd63 None admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:36:36.791266 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 597/1189] 10.222.0.13 () {56 vars in 1099 bytes} [Wed Aug 31 09:36:36 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:36:36.798020 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc723cc0-def2-4f53-ac24-e894b7f1694d None admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:36:36.858524 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=100 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:36:36.900788 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc723cc0-def2-4f53-ac24-e894b7f1694d None admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:36:36.938093 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7377640a-97e0-49f1-ab6a-883b09cbf9c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:36:36.938733 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7377640a-97e0-49f1-ab6a-883b09cbf9c2 None admin] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:36:36.939012 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7377640a-97e0-49f1-ab6a-883b09cbf9c2 None admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:36:36.939282 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7377640a-97e0-49f1-ab6a-883b09cbf9c2 None admin] PATH_INFO: `/v3/users/1400b17335734644b81746547847b228` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:36:36.940417 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7377640a-97e0-49f1-ab6a-883b09cbf9c2 None admin] RBAC: Authorizing `identity:get_user(user_id=1400b17335734644b81746547847b228)` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:36:36.941870 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7377640a-97e0-49f1-ab6a-883b09cbf9c2 None admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:36:36.943461 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 593/1190] 10.222.0.13 () {56 vars in 1177 bytes} [Wed Aug 31 09:36:36 2022] GET /identity/v3/users/1400b17335734644b81746547847b228 => generated 311 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:36:36.950023 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f55e2319-a740-44ab-9dd9-59de00aedd63 None admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:36:37.049281 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f55e2319-a740-44ab-9dd9-59de00aedd63 None admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:36:37.084945 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bcb20af-ae96-4b17-8f79-ece0f7efde61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:36:37.085618 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bcb20af-ae96-4b17-8f79-ece0f7efde61 None admin] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:36:37.085931 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bcb20af-ae96-4b17-8f79-ece0f7efde61 None admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:36:37.086205 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bcb20af-ae96-4b17-8f79-ece0f7efde61 None admin] PATH_INFO: `/v3/projects/bb9b6d7419c446ffa65390a4a9246b83` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:36:37.087338 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bcb20af-ae96-4b17-8f79-ece0f7efde61 None admin] RBAC: Authorizing `identity:get_project(project_id=bb9b6d7419c446ffa65390a4a9246b83)` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:36:37.088777 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bcb20af-ae96-4b17-8f79-ece0f7efde61 None admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:36:37.090486 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 598/1191] 10.222.0.13 () {56 vars in 1186 bytes} [Wed Aug 31 09:36:36 2022] GET /identity/v3/projects/bb9b6d7419c446ffa65390a4a9246b83 => generated 361 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:36:37.096545 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7377640a-97e0-49f1-ab6a-883b09cbf9c2 None admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:36:37.200103 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7377640a-97e0-49f1-ab6a-883b09cbf9c2 None admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:36:37.226491 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=100 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:36:37.236005 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d37958-c40f-44c4-a3aa-e9338364e385 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:36:37.236684 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d37958-c40f-44c4-a3aa-e9338364e385 None admin] REQUEST_METHOD: `PUT` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:36:37.236959 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d37958-c40f-44c4-a3aa-e9338364e385 None admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:36:37.237227 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d37958-c40f-44c4-a3aa-e9338364e385 None admin] PATH_INFO: `/v3/projects/bb9b6d7419c446ffa65390a4a9246b83/users/1400b17335734644b81746547847b228/roles/5d4ee63a3add47fc9a6b8b16d39e0a1f` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:36:37.239133 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38d37958-c40f-44c4-a3aa-e9338364e385 None admin] RBAC: Authorizing `identity:create_grant(project_id=bb9b6d7419c446ffa65390a4a9246b83, user_id=1400b17335734644b81746547847b228, role_id=5d4ee63a3add47fc9a6b8b16d39e0a1f)` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:36:37.240553 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38d37958-c40f-44c4-a3aa-e9338364e385 None admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:36:37.253891 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 594/1192] 10.222.0.13 () {58 vars in 1439 bytes} [Wed Aug 31 09:36:37 2022] PUT /identity/v3/projects/bb9b6d7419c446ffa65390a4a9246b83/users/1400b17335734644b81746547847b228/roles/5d4ee63a3add47fc9a6b8b16d39e0a1f => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:36:39.153998 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7acc08a7-7deb-426b-9084-d05995486f0a None None] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:36:39.154528 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7acc08a7-7deb-426b-9084-d05995486f0a None None] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:36:39.154528 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7acc08a7-7deb-426b-9084-d05995486f0a None None] PATH_INFO: `/` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:36:39.155854 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 599/1193] 10.222.0.13 () {54 vars in 927 bytes} [Wed Aug 31 09:36:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:36:39.161146 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f45feb-8e39-4258-b22e-d4cf3c6ee432 None None] REQUEST_METHOD: `POST` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:36:39.161565 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f45feb-8e39-4258-b22e-d4cf3c6ee432 None None] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:36:39.161723 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f45feb-8e39-4258-b22e-d4cf3c6ee432 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:36:39.179764 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.auth.core [None req-20f45feb-8e39-4258-b22e-d4cf3c6ee432 None None] MFA Rules not processed for user `fd836cf590a145df9acf19cf3ae2ac56`. Rule list: `[]` (Enabled: `True`). {{(pid=88114) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:36:39.180952 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.fernet_utils [None req-20f45feb-8e39-4258-b22e-d4cf3c6ee432 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88114) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:36:39.247513 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 595/1194] 10.222.0.13 () {58 vars in 1024 bytes} [Wed Aug 31 09:36:39 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 09:36:39.514560 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7acc08a7-7deb-426b-9084-d05995486f0a None None] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:36:39.543223 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=100 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:36:39.615416 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7acc08a7-7deb-426b-9084-d05995486f0a None None] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:36:39.652163 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca37ee2-7455-4bea-91ec-2265d79a0375 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:36:39.652783 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca37ee2-7455-4bea-91ec-2265d79a0375 None admin] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:36:39.653056 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca37ee2-7455-4bea-91ec-2265d79a0375 None admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:36:39.653321 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca37ee2-7455-4bea-91ec-2265d79a0375 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:36:39.663392 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ca37ee2-7455-4bea-91ec-2265d79a0375 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 09:36:39.668124 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application [None req-3ca37ee2-7455-4bea-91ec-2265d79a0375 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 09:36:39.668124 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:36:39.668124 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:36:39.668124 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:36:39.668124 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:36:39.668124 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:36:39.668124 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:36:39.668124 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:36:39.668124 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:36:39.668124 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:36:39.668124 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:36:39.668124 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:36:39.668124 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 09:36:39.668124 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 09:36:39.668124 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 09:36:39.668124 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 09:36:39.669242 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 09:36:39.669242 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 09:36:39.669242 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 09:36:39.669242 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 09:36:39.669242 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:36:39.669242 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:36:39.669242 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:36:39.669242 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:36:39.669242 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:36:39.669242 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:36:39.669242 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:36:39.669242 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application with Lock( Aug 31 09:36:39.669242 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:36:39.669242 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:36:39.669242 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:36:39.669242 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:36:39.670348 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:36:39.670348 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:36:39.670348 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:36:39.670348 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:36:39.670348 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 09:36:39.670348 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 09:36:39.670348 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 09:36:39.670348 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 09:36:39.670348 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 09:36:39.670348 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 09:36:39.670348 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 09:36:39.670348 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application Aug 31 09:36:39.670348 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 600/1195] 10.222.0.13 () {56 vars in 1096 bytes} [Wed Aug 31 09:36:39 2022] GET /identity/v3/roles/admin => generated 83 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:36:39.675417 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f45feb-8e39-4258-b22e-d4cf3c6ee432 None None] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:36:39.771854 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f45feb-8e39-4258-b22e-d4cf3c6ee432 None None] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:36:39.788453 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:36:39.805898 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d57092f-57c7-4b86-a2d2-c9a4351aff9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:36:39.806493 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d57092f-57c7-4b86-a2d2-c9a4351aff9d None admin] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:36:39.806765 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d57092f-57c7-4b86-a2d2-c9a4351aff9d None admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:36:39.807586 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d57092f-57c7-4b86-a2d2-c9a4351aff9d None admin] PATH_INFO: `/v3/roles` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:36:39.808235 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d57092f-57c7-4b86-a2d2-c9a4351aff9d None admin] RBAC: Adding query filter params (name=admin) {{(pid=88114) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:36:39.808533 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d57092f-57c7-4b86-a2d2-c9a4351aff9d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:36:39.809912 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d57092f-57c7-4b86-a2d2-c9a4351aff9d None admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:36:39.817812 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 596/1196] 10.222.0.13 () {56 vars in 1099 bytes} [Wed Aug 31 09:36:39 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:36:39.824270 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca37ee2-7455-4bea-91ec-2265d79a0375 None admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:36:39.912884 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=100 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:36:39.925688 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca37ee2-7455-4bea-91ec-2265d79a0375 None admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:36:39.959918 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2341cc3c-4a5f-4c0d-bb1a-0b0612fe0ba4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:36:39.960537 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2341cc3c-4a5f-4c0d-bb1a-0b0612fe0ba4 None admin] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:36:39.960810 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2341cc3c-4a5f-4c0d-bb1a-0b0612fe0ba4 None admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:36:39.961072 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2341cc3c-4a5f-4c0d-bb1a-0b0612fe0ba4 None admin] PATH_INFO: `/v3/users/1400b17335734644b81746547847b228` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:36:39.962195 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2341cc3c-4a5f-4c0d-bb1a-0b0612fe0ba4 None admin] RBAC: Authorizing `identity:get_user(user_id=1400b17335734644b81746547847b228)` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:36:39.963663 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2341cc3c-4a5f-4c0d-bb1a-0b0612fe0ba4 None admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:36:39.965236 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 601/1197] 10.222.0.13 () {56 vars in 1177 bytes} [Wed Aug 31 09:36:39 2022] GET /identity/v3/users/1400b17335734644b81746547847b228 => generated 311 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:36:39.971403 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d57092f-57c7-4b86-a2d2-c9a4351aff9d None admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:36:40.073834 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d57092f-57c7-4b86-a2d2-c9a4351aff9d None admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:36:40.107279 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e8081e8-7e61-450e-b813-ce034d6ec1de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:36:40.107881 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8081e8-7e61-450e-b813-ce034d6ec1de None admin] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:36:40.108153 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8081e8-7e61-450e-b813-ce034d6ec1de None admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:36:40.108419 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8081e8-7e61-450e-b813-ce034d6ec1de None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:36:40.156470 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=100 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:36:40.175899 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e8081e8-7e61-450e-b813-ce034d6ec1de None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:36:40.177354 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e8081e8-7e61-450e-b813-ce034d6ec1de None admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:36:40.275424 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 597/1198] 10.222.0.13 () {58 vars in 1282 bytes} [Wed Aug 31 09:36:39 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 31 09:36:40.282142 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2341cc3c-4a5f-4c0d-bb1a-0b0612fe0ba4 None admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:36:40.380135 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2341cc3c-4a5f-4c0d-bb1a-0b0612fe0ba4 None admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:36:40.414319 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f7c25d-b94c-4d16-bd98-87adfc6628df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:36:40.414946 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f7c25d-b94c-4d16-bd98-87adfc6628df None admin] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:36:40.415224 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f7c25d-b94c-4d16-bd98-87adfc6628df None admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:36:40.415498 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f7c25d-b94c-4d16-bd98-87adfc6628df None admin] PATH_INFO: `/v3/projects/bb9b6d7419c446ffa65390a4a9246b83` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:36:40.416642 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72f7c25d-b94c-4d16-bd98-87adfc6628df None admin] RBAC: Authorizing `identity:get_project(project_id=bb9b6d7419c446ffa65390a4a9246b83)` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:36:40.418291 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72f7c25d-b94c-4d16-bd98-87adfc6628df None admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:36:40.419979 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 602/1199] 10.222.0.13 () {56 vars in 1186 bytes} [Wed Aug 31 09:36:40 2022] GET /identity/v3/projects/bb9b6d7419c446ffa65390a4a9246b83 => generated 361 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:36:40.426237 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e8081e8-7e61-450e-b813-ce034d6ec1de None admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:36:40.507992 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=100 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:36:40.527873 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e8081e8-7e61-450e-b813-ce034d6ec1de None admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:36:40.561391 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b0f0f12-93bf-48a7-b716-de4c7f81596e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:36:40.562007 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0f0f12-93bf-48a7-b716-de4c7f81596e None admin] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:36:40.562282 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0f0f12-93bf-48a7-b716-de4c7f81596e None admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:36:40.562550 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0f0f12-93bf-48a7-b716-de4c7f81596e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:36:40.563266 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b0f0f12-93bf-48a7-b716-de4c7f81596e None admin] RBAC: Adding query filter params (role.id=5d4ee63a3add47fc9a6b8b16d39e0a1f, scope.project.id=bb9b6d7419c446ffa65390a4a9246b83, user.id=1400b17335734644b81746547847b228) {{(pid=88114) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:36:40.563566 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b0f0f12-93bf-48a7-b716-de4c7f81596e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:36:40.564988 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b0f0f12-93bf-48a7-b716-de4c7f81596e None admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:36:40.570789 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 598/1200] 10.222.0.13 () {56 vars in 1374 bytes} [Wed Aug 31 09:36:40 2022] GET /identity/v3/role_assignments?user.id=1400b17335734644b81746547847b228&scope.project.id=bb9b6d7419c446ffa65390a4a9246b83&role.id=5d4ee63a3add47fc9a6b8b16d39e0a1f => generated 608 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:36:50.410160 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=88 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:36:50.568105 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=28 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:37:37.451882 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54c4c5e-c81f-42e6-a695-8e365025eb96 None None] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:37:37.451882 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54c4c5e-c81f-42e6-a695-8e365025eb96 None None] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:37:37.451882 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54c4c5e-c81f-42e6-a695-8e365025eb96 None None] PATH_INFO: `/` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:37:37.452844 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 603/1201] 10.222.0.13 () {54 vars in 927 bytes} [Wed Aug 31 09:37:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:37:37.459057 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d77b356-b7e4-40ba-8470-a95b7369fe1b None None] REQUEST_METHOD: `POST` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:37:37.459609 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d77b356-b7e4-40ba-8470-a95b7369fe1b None None] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:37:37.459609 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d77b356-b7e4-40ba-8470-a95b7369fe1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:37:37.484076 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.auth.core [None req-1d77b356-b7e4-40ba-8470-a95b7369fe1b None None] MFA Rules not processed for user `fd836cf590a145df9acf19cf3ae2ac56`. Rule list: `[]` (Enabled: `True`). {{(pid=88114) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:37:37.485614 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.fernet_utils [None req-1d77b356-b7e4-40ba-8470-a95b7369fe1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88114) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:37:37.593868 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 599/1202] 10.222.0.13 () {58 vars in 1024 bytes} [Wed Aug 31 09:37:37 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 09:37:37.900498 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e54c4c5e-c81f-42e6-a695-8e365025eb96 None None] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:37:38.038146 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e54c4c5e-c81f-42e6-a695-8e365025eb96 None None] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:37:38.092513 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b26125c-385c-4037-a316-91acbeea5d31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:37:38.093573 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b26125c-385c-4037-a316-91acbeea5d31 None admin] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:37:38.093821 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b26125c-385c-4037-a316-91acbeea5d31 None admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:37:38.094173 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b26125c-385c-4037-a316-91acbeea5d31 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:37:38.111603 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b26125c-385c-4037-a316-91acbeea5d31 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:37:38.115055 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b26125c-385c-4037-a316-91acbeea5d31 None admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:37:38.130171 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application [None req-8b26125c-385c-4037-a316-91acbeea5d31 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 09:37:38.130171 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:37:38.130171 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:37:38.130171 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:37:38.130171 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:37:38.130171 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:37:38.130171 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:37:38.130171 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:37:38.130171 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:37:38.130171 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:37:38.130171 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:37:38.130171 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:37:38.130171 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 09:37:38.130171 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 09:37:38.130171 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 09:37:38.130171 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 09:37:38.133706 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:37:38.133706 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:37:38.133706 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:37:38.133706 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:37:38.133706 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:37:38.133706 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:37:38.133706 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:37:38.133706 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application with Lock( Aug 31 09:37:38.133706 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:37:38.133706 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:37:38.133706 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:37:38.133706 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:37:38.133706 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:37:38.133706 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:37:38.133706 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:37:38.133706 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:37:38.133706 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 09:37:38.134877 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 09:37:38.134877 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 09:37:38.134877 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application Aug 31 09:37:38.134877 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 604/1203] 10.222.0.13 () {56 vars in 1108 bytes} [Wed Aug 31 09:37:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:37:38.138009 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d77b356-b7e4-40ba-8470-a95b7369fe1b None None] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:37:38.275643 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d77b356-b7e4-40ba-8470-a95b7369fe1b None None] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:37:38.322203 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=100 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:37:38.329112 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c50e48a-8960-44a1-9319-6148a75af116 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:37:38.329774 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c50e48a-8960-44a1-9319-6148a75af116 None admin] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:37:38.330106 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c50e48a-8960-44a1-9319-6148a75af116 None admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:37:38.330356 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c50e48a-8960-44a1-9319-6148a75af116 None admin] PATH_INFO: `/v3/domains` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:37:38.331002 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c50e48a-8960-44a1-9319-6148a75af116 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88114) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:37:38.331308 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c50e48a-8960-44a1-9319-6148a75af116 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:37:38.332705 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c50e48a-8960-44a1-9319-6148a75af116 None admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:37:38.351090 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 600/1204] 10.222.0.13 () {56 vars in 1109 bytes} [Wed Aug 31 09:37:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:37:38.357363 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b26125c-385c-4037-a316-91acbeea5d31 None admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:37:38.433497 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=100 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:37:38.506222 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b26125c-385c-4037-a316-91acbeea5d31 None admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:37:38.553535 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4878bd48-51af-44c9-97aa-a54d7c616909 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:37:38.553535 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4878bd48-51af-44c9-97aa-a54d7c616909 None admin] REQUEST_METHOD: `POST` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:37:38.553535 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4878bd48-51af-44c9-97aa-a54d7c616909 None admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:37:38.553535 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4878bd48-51af-44c9-97aa-a54d7c616909 None admin] PATH_INFO: `/v3/users` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:37:38.554287 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4878bd48-51af-44c9-97aa-a54d7c616909 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:37:38.554447 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4878bd48-51af-44c9-97aa-a54d7c616909 None admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:37:38.580963 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 605/1205] 10.222.0.13 () {60 vars in 1132 bytes} [Wed Aug 31 09:37:38 2022] POST /identity/v3/users => generated 251 bytes in 224 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 09:37:40.684895 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db71ced-123f-4733-a3ce-f1760a5fa178 None None] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:37:40.685516 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db71ced-123f-4733-a3ce-f1760a5fa178 None None] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:37:40.685516 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db71ced-123f-4733-a3ce-f1760a5fa178 None None] PATH_INFO: `/` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:37:40.687111 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 601/1206] 10.222.0.13 () {54 vars in 927 bytes} [Wed Aug 31 09:37:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:37:40.692626 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96774be-b659-4bb1-a1bd-3b10ec4f5a4f None None] REQUEST_METHOD: `POST` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:37:40.693176 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96774be-b659-4bb1-a1bd-3b10ec4f5a4f None None] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:37:40.693176 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96774be-b659-4bb1-a1bd-3b10ec4f5a4f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:37:40.713171 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.auth.core [None req-a96774be-b659-4bb1-a1bd-3b10ec4f5a4f None None] MFA Rules not processed for user `fd836cf590a145df9acf19cf3ae2ac56`. Rule list: `[]` (Enabled: `True`). {{(pid=88113) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:37:40.714403 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.fernet_utils [None req-a96774be-b659-4bb1-a1bd-3b10ec4f5a4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:37:40.795631 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 606/1207] 10.222.0.13 () {58 vars in 1024 bytes} [Wed Aug 31 09:37:40 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 09:37:41.078959 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db71ced-123f-4733-a3ce-f1760a5fa178 None None] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:37:41.202977 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db71ced-123f-4733-a3ce-f1760a5fa178 None None] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:37:41.242947 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1202297-7a36-4a47-b58d-1f894654633d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:37:41.242947 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1202297-7a36-4a47-b58d-1f894654633d None admin] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:37:41.243248 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1202297-7a36-4a47-b58d-1f894654633d None admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:37:41.243467 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1202297-7a36-4a47-b58d-1f894654633d None admin] PATH_INFO: `/v3/roles/service` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:37:41.255218 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1202297-7a36-4a47-b58d-1f894654633d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 09:37:41.261209 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application [None req-d1202297-7a36-4a47-b58d-1f894654633d None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 31 09:37:41.261209 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:37:41.261209 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:37:41.261209 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:37:41.261209 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:37:41.261209 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:37:41.261209 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:37:41.261209 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:37:41.261209 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:37:41.261209 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:37:41.261209 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:37:41.261209 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:37:41.261209 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 09:37:41.261209 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 09:37:41.261209 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 09:37:41.261209 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 09:37:41.262278 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 09:37:41.262278 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 09:37:41.262278 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 09:37:41.262278 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 09:37:41.262278 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:37:41.262278 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:37:41.262278 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:37:41.262278 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:37:41.262278 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:37:41.262278 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:37:41.262278 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:37:41.262278 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application with Lock( Aug 31 09:37:41.262278 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:37:41.262278 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:37:41.262278 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:37:41.262278 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:37:41.263192 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:37:41.263192 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:37:41.263192 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:37:41.263192 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:37:41.263192 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 09:37:41.263192 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 09:37:41.263192 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 09:37:41.263192 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 09:37:41.263192 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 09:37:41.263192 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 09:37:41.263192 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 31 09:37:41.263192 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application Aug 31 09:37:41.263192 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 602/1208] 10.222.0.13 () {56 vars in 1102 bytes} [Wed Aug 31 09:37:41 2022] GET /identity/v3/roles/service => generated 85 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:37:41.270387 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a96774be-b659-4bb1-a1bd-3b10ec4f5a4f None None] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:37:41.315517 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:37:41.386411 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a96774be-b659-4bb1-a1bd-3b10ec4f5a4f None None] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:37:41.426676 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-577b6352-35e1-449e-802b-91b4771f7772 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:37:41.427291 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577b6352-35e1-449e-802b-91b4771f7772 None admin] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:37:41.427573 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577b6352-35e1-449e-802b-91b4771f7772 None admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:37:41.430871 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577b6352-35e1-449e-802b-91b4771f7772 None admin] PATH_INFO: `/v3/roles` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:37:41.430871 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-577b6352-35e1-449e-802b-91b4771f7772 None admin] RBAC: Adding query filter params (name=service) {{(pid=88113) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:37:41.430871 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-577b6352-35e1-449e-802b-91b4771f7772 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:37:41.431471 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-577b6352-35e1-449e-802b-91b4771f7772 None admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:37:41.440828 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 607/1209] 10.222.0.13 () {56 vars in 1103 bytes} [Wed Aug 31 09:37:41 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:37:41.448332 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1202297-7a36-4a47-b58d-1f894654633d None admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:37:41.530429 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=100 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:37:41.613943 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1202297-7a36-4a47-b58d-1f894654633d None admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:37:41.665701 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2804d79-2eef-49fd-9da4-d24a3b4e8034 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:37:41.666372 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2804d79-2eef-49fd-9da4-d24a3b4e8034 None admin] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:37:41.666622 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2804d79-2eef-49fd-9da4-d24a3b4e8034 None admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:37:41.666932 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2804d79-2eef-49fd-9da4-d24a3b4e8034 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:37:41.681170 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2804d79-2eef-49fd-9da4-d24a3b4e8034 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:37:41.682647 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2804d79-2eef-49fd-9da4-d24a3b4e8034 None admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:37:41.699591 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application [None req-f2804d79-2eef-49fd-9da4-d24a3b4e8034 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 09:37:41.699591 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:37:41.699591 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:37:41.699591 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:37:41.699591 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:37:41.699591 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:37:41.699591 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:37:41.699591 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:37:41.699591 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:37:41.699591 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:37:41.699591 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:37:41.699591 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:37:41.699591 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 09:37:41.699591 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 09:37:41.699591 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 09:37:41.699591 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 09:37:41.706218 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:37:41.706218 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:37:41.706218 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:37:41.706218 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:37:41.706218 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:37:41.706218 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:37:41.706218 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:37:41.706218 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application with Lock( Aug 31 09:37:41.706218 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:37:41.706218 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:37:41.706218 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:37:41.706218 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:37:41.706218 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:37:41.706218 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:37:41.706218 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:37:41.706218 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:37:41.706218 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 09:37:41.707339 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 09:37:41.707339 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 09:37:41.707339 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application Aug 31 09:37:41.707339 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 603/1210] 10.222.0.13 () {56 vars in 1108 bytes} [Wed Aug 31 09:37:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 253 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:37:41.707973 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-577b6352-35e1-449e-802b-91b4771f7772 None admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:37:41.846442 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=100 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:37:41.852074 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-577b6352-35e1-449e-802b-91b4771f7772 None admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:37:41.911029 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75bd6b78-3b37-4192-b9f6-bb9d940ae0a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:37:41.911719 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75bd6b78-3b37-4192-b9f6-bb9d940ae0a4 None admin] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:37:41.912150 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75bd6b78-3b37-4192-b9f6-bb9d940ae0a4 None admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:37:41.912430 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75bd6b78-3b37-4192-b9f6-bb9d940ae0a4 None admin] PATH_INFO: `/v3/domains` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:37:41.914873 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75bd6b78-3b37-4192-b9f6-bb9d940ae0a4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88113) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:37:41.915279 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75bd6b78-3b37-4192-b9f6-bb9d940ae0a4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:37:41.916761 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75bd6b78-3b37-4192-b9f6-bb9d940ae0a4 None admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:37:41.930584 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 608/1211] 10.222.0.13 () {56 vars in 1109 bytes} [Wed Aug 31 09:37:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:37:41.936816 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2804d79-2eef-49fd-9da4-d24a3b4e8034 None admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:37:42.050121 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2804d79-2eef-49fd-9da4-d24a3b4e8034 None admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:37:42.051907 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=100 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:37:42.102087 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c485aa-b6bb-44f7-9e98-b49d7d670b47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:37:42.102087 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c485aa-b6bb-44f7-9e98-b49d7d670b47 None admin] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:37:42.102087 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c485aa-b6bb-44f7-9e98-b49d7d670b47 None admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:37:42.102471 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c485aa-b6bb-44f7-9e98-b49d7d670b47 None admin] PATH_INFO: `/v3/users/placement` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:37:42.111544 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1c485aa-b6bb-44f7-9e98-b49d7d670b47 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:37:42.113296 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1c485aa-b6bb-44f7-9e98-b49d7d670b47 None admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:37:42.130181 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application [None req-c1c485aa-b6bb-44f7-9e98-b49d7d670b47 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 31 09:37:42.130181 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:37:42.130181 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:37:42.130181 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:37:42.130181 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:37:42.130181 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:37:42.130181 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:37:42.130181 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:37:42.130181 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:37:42.130181 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:37:42.130181 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:37:42.130181 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:37:42.130181 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 31 09:37:42.130181 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 31 09:37:42.130181 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 31 09:37:42.130181 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 31 09:37:42.131416 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:37:42.131416 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:37:42.131416 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 31 09:37:42.131416 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 09:37:42.131416 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 31 09:37:42.131416 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 09:37:42.131416 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:37:42.131416 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:37:42.131416 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:37:42.131416 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:37:42.131416 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:37:42.131416 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application with Lock( Aug 31 09:37:42.131416 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:37:42.131416 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:37:42.131416 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:37:42.131416 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:37:42.131416 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:37:42.131416 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:37:42.132645 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:37:42.132645 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:37:42.132645 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 31 09:37:42.132645 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 31 09:37:42.132645 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 31 09:37:42.132645 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 31 09:37:42.132645 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 31 09:37:42.132645 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 31 09:37:42.132645 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 31 09:37:42.132645 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application Aug 31 09:37:42.138202 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 604/1212] 10.222.0.13 () {56 vars in 1108 bytes} [Wed Aug 31 09:37:41 2022] GET /identity/v3/users/placement => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:37:42.140186 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75bd6b78-3b37-4192-b9f6-bb9d940ae0a4 None admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:37:42.265063 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75bd6b78-3b37-4192-b9f6-bb9d940ae0a4 None admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:37:42.307027 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cfaead0-04c1-460e-901c-5e8e55f28fee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:37:42.307638 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cfaead0-04c1-460e-901c-5e8e55f28fee None admin] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:37:42.307922 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cfaead0-04c1-460e-901c-5e8e55f28fee None admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:37:42.308196 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cfaead0-04c1-460e-901c-5e8e55f28fee None admin] PATH_INFO: `/v3/users` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:37:42.308938 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cfaead0-04c1-460e-901c-5e8e55f28fee None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=88113) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:37:42.309247 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cfaead0-04c1-460e-901c-5e8e55f28fee None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:37:42.311100 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cfaead0-04c1-460e-901c-5e8e55f28fee None admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:37:42.334831 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 609/1213] 10.222.0.13 () {56 vars in 1143 bytes} [Wed Aug 31 09:37:42 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:37:42.342259 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c485aa-b6bb-44f7-9e98-b49d7d670b47 None admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:37:42.470022 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c485aa-b6bb-44f7-9e98-b49d7d670b47 None admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:37:42.512282 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a19856a-ee6a-4d10-92da-0da55b103280 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:37:42.513004 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a19856a-ee6a-4d10-92da-0da55b103280 None admin] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:37:42.513294 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a19856a-ee6a-4d10-92da-0da55b103280 None admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:37:42.513660 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a19856a-ee6a-4d10-92da-0da55b103280 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:37:42.563707 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=100 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:37:42.622406 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a19856a-ee6a-4d10-92da-0da55b103280 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:37:42.624727 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a19856a-ee6a-4d10-92da-0da55b103280 None admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:37:42.753675 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 605/1214] 10.222.0.13 () {58 vars in 1282 bytes} [Wed Aug 31 09:37:42 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 412 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 31 09:37:42.755434 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cfaead0-04c1-460e-901c-5e8e55f28fee None admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:37:42.778508 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=100 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:37:42.884416 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cfaead0-04c1-460e-901c-5e8e55f28fee None admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:37:42.928868 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-769b0afd-28d3-4b91-a63f-9ddb839c129d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:37:42.931227 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769b0afd-28d3-4b91-a63f-9ddb839c129d None admin] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:37:42.931227 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769b0afd-28d3-4b91-a63f-9ddb839c129d None admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:37:42.931411 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769b0afd-28d3-4b91-a63f-9ddb839c129d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:37:42.942528 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-769b0afd-28d3-4b91-a63f-9ddb839c129d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:37:42.944409 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-769b0afd-28d3-4b91-a63f-9ddb839c129d None admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:37:42.961335 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application [None req-769b0afd-28d3-4b91-a63f-9ddb839c129d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 09:37:42.961335 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:37:42.961335 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:37:42.961335 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:37:42.961335 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:37:42.961335 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:37:42.961335 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:37:42.961335 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:37:42.961335 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:37:42.961335 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:37:42.961335 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:37:42.961335 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:37:42.961335 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 09:37:42.961335 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 09:37:42.961335 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 09:37:42.961335 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 09:37:42.962699 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:37:42.962699 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:37:42.962699 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:37:42.962699 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:37:42.962699 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:37:42.962699 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:37:42.962699 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:37:42.962699 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application with Lock( Aug 31 09:37:42.962699 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:37:42.962699 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:37:42.962699 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:37:42.962699 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:37:42.962699 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:37:42.962699 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:37:42.962699 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:37:42.962699 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:37:42.962699 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 09:37:42.963606 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 09:37:42.963606 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 09:37:42.963606 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application Aug 31 09:37:42.967499 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 610/1215] 10.222.0.13 () {56 vars in 1108 bytes} [Wed Aug 31 09:37:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:37:42.968360 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a19856a-ee6a-4d10-92da-0da55b103280 None admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:37:43.037611 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=100 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:37:43.091933 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a19856a-ee6a-4d10-92da-0da55b103280 None admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:37:43.133848 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3655baf6-62e6-4e04-976d-da3affb12810 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:37:43.134941 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3655baf6-62e6-4e04-976d-da3affb12810 None admin] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:37:43.135429 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3655baf6-62e6-4e04-976d-da3affb12810 None admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:37:43.135953 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3655baf6-62e6-4e04-976d-da3affb12810 None admin] PATH_INFO: `/v3/domains` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:37:43.136730 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3655baf6-62e6-4e04-976d-da3affb12810 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88114) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:37:43.137166 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3655baf6-62e6-4e04-976d-da3affb12810 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:37:43.138743 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3655baf6-62e6-4e04-976d-da3affb12810 None admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:37:43.157458 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 606/1216] 10.222.0.13 () {56 vars in 1109 bytes} [Wed Aug 31 09:37:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:37:43.159227 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-769b0afd-28d3-4b91-a63f-9ddb839c129d None admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:37:43.243150 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=100 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:37:43.283275 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-769b0afd-28d3-4b91-a63f-9ddb839c129d None admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:37:43.321675 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4654f52a-371e-44ea-bba3-51a107c22703 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:37:43.322329 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4654f52a-371e-44ea-bba3-51a107c22703 None admin] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:37:43.322682 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4654f52a-371e-44ea-bba3-51a107c22703 None admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:37:43.323032 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4654f52a-371e-44ea-bba3-51a107c22703 None admin] PATH_INFO: `/v3/projects/service` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:37:43.329901 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4654f52a-371e-44ea-bba3-51a107c22703 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:37:43.331416 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4654f52a-371e-44ea-bba3-51a107c22703 None admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:37:43.343372 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application [None req-4654f52a-371e-44ea-bba3-51a107c22703 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 09:37:43.343372 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:37:43.343372 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:37:43.343372 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:37:43.343372 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:37:43.343372 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:37:43.343372 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:37:43.343372 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:37:43.343372 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:37:43.343372 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:37:43.343372 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:37:43.343372 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:37:43.343372 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 31 09:37:43.343372 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 31 09:37:43.343372 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 31 09:37:43.344503 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 31 09:37:43.344503 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:37:43.344503 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:37:43.344503 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:37:43.344503 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:37:43.344503 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:37:43.344503 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:37:43.344503 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:37:43.344503 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application with Lock( Aug 31 09:37:43.344503 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:37:43.344503 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:37:43.344503 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:37:43.344503 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:37:43.344503 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:37:43.344503 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:37:43.344503 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:37:43.344503 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:37:43.345751 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 31 09:37:43.345751 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 31 09:37:43.345751 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 31 09:37:43.345751 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 31 09:37:43.345751 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 31 09:37:43.345751 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 31 09:37:43.345751 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 09:37:43.345751 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application Aug 31 09:37:43.351270 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 611/1217] 10.222.0.13 () {56 vars in 1111 bytes} [Wed Aug 31 09:37:43 2022] GET /identity/v3/projects/service => generated 88 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:37:43.352821 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3655baf6-62e6-4e04-976d-da3affb12810 None admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:37:43.472578 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3655baf6-62e6-4e04-976d-da3affb12810 None admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:37:43.506083 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=100 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:37:43.520956 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f4661ea-13bf-4e7e-9a26-d60bc0231a01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:37:43.521570 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4661ea-13bf-4e7e-9a26-d60bc0231a01 None admin] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:37:43.521849 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4661ea-13bf-4e7e-9a26-d60bc0231a01 None admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:37:43.522119 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4661ea-13bf-4e7e-9a26-d60bc0231a01 None admin] PATH_INFO: `/v3/projects` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:37:43.522814 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f4661ea-13bf-4e7e-9a26-d60bc0231a01 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88114) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:37:43.523707 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f4661ea-13bf-4e7e-9a26-d60bc0231a01 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:37:43.525179 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f4661ea-13bf-4e7e-9a26-d60bc0231a01 None admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:37:43.539600 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 607/1218] 10.222.0.13 () {56 vars in 1148 bytes} [Wed Aug 31 09:37:43 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:37:43.545286 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4654f52a-371e-44ea-bba3-51a107c22703 None admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:37:43.667211 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4654f52a-371e-44ea-bba3-51a107c22703 None admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:37:43.701330 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=100 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:37:43.707832 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f01d7490-cc57-43c7-bc51-9ce89cf21fcd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:37:43.708573 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f01d7490-cc57-43c7-bc51-9ce89cf21fcd None admin] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:37:43.709305 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f01d7490-cc57-43c7-bc51-9ce89cf21fcd None admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:37:43.709675 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f01d7490-cc57-43c7-bc51-9ce89cf21fcd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:37:43.710610 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f01d7490-cc57-43c7-bc51-9ce89cf21fcd None admin] RBAC: Adding query filter params (role.id=e24ba4fd91584883af350cc84bc96c1d, scope.project.id=49548eb44d62415582af820ab66decee, user.id=1ff3aff5ef444910be6dbb989d0e3194) {{(pid=88113) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:37:43.710963 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f01d7490-cc57-43c7-bc51-9ce89cf21fcd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:37:43.712448 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f01d7490-cc57-43c7-bc51-9ce89cf21fcd None admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:37:43.721420 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 612/1219] 10.222.0.13 () {56 vars in 1374 bytes} [Wed Aug 31 09:37:43 2022] GET /identity/v3/role_assignments?user.id=1ff3aff5ef444910be6dbb989d0e3194&scope.project.id=49548eb44d62415582af820ab66decee&role.id=e24ba4fd91584883af350cc84bc96c1d => generated 259 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:37:45.657473 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb503c1-64a1-41c9-822e-42c383ebb382 None None] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:37:45.658012 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb503c1-64a1-41c9-822e-42c383ebb382 None None] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:37:45.658012 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb503c1-64a1-41c9-822e-42c383ebb382 None None] PATH_INFO: `/` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:37:45.659329 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 608/1220] 10.222.0.13 () {54 vars in 927 bytes} [Wed Aug 31 09:37:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:37:45.665427 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367a48a8-0821-498a-bbe5-51f8461a15f1 None None] REQUEST_METHOD: `POST` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:37:45.665975 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367a48a8-0821-498a-bbe5-51f8461a15f1 None None] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:37:45.665975 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367a48a8-0821-498a-bbe5-51f8461a15f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:37:45.686346 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.auth.core [None req-367a48a8-0821-498a-bbe5-51f8461a15f1 None None] MFA Rules not processed for user `fd836cf590a145df9acf19cf3ae2ac56`. Rule list: `[]` (Enabled: `True`). {{(pid=88113) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:37:45.687636 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.fernet_utils [None req-367a48a8-0821-498a-bbe5-51f8461a15f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:37:45.765820 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 613/1221] 10.222.0.13 () {58 vars in 1024 bytes} [Wed Aug 31 09:37:45 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 31 09:37:46.035481 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb503c1-64a1-41c9-822e-42c383ebb382 None None] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:37:46.149340 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb503c1-64a1-41c9-822e-42c383ebb382 None None] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:37:46.188478 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfec2ea6-0151-4440-9035-5ad006255b78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:37:46.189153 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfec2ea6-0151-4440-9035-5ad006255b78 None admin] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:37:46.189460 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfec2ea6-0151-4440-9035-5ad006255b78 None admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:37:46.189733 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfec2ea6-0151-4440-9035-5ad006255b78 None admin] PATH_INFO: `/v3/roles/service` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:37:46.202153 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfec2ea6-0151-4440-9035-5ad006255b78 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 09:37:46.207081 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application [None req-bfec2ea6-0151-4440-9035-5ad006255b78 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 31 09:37:46.207081 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:37:46.207081 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:37:46.207081 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:37:46.207081 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:37:46.207081 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:37:46.207081 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:37:46.207081 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:37:46.207081 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:37:46.207081 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:37:46.207081 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:37:46.207081 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:37:46.207081 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 09:37:46.207081 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 09:37:46.207081 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 09:37:46.207081 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 09:37:46.208285 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 09:37:46.208285 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 09:37:46.208285 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 09:37:46.208285 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 09:37:46.208285 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:37:46.208285 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:37:46.208285 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:37:46.208285 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:37:46.208285 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:37:46.208285 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:37:46.208285 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:37:46.208285 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application with Lock( Aug 31 09:37:46.208285 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:37:46.208285 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:37:46.208285 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:37:46.208285 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:37:46.209480 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:37:46.209480 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:37:46.209480 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:37:46.209480 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:37:46.209480 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 09:37:46.209480 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 09:37:46.209480 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 09:37:46.209480 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 09:37:46.209480 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 09:37:46.209480 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 09:37:46.209480 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 31 09:37:46.209480 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application Aug 31 09:37:46.209480 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 609/1222] 10.222.0.13 () {56 vars in 1102 bytes} [Wed Aug 31 09:37:46 2022] GET /identity/v3/roles/service => generated 85 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:37:46.215293 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-367a48a8-0821-498a-bbe5-51f8461a15f1 None None] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:37:46.318915 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-367a48a8-0821-498a-bbe5-51f8461a15f1 None None] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:37:46.344198 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=100 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:37:46.356854 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d9fae5-9737-4097-939f-ff038b067b69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:37:46.357625 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d9fae5-9737-4097-939f-ff038b067b69 None admin] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:37:46.357948 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d9fae5-9737-4097-939f-ff038b067b69 None admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:37:46.358227 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d9fae5-9737-4097-939f-ff038b067b69 None admin] PATH_INFO: `/v3/roles` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:37:46.359041 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60d9fae5-9737-4097-939f-ff038b067b69 None admin] RBAC: Adding query filter params (name=service) {{(pid=88113) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:37:46.359389 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60d9fae5-9737-4097-939f-ff038b067b69 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:37:46.360971 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60d9fae5-9737-4097-939f-ff038b067b69 None admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:37:46.370437 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 614/1223] 10.222.0.13 () {56 vars in 1103 bytes} [Wed Aug 31 09:37:46 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:37:46.376242 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfec2ea6-0151-4440-9035-5ad006255b78 None admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:37:46.424412 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=100 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:37:46.488409 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfec2ea6-0151-4440-9035-5ad006255b78 None admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:37:46.526200 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85579cbe-8eed-451f-bdb2-4cbdeed3a09b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:37:46.526850 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85579cbe-8eed-451f-bdb2-4cbdeed3a09b None admin] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:37:46.527194 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85579cbe-8eed-451f-bdb2-4cbdeed3a09b None admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:37:46.527403 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85579cbe-8eed-451f-bdb2-4cbdeed3a09b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:37:46.538067 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85579cbe-8eed-451f-bdb2-4cbdeed3a09b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:37:46.539613 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85579cbe-8eed-451f-bdb2-4cbdeed3a09b None admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:37:46.555538 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application [None req-85579cbe-8eed-451f-bdb2-4cbdeed3a09b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 09:37:46.555538 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:37:46.555538 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:37:46.555538 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:37:46.555538 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:37:46.555538 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:37:46.555538 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:37:46.555538 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:37:46.555538 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:37:46.555538 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:37:46.555538 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:37:46.555538 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:37:46.555538 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 09:37:46.555538 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 09:37:46.555538 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 09:37:46.555538 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 09:37:46.557451 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:37:46.557451 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:37:46.557451 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:37:46.557451 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:37:46.557451 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:37:46.557451 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:37:46.557451 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:37:46.557451 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application with Lock( Aug 31 09:37:46.557451 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:37:46.557451 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:37:46.557451 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:37:46.557451 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:37:46.557451 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:37:46.557451 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:37:46.557451 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:37:46.557451 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:37:46.557451 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 09:37:46.558592 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 09:37:46.558592 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 09:37:46.558592 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application Aug 31 09:37:46.558592 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 610/1224] 10.222.0.13 () {56 vars in 1108 bytes} [Wed Aug 31 09:37:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:37:46.562861 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d9fae5-9737-4097-939f-ff038b067b69 None admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:37:46.685308 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d9fae5-9737-4097-939f-ff038b067b69 None admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:37:46.722363 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac26b718-ba10-46c9-973e-1d394591ffde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:37:46.723090 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac26b718-ba10-46c9-973e-1d394591ffde None admin] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:37:46.723372 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac26b718-ba10-46c9-973e-1d394591ffde None admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:37:46.723644 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac26b718-ba10-46c9-973e-1d394591ffde None admin] PATH_INFO: `/v3/domains` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:37:46.724339 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac26b718-ba10-46c9-973e-1d394591ffde None admin] RBAC: Adding query filter params (name=Default) {{(pid=88113) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:37:46.724644 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac26b718-ba10-46c9-973e-1d394591ffde None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:37:46.726675 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac26b718-ba10-46c9-973e-1d394591ffde None admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:37:46.740156 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 615/1225] 10.222.0.13 () {56 vars in 1109 bytes} [Wed Aug 31 09:37:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:37:46.745893 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85579cbe-8eed-451f-bdb2-4cbdeed3a09b None admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:37:46.852577 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=100 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:37:46.860635 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85579cbe-8eed-451f-bdb2-4cbdeed3a09b None admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:37:46.902178 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c97bbbf6-6cbd-46af-918d-a16f74a7da11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:37:46.902750 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97bbbf6-6cbd-46af-918d-a16f74a7da11 None admin] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:37:46.903095 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97bbbf6-6cbd-46af-918d-a16f74a7da11 None admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:37:46.903286 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97bbbf6-6cbd-46af-918d-a16f74a7da11 None admin] PATH_INFO: `/v3/users/placement` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:37:46.912007 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c97bbbf6-6cbd-46af-918d-a16f74a7da11 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:37:46.913731 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c97bbbf6-6cbd-46af-918d-a16f74a7da11 None admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:37:46.927598 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application [None req-c97bbbf6-6cbd-46af-918d-a16f74a7da11 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 31 09:37:46.927598 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:37:46.927598 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:37:46.927598 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:37:46.927598 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:37:46.927598 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:37:46.927598 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:37:46.927598 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:37:46.927598 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:37:46.927598 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:37:46.927598 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:37:46.927598 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:37:46.927598 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 31 09:37:46.927598 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 31 09:37:46.927598 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 31 09:37:46.927598 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 31 09:37:46.928933 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:37:46.928933 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:37:46.928933 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 31 09:37:46.928933 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 09:37:46.928933 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 31 09:37:46.928933 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 09:37:46.928933 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:37:46.928933 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:37:46.928933 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:37:46.928933 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:37:46.928933 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:37:46.928933 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application with Lock( Aug 31 09:37:46.928933 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:37:46.928933 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:37:46.928933 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:37:46.928933 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:37:46.928933 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:37:46.928933 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:37:46.930257 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:37:46.930257 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:37:46.930257 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 31 09:37:46.930257 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 31 09:37:46.930257 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 31 09:37:46.930257 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 31 09:37:46.930257 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 31 09:37:46.930257 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 31 09:37:46.930257 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 31 09:37:46.930257 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application Aug 31 09:37:46.930257 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 611/1226] 10.222.0.13 () {56 vars in 1108 bytes} [Wed Aug 31 09:37:46 2022] GET /identity/v3/users/placement => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:37:46.935642 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac26b718-ba10-46c9-973e-1d394591ffde None admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:37:46.997031 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=100 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:37:47.063311 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac26b718-ba10-46c9-973e-1d394591ffde None admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:37:47.105541 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f830173b-37cc-4ab9-b32b-dfc0af1db521 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:37:47.106144 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f830173b-37cc-4ab9-b32b-dfc0af1db521 None admin] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:37:47.106428 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f830173b-37cc-4ab9-b32b-dfc0af1db521 None admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:37:47.106674 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f830173b-37cc-4ab9-b32b-dfc0af1db521 None admin] PATH_INFO: `/v3/users` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:37:47.107408 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f830173b-37cc-4ab9-b32b-dfc0af1db521 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=88113) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:37:47.107706 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f830173b-37cc-4ab9-b32b-dfc0af1db521 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:37:47.109190 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f830173b-37cc-4ab9-b32b-dfc0af1db521 None admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:37:47.133031 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 616/1227] 10.222.0.13 () {56 vars in 1143 bytes} [Wed Aug 31 09:37:46 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:37:47.138869 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c97bbbf6-6cbd-46af-918d-a16f74a7da11 None admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:37:47.256039 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c97bbbf6-6cbd-46af-918d-a16f74a7da11 None admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:37:47.296738 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a99d601b-c144-4259-b0c7-f3c974f67334 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:37:47.297533 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99d601b-c144-4259-b0c7-f3c974f67334 None admin] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:37:47.297662 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99d601b-c144-4259-b0c7-f3c974f67334 None admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:37:47.297916 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99d601b-c144-4259-b0c7-f3c974f67334 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:37:47.311159 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a99d601b-c144-4259-b0c7-f3c974f67334 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:37:47.313544 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a99d601b-c144-4259-b0c7-f3c974f67334 None admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:37:47.331939 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application [None req-a99d601b-c144-4259-b0c7-f3c974f67334 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 09:37:47.331939 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:37:47.331939 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:37:47.331939 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:37:47.331939 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:37:47.331939 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:37:47.331939 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:37:47.331939 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:37:47.331939 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:37:47.331939 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:37:47.331939 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:37:47.331939 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:37:47.331939 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 09:37:47.331939 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 09:37:47.331939 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 09:37:47.331939 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 09:37:47.338418 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:37:47.338418 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:37:47.338418 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:37:47.338418 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:37:47.338418 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:37:47.338418 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:37:47.338418 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:37:47.338418 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application with Lock( Aug 31 09:37:47.338418 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:37:47.338418 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:37:47.338418 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:37:47.338418 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:37:47.338418 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:37:47.338418 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:37:47.338418 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:37:47.338418 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:37:47.338418 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 09:37:47.339702 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 09:37:47.339702 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 09:37:47.339702 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application Aug 31 09:37:47.339702 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 612/1228] 10.222.0.13 () {56 vars in 1108 bytes} [Wed Aug 31 09:37:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:37:47.339985 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f830173b-37cc-4ab9-b32b-dfc0af1db521 None admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:37:47.454400 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f830173b-37cc-4ab9-b32b-dfc0af1db521 None admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:37:47.456839 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=100 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:37:47.500521 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f46a388-0091-4b37-a4cc-f66f2d3c98f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:37:47.501293 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f46a388-0091-4b37-a4cc-f66f2d3c98f5 None admin] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:37:47.501775 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f46a388-0091-4b37-a4cc-f66f2d3c98f5 None admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:37:47.502206 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f46a388-0091-4b37-a4cc-f66f2d3c98f5 None admin] PATH_INFO: `/v3/domains` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:37:47.503157 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f46a388-0091-4b37-a4cc-f66f2d3c98f5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88113) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:37:47.503521 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f46a388-0091-4b37-a4cc-f66f2d3c98f5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:37:47.505620 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f46a388-0091-4b37-a4cc-f66f2d3c98f5 None admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:37:47.522623 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 617/1229] 10.222.0.13 () {56 vars in 1109 bytes} [Wed Aug 31 09:37:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:37:47.531977 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a99d601b-c144-4259-b0c7-f3c974f67334 None admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:37:47.543742 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=100 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:37:47.650825 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a99d601b-c144-4259-b0c7-f3c974f67334 None admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:37:47.688909 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f58b0616-0fda-4870-9e2a-6296b05e0a88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:37:47.689587 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58b0616-0fda-4870-9e2a-6296b05e0a88 None admin] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:37:47.689865 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58b0616-0fda-4870-9e2a-6296b05e0a88 None admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:37:47.690137 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58b0616-0fda-4870-9e2a-6296b05e0a88 None admin] PATH_INFO: `/v3/projects/service` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:37:47.697985 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f58b0616-0fda-4870-9e2a-6296b05e0a88 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:37:47.699427 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f58b0616-0fda-4870-9e2a-6296b05e0a88 None admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:37:47.713962 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application [None req-f58b0616-0fda-4870-9e2a-6296b05e0a88 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 09:37:47.713962 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:37:47.713962 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:37:47.713962 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:37:47.713962 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:37:47.713962 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:37:47.713962 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:37:47.713962 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:37:47.713962 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:37:47.713962 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:37:47.713962 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:37:47.713962 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:37:47.713962 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 31 09:37:47.713962 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 31 09:37:47.713962 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 31 09:37:47.715009 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 31 09:37:47.715009 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:37:47.715009 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:37:47.715009 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:37:47.715009 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:37:47.715009 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:37:47.715009 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:37:47.715009 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:37:47.715009 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application with Lock( Aug 31 09:37:47.715009 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:37:47.715009 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:37:47.715009 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:37:47.715009 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:37:47.715009 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:37:47.715009 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:37:47.715009 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:37:47.715009 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:37:47.716195 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 31 09:37:47.716195 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 31 09:37:47.716195 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 31 09:37:47.716195 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 31 09:37:47.716195 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 31 09:37:47.716195 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 31 09:37:47.716195 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 09:37:47.716195 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application Aug 31 09:37:47.716195 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 613/1230] 10.222.0.13 () {56 vars in 1111 bytes} [Wed Aug 31 09:37:47 2022] GET /identity/v3/projects/service => generated 88 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:37:47.721294 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f46a388-0091-4b37-a4cc-f66f2d3c98f5 None admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:37:47.835716 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f46a388-0091-4b37-a4cc-f66f2d3c98f5 None admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:37:47.874453 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c70f5650-3325-499a-bc92-db0cf88e26da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:37:47.875193 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70f5650-3325-499a-bc92-db0cf88e26da None admin] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:37:47.875505 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70f5650-3325-499a-bc92-db0cf88e26da None admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:37:47.875797 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70f5650-3325-499a-bc92-db0cf88e26da None admin] PATH_INFO: `/v3/projects` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:37:47.876571 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c70f5650-3325-499a-bc92-db0cf88e26da None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88113) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:37:47.876905 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c70f5650-3325-499a-bc92-db0cf88e26da None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:37:47.878580 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c70f5650-3325-499a-bc92-db0cf88e26da None admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:37:47.892389 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 618/1231] 10.222.0.13 () {56 vars in 1148 bytes} [Wed Aug 31 09:37:47 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:37:47.900669 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f58b0616-0fda-4870-9e2a-6296b05e0a88 None admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:37:47.982379 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=100 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:37:48.018763 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f58b0616-0fda-4870-9e2a-6296b05e0a88 None admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:37:48.053970 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7131ef02-70ec-4ff6-ac7d-e4b0addc7652 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:37:48.054997 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7131ef02-70ec-4ff6-ac7d-e4b0addc7652 None admin] REQUEST_METHOD: `PUT` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:37:48.055193 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7131ef02-70ec-4ff6-ac7d-e4b0addc7652 None admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:37:48.055468 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7131ef02-70ec-4ff6-ac7d-e4b0addc7652 None admin] PATH_INFO: `/v3/projects/49548eb44d62415582af820ab66decee/users/1ff3aff5ef444910be6dbb989d0e3194/roles/e24ba4fd91584883af350cc84bc96c1d` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:37:48.071470 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7131ef02-70ec-4ff6-ac7d-e4b0addc7652 None admin] RBAC: Authorizing `identity:create_grant(project_id=49548eb44d62415582af820ab66decee, user_id=1ff3aff5ef444910be6dbb989d0e3194, role_id=e24ba4fd91584883af350cc84bc96c1d)` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:37:48.072941 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7131ef02-70ec-4ff6-ac7d-e4b0addc7652 None admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:37:48.087253 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 614/1232] 10.222.0.13 () {58 vars in 1439 bytes} [Wed Aug 31 09:37:47 2022] PUT /identity/v3/projects/49548eb44d62415582af820ab66decee/users/1ff3aff5ef444910be6dbb989d0e3194/roles/e24ba4fd91584883af350cc84bc96c1d => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:37:50.171689 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad6d393-bf1b-4c32-94ad-feea5b62da30 None None] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:37:50.172221 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad6d393-bf1b-4c32-94ad-feea5b62da30 None None] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:37:50.172221 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad6d393-bf1b-4c32-94ad-feea5b62da30 None None] PATH_INFO: `/` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:37:50.173597 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 619/1233] 10.222.0.13 () {54 vars in 927 bytes} [Wed Aug 31 09:37:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:37:50.178876 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a677574e-b50c-4aac-a33b-1faa2dc71824 None None] REQUEST_METHOD: `POST` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:37:50.179918 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a677574e-b50c-4aac-a33b-1faa2dc71824 None None] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:37:50.180078 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a677574e-b50c-4aac-a33b-1faa2dc71824 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:37:50.200800 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.auth.core [None req-a677574e-b50c-4aac-a33b-1faa2dc71824 None None] MFA Rules not processed for user `fd836cf590a145df9acf19cf3ae2ac56`. Rule list: `[]` (Enabled: `True`). {{(pid=88114) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:37:50.202067 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.fernet_utils [None req-a677574e-b50c-4aac-a33b-1faa2dc71824 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88114) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:37:50.277599 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 615/1234] 10.222.0.13 () {58 vars in 1024 bytes} [Wed Aug 31 09:37:50 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 09:37:50.555352 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad6d393-bf1b-4c32-94ad-feea5b62da30 None None] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:37:50.583229 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=100 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:37:50.675407 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad6d393-bf1b-4c32-94ad-feea5b62da30 None None] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:37:50.718845 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b681070e-ed1e-4ec1-b582-b15e899629b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:37:50.719478 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b681070e-ed1e-4ec1-b582-b15e899629b7 None admin] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:37:50.719819 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b681070e-ed1e-4ec1-b582-b15e899629b7 None admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:37:50.720026 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b681070e-ed1e-4ec1-b582-b15e899629b7 None admin] PATH_INFO: `/v3/roles/service` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:37:50.733473 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b681070e-ed1e-4ec1-b582-b15e899629b7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 09:37:50.738330 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application [None req-b681070e-ed1e-4ec1-b582-b15e899629b7 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 31 09:37:50.738330 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:37:50.738330 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:37:50.738330 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:37:50.738330 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:37:50.738330 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:37:50.738330 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:37:50.738330 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:37:50.738330 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:37:50.738330 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:37:50.738330 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:37:50.738330 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:37:50.738330 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 09:37:50.738330 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 09:37:50.738330 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 09:37:50.738330 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 09:37:50.739469 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 09:37:50.739469 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 09:37:50.739469 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 09:37:50.739469 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 09:37:50.739469 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:37:50.739469 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:37:50.739469 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:37:50.739469 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:37:50.739469 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:37:50.739469 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:37:50.739469 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:37:50.739469 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application with Lock( Aug 31 09:37:50.739469 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:37:50.739469 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:37:50.739469 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:37:50.739469 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:37:50.740312 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:37:50.740312 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:37:50.740312 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:37:50.740312 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:37:50.740312 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 09:37:50.740312 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 09:37:50.740312 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 09:37:50.740312 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 09:37:50.740312 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 09:37:50.740312 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 09:37:50.740312 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 31 09:37:50.740312 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application Aug 31 09:37:50.740312 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 620/1235] 10.222.0.13 () {56 vars in 1102 bytes} [Wed Aug 31 09:37:50 2022] GET /identity/v3/roles/service => generated 85 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:37:50.747892 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a677574e-b50c-4aac-a33b-1faa2dc71824 None None] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:37:50.800777 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:37:50.864048 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a677574e-b50c-4aac-a33b-1faa2dc71824 None None] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:37:50.908465 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a32d9d1-d5f3-4c9e-a8b1-e46c8fb43b46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:37:50.909275 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a32d9d1-d5f3-4c9e-a8b1-e46c8fb43b46 None admin] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:37:50.909628 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a32d9d1-d5f3-4c9e-a8b1-e46c8fb43b46 None admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:37:50.909980 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a32d9d1-d5f3-4c9e-a8b1-e46c8fb43b46 None admin] PATH_INFO: `/v3/roles` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:37:50.910731 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a32d9d1-d5f3-4c9e-a8b1-e46c8fb43b46 None admin] RBAC: Adding query filter params (name=service) {{(pid=88114) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:37:50.911055 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a32d9d1-d5f3-4c9e-a8b1-e46c8fb43b46 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:37:50.912847 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a32d9d1-d5f3-4c9e-a8b1-e46c8fb43b46 None admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:37:50.923729 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 616/1236] 10.222.0.13 () {56 vars in 1103 bytes} [Wed Aug 31 09:37:50 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:37:50.930273 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b681070e-ed1e-4ec1-b582-b15e899629b7 None admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:37:51.021372 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=100 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:37:51.042623 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b681070e-ed1e-4ec1-b582-b15e899629b7 None admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:37:51.082161 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48541f7b-5e56-4a5e-bffb-c31c0447b89b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:37:51.082896 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48541f7b-5e56-4a5e-bffb-c31c0447b89b None admin] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:37:51.083185 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48541f7b-5e56-4a5e-bffb-c31c0447b89b None admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:37:51.083466 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48541f7b-5e56-4a5e-bffb-c31c0447b89b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:37:51.096787 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48541f7b-5e56-4a5e-bffb-c31c0447b89b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:37:51.098420 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48541f7b-5e56-4a5e-bffb-c31c0447b89b None admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:37:51.118671 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application [None req-48541f7b-5e56-4a5e-bffb-c31c0447b89b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 09:37:51.118671 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:37:51.118671 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:37:51.118671 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:37:51.118671 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:37:51.118671 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:37:51.118671 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:37:51.118671 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:37:51.118671 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:37:51.118671 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:37:51.118671 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:37:51.118671 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:37:51.118671 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 09:37:51.118671 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 09:37:51.118671 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 09:37:51.118671 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 09:37:51.119777 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:37:51.119777 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:37:51.119777 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:37:51.119777 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:37:51.119777 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:37:51.119777 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:37:51.119777 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:37:51.119777 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application with Lock( Aug 31 09:37:51.119777 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:37:51.119777 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:37:51.119777 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:37:51.119777 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:37:51.119777 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:37:51.119777 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:37:51.119777 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:37:51.119777 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:37:51.119777 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 09:37:51.120729 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 09:37:51.120729 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 09:37:51.120729 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application Aug 31 09:37:51.120729 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 621/1237] 10.222.0.13 () {56 vars in 1108 bytes} [Wed Aug 31 09:37:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:37:51.125953 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a32d9d1-d5f3-4c9e-a8b1-e46c8fb43b46 None admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:37:51.249767 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a32d9d1-d5f3-4c9e-a8b1-e46c8fb43b46 None admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:37:51.259694 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=100 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:37:51.292114 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff07bb4-03ab-467c-ad2a-9154bd1c3a3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:37:51.292818 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff07bb4-03ab-467c-ad2a-9154bd1c3a3e None admin] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:37:51.293122 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff07bb4-03ab-467c-ad2a-9154bd1c3a3e None admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:37:51.293474 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff07bb4-03ab-467c-ad2a-9154bd1c3a3e None admin] PATH_INFO: `/v3/domains` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:37:51.294131 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eff07bb4-03ab-467c-ad2a-9154bd1c3a3e None admin] RBAC: Adding query filter params (name=Default) {{(pid=88114) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:37:51.294547 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eff07bb4-03ab-467c-ad2a-9154bd1c3a3e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:37:51.296236 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eff07bb4-03ab-467c-ad2a-9154bd1c3a3e None admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:37:51.310209 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 617/1238] 10.222.0.13 () {56 vars in 1109 bytes} [Wed Aug 31 09:37:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:37:51.317110 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48541f7b-5e56-4a5e-bffb-c31c0447b89b None admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:37:51.434514 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48541f7b-5e56-4a5e-bffb-c31c0447b89b None admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:37:51.471975 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b25cce3-de54-491b-a0ab-fbd8761f0da0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:37:51.472615 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b25cce3-de54-491b-a0ab-fbd8761f0da0 None admin] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:37:51.472896 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b25cce3-de54-491b-a0ab-fbd8761f0da0 None admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:37:51.473164 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b25cce3-de54-491b-a0ab-fbd8761f0da0 None admin] PATH_INFO: `/v3/users/placement` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:37:51.476541 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=100 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:37:51.481996 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b25cce3-de54-491b-a0ab-fbd8761f0da0 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:37:51.483475 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b25cce3-de54-491b-a0ab-fbd8761f0da0 None admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:37:51.495682 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application [None req-9b25cce3-de54-491b-a0ab-fbd8761f0da0 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 31 09:37:51.495682 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:37:51.495682 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:37:51.495682 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:37:51.495682 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:37:51.495682 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:37:51.495682 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:37:51.495682 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:37:51.495682 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:37:51.495682 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:37:51.495682 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:37:51.495682 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:37:51.495682 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 31 09:37:51.495682 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 31 09:37:51.495682 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 31 09:37:51.495682 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 31 09:37:51.496707 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:37:51.496707 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:37:51.496707 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 31 09:37:51.496707 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 09:37:51.496707 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 31 09:37:51.496707 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 09:37:51.496707 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:37:51.496707 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:37:51.496707 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:37:51.496707 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:37:51.496707 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:37:51.496707 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application with Lock( Aug 31 09:37:51.496707 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:37:51.496707 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:37:51.496707 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:37:51.496707 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:37:51.496707 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:37:51.496707 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:37:51.497704 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:37:51.497704 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:37:51.497704 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 31 09:37:51.497704 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 31 09:37:51.497704 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 31 09:37:51.497704 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 31 09:37:51.497704 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 31 09:37:51.497704 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 31 09:37:51.497704 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 31 09:37:51.497704 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application Aug 31 09:37:51.497704 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 622/1239] 10.222.0.13 () {56 vars in 1108 bytes} [Wed Aug 31 09:37:51 2022] GET /identity/v3/users/placement => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:37:51.503201 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff07bb4-03ab-467c-ad2a-9154bd1c3a3e None admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:37:51.620481 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff07bb4-03ab-467c-ad2a-9154bd1c3a3e None admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:37:51.661915 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-969e5c90-1729-4049-a9ed-86511aad8087 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:37:51.662526 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969e5c90-1729-4049-a9ed-86511aad8087 None admin] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:37:51.662803 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969e5c90-1729-4049-a9ed-86511aad8087 None admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:37:51.663072 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969e5c90-1729-4049-a9ed-86511aad8087 None admin] PATH_INFO: `/v3/users` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:37:51.664069 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-969e5c90-1729-4049-a9ed-86511aad8087 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=88114) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:37:51.664373 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-969e5c90-1729-4049-a9ed-86511aad8087 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:37:51.666026 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-969e5c90-1729-4049-a9ed-86511aad8087 None admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:37:51.687458 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 618/1240] 10.222.0.13 () {56 vars in 1143 bytes} [Wed Aug 31 09:37:51 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:37:51.693351 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b25cce3-de54-491b-a0ab-fbd8761f0da0 None admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:37:51.813648 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b25cce3-de54-491b-a0ab-fbd8761f0da0 None admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:37:51.851762 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16dbc6bd-53c5-4de7-8a95-f10978eb07be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:37:51.852362 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16dbc6bd-53c5-4de7-8a95-f10978eb07be None admin] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:37:51.852640 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16dbc6bd-53c5-4de7-8a95-f10978eb07be None admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:37:51.852936 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16dbc6bd-53c5-4de7-8a95-f10978eb07be None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:37:51.932167 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=100 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:37:51.943792 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16dbc6bd-53c5-4de7-8a95-f10978eb07be None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:37:51.946161 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16dbc6bd-53c5-4de7-8a95-f10978eb07be None admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:37:52.072864 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 623/1241] 10.222.0.13 () {58 vars in 1282 bytes} [Wed Aug 31 09:37:51 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 380 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 31 09:37:52.078945 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-969e5c90-1729-4049-a9ed-86511aad8087 None admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:37:52.112530 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=100 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:37:52.199464 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-969e5c90-1729-4049-a9ed-86511aad8087 None admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:37:52.238910 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69dc88f9-9928-44c8-b5d5-150781adf46e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:37:52.239596 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69dc88f9-9928-44c8-b5d5-150781adf46e None admin] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:37:52.239880 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69dc88f9-9928-44c8-b5d5-150781adf46e None admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:37:52.240193 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69dc88f9-9928-44c8-b5d5-150781adf46e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:37:52.252878 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69dc88f9-9928-44c8-b5d5-150781adf46e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:37:52.254409 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69dc88f9-9928-44c8-b5d5-150781adf46e None admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:37:52.273593 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application [None req-69dc88f9-9928-44c8-b5d5-150781adf46e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 09:37:52.273593 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:37:52.273593 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:37:52.273593 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:37:52.273593 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:37:52.273593 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:37:52.273593 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:37:52.273593 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:37:52.273593 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:37:52.273593 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:37:52.273593 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:37:52.273593 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:37:52.273593 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 09:37:52.273593 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 09:37:52.273593 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 09:37:52.273593 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 09:37:52.275016 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:37:52.275016 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:37:52.275016 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:37:52.275016 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:37:52.275016 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:37:52.275016 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:37:52.275016 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:37:52.275016 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application with Lock( Aug 31 09:37:52.275016 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:37:52.275016 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:37:52.275016 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:37:52.275016 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:37:52.275016 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:37:52.275016 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:37:52.275016 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:37:52.275016 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:37:52.275016 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 09:37:52.276211 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 09:37:52.276211 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 09:37:52.276211 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application Aug 31 09:37:52.276211 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 619/1242] 10.222.0.13 () {56 vars in 1108 bytes} [Wed Aug 31 09:37:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:37:52.280639 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16dbc6bd-53c5-4de7-8a95-f10978eb07be None admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:37:52.386377 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=100 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:37:52.393644 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16dbc6bd-53c5-4de7-8a95-f10978eb07be None admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:37:52.432047 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f1682f-717e-47ed-a437-f374281b5e04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:37:52.432683 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f1682f-717e-47ed-a437-f374281b5e04 None admin] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:37:52.432956 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f1682f-717e-47ed-a437-f374281b5e04 None admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:37:52.433231 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f1682f-717e-47ed-a437-f374281b5e04 None admin] PATH_INFO: `/v3/domains` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:37:52.434142 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64f1682f-717e-47ed-a437-f374281b5e04 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88113) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:37:52.434390 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64f1682f-717e-47ed-a437-f374281b5e04 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:37:52.435883 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64f1682f-717e-47ed-a437-f374281b5e04 None admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:37:52.448578 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 624/1243] 10.222.0.13 () {56 vars in 1109 bytes} [Wed Aug 31 09:37:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:37:52.454734 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69dc88f9-9928-44c8-b5d5-150781adf46e None admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:37:52.542583 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=100 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:37:52.572207 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69dc88f9-9928-44c8-b5d5-150781adf46e None admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:37:52.610231 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96eeae1a-77e7-406a-8c7a-76bfe0534e61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:37:52.611018 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96eeae1a-77e7-406a-8c7a-76bfe0534e61 None admin] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:37:52.611394 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96eeae1a-77e7-406a-8c7a-76bfe0534e61 None admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:37:52.611709 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96eeae1a-77e7-406a-8c7a-76bfe0534e61 None admin] PATH_INFO: `/v3/projects/service` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:37:52.619768 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96eeae1a-77e7-406a-8c7a-76bfe0534e61 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:37:52.621314 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96eeae1a-77e7-406a-8c7a-76bfe0534e61 None admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:37:52.633666 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application [None req-96eeae1a-77e7-406a-8c7a-76bfe0534e61 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 09:37:52.633666 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:37:52.633666 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:37:52.633666 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:37:52.633666 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:37:52.633666 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:37:52.633666 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:37:52.633666 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:37:52.633666 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:37:52.633666 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:37:52.633666 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:37:52.633666 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:37:52.633666 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 31 09:37:52.633666 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 31 09:37:52.633666 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 31 09:37:52.636517 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 31 09:37:52.636517 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:37:52.636517 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:37:52.636517 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:37:52.636517 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:37:52.636517 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:37:52.636517 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:37:52.636517 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:37:52.636517 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application with Lock( Aug 31 09:37:52.636517 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:37:52.636517 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:37:52.636517 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:37:52.636517 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:37:52.636517 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:37:52.636517 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:37:52.636517 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:37:52.636517 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:37:52.637644 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 31 09:37:52.637644 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 31 09:37:52.637644 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 31 09:37:52.637644 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 31 09:37:52.637644 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 31 09:37:52.637644 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 31 09:37:52.637644 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 09:37:52.637644 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application Aug 31 09:37:52.637644 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 620/1244] 10.222.0.13 () {56 vars in 1111 bytes} [Wed Aug 31 09:37:52 2022] GET /identity/v3/projects/service => generated 88 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 09:37:52.641094 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f1682f-717e-47ed-a437-f374281b5e04 None admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:37:52.762217 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f1682f-717e-47ed-a437-f374281b5e04 None admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:37:52.805658 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2515fbc-fa6a-4657-9632-7818d7e6283c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:37:52.806321 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2515fbc-fa6a-4657-9632-7818d7e6283c None admin] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:37:52.806796 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2515fbc-fa6a-4657-9632-7818d7e6283c None admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:37:52.807128 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2515fbc-fa6a-4657-9632-7818d7e6283c None admin] PATH_INFO: `/v3/projects` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:37:52.807868 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2515fbc-fa6a-4657-9632-7818d7e6283c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88113) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:37:52.808328 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2515fbc-fa6a-4657-9632-7818d7e6283c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:37:52.810156 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2515fbc-fa6a-4657-9632-7818d7e6283c None admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:37:52.828048 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 625/1245] 10.222.0.13 () {56 vars in 1148 bytes} [Wed Aug 31 09:37:52 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:37:52.834606 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96eeae1a-77e7-406a-8c7a-76bfe0534e61 None admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:37:52.967045 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96eeae1a-77e7-406a-8c7a-76bfe0534e61 None admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:37:53.020221 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0246b4ac-7d93-4d9b-b06a-6527cc016331 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:37:53.020864 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0246b4ac-7d93-4d9b-b06a-6527cc016331 None admin] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:37:53.021214 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0246b4ac-7d93-4d9b-b06a-6527cc016331 None admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:37:53.021677 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0246b4ac-7d93-4d9b-b06a-6527cc016331 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:37:53.022574 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0246b4ac-7d93-4d9b-b06a-6527cc016331 None admin] RBAC: Adding query filter params (role.id=e24ba4fd91584883af350cc84bc96c1d, scope.project.id=49548eb44d62415582af820ab66decee, user.id=1ff3aff5ef444910be6dbb989d0e3194) {{(pid=88114) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:37:53.024253 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0246b4ac-7d93-4d9b-b06a-6527cc016331 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:37:53.026035 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0246b4ac-7d93-4d9b-b06a-6527cc016331 None admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:37:53.042001 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 621/1246] 10.222.0.13 () {56 vars in 1374 bytes} [Wed Aug 31 09:37:52 2022] GET /identity/v3/role_assignments?user.id=1ff3aff5ef444910be6dbb989d0e3194&scope.project.id=49548eb44d62415582af820ab66decee&role.id=e24ba4fd91584883af350cc84bc96c1d => generated 608 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:37:55.047871 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed933d6a-db1e-468c-a734-4331ec53b713 None None] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:37:55.048434 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed933d6a-db1e-468c-a734-4331ec53b713 None None] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:37:55.048434 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed933d6a-db1e-468c-a734-4331ec53b713 None None] PATH_INFO: `/` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:37:55.049772 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 626/1247] 10.222.0.13 () {54 vars in 927 bytes} [Wed Aug 31 09:37:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 31 09:37:55.055764 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f47aa5-edfe-471d-9959-866d83d5b59d None None] REQUEST_METHOD: `POST` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:37:55.056308 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f47aa5-edfe-471d-9959-866d83d5b59d None None] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:37:55.056514 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f47aa5-edfe-471d-9959-866d83d5b59d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:37:55.064008 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=100 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:37:55.077627 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.auth.core [None req-d1f47aa5-edfe-471d-9959-866d83d5b59d None None] MFA Rules not processed for user `fd836cf590a145df9acf19cf3ae2ac56`. Rule list: `[]` (Enabled: `True`). {{(pid=88114) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:37:55.078899 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.fernet_utils [None req-d1f47aa5-edfe-471d-9959-866d83d5b59d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88114) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:37:55.154063 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 622/1248] 10.222.0.13 () {58 vars in 1024 bytes} [Wed Aug 31 09:37:55 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 09:37:55.436240 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed933d6a-db1e-468c-a734-4331ec53b713 None None] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:37:55.456350 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=100 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:37:55.555858 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed933d6a-db1e-468c-a734-4331ec53b713 None None] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:37:55.595266 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb1725f-efae-45b1-b62d-0f170eba003b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:37:55.595949 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb1725f-efae-45b1-b62d-0f170eba003b None admin] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:37:55.596231 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb1725f-efae-45b1-b62d-0f170eba003b None admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:37:55.596501 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb1725f-efae-45b1-b62d-0f170eba003b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:37:55.609592 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbb1725f-efae-45b1-b62d-0f170eba003b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 09:37:55.614489 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application [None req-fbb1725f-efae-45b1-b62d-0f170eba003b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 09:37:55.614489 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:37:55.614489 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:37:55.614489 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:37:55.614489 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:37:55.614489 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:37:55.614489 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:37:55.614489 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:37:55.614489 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:37:55.614489 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:37:55.614489 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:37:55.614489 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:37:55.614489 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 09:37:55.614489 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 09:37:55.614489 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 09:37:55.614489 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 09:37:55.615497 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 09:37:55.615497 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 09:37:55.615497 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 09:37:55.615497 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 09:37:55.615497 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:37:55.615497 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:37:55.615497 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:37:55.615497 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:37:55.615497 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:37:55.615497 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:37:55.615497 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:37:55.615497 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application with Lock( Aug 31 09:37:55.615497 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:37:55.615497 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:37:55.615497 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:37:55.615497 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:37:55.617069 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:37:55.617069 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:37:55.617069 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:37:55.617069 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:37:55.617069 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 09:37:55.617069 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 09:37:55.617069 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 09:37:55.617069 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 09:37:55.617069 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 09:37:55.617069 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 09:37:55.617069 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 09:37:55.617069 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application Aug 31 09:37:55.617069 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 627/1249] 10.222.0.13 () {56 vars in 1096 bytes} [Wed Aug 31 09:37:55 2022] GET /identity/v3/roles/admin => generated 83 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:37:55.622789 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f47aa5-edfe-471d-9959-866d83d5b59d None None] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:37:55.743844 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f47aa5-edfe-471d-9959-866d83d5b59d None None] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:37:55.777684 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=100 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:37:55.784719 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-672b083b-df5e-48cc-94c0-3d63392e7696 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:37:55.785718 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672b083b-df5e-48cc-94c0-3d63392e7696 None admin] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:37:55.786192 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672b083b-df5e-48cc-94c0-3d63392e7696 None admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:37:55.786661 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672b083b-df5e-48cc-94c0-3d63392e7696 None admin] PATH_INFO: `/v3/roles` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:37:55.787639 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-672b083b-df5e-48cc-94c0-3d63392e7696 None admin] RBAC: Adding query filter params (name=admin) {{(pid=88114) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:37:55.788078 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-672b083b-df5e-48cc-94c0-3d63392e7696 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:37:55.791051 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-672b083b-df5e-48cc-94c0-3d63392e7696 None admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:37:55.804858 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 623/1250] 10.222.0.13 () {56 vars in 1099 bytes} [Wed Aug 31 09:37:55 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:37:55.811066 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb1725f-efae-45b1-b62d-0f170eba003b None admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:37:55.922699 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=100 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:37:55.952673 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb1725f-efae-45b1-b62d-0f170eba003b None admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:37:55.996928 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21bc63e0-a495-406a-a6e8-93fedd1a1b68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:37:55.997678 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21bc63e0-a495-406a-a6e8-93fedd1a1b68 None admin] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:37:55.997955 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21bc63e0-a495-406a-a6e8-93fedd1a1b68 None admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:37:55.998259 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21bc63e0-a495-406a-a6e8-93fedd1a1b68 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:37:56.012816 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21bc63e0-a495-406a-a6e8-93fedd1a1b68 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:37:56.014336 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21bc63e0-a495-406a-a6e8-93fedd1a1b68 None admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:37:56.032908 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application [None req-21bc63e0-a495-406a-a6e8-93fedd1a1b68 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 09:37:56.032908 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:37:56.032908 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:37:56.032908 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:37:56.032908 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:37:56.032908 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:37:56.032908 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:37:56.032908 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:37:56.032908 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:37:56.032908 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:37:56.032908 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:37:56.032908 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:37:56.032908 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 09:37:56.032908 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 09:37:56.032908 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 09:37:56.032908 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 09:37:56.034274 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:37:56.034274 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:37:56.034274 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:37:56.034274 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:37:56.034274 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:37:56.034274 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:37:56.034274 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:37:56.034274 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application with Lock( Aug 31 09:37:56.034274 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:37:56.034274 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:37:56.034274 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:37:56.034274 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:37:56.034274 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:37:56.034274 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:37:56.034274 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:37:56.034274 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:37:56.034274 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 09:37:56.035363 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 09:37:56.035363 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 09:37:56.035363 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application Aug 31 09:37:56.035363 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 628/1251] 10.222.0.13 () {56 vars in 1108 bytes} [Wed Aug 31 09:37:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 09:37:56.042536 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-672b083b-df5e-48cc-94c0-3d63392e7696 None admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:37:56.163539 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-672b083b-df5e-48cc-94c0-3d63392e7696 None admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:37:56.204069 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db4e4493-5d7d-41a0-be22-aceca03884ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:37:56.204723 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4e4493-5d7d-41a0-be22-aceca03884ac None admin] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:37:56.205043 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4e4493-5d7d-41a0-be22-aceca03884ac None admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:37:56.205321 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4e4493-5d7d-41a0-be22-aceca03884ac None admin] PATH_INFO: `/v3/domains` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:37:56.206036 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db4e4493-5d7d-41a0-be22-aceca03884ac None admin] RBAC: Adding query filter params (name=Default) {{(pid=88114) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:37:56.206342 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db4e4493-5d7d-41a0-be22-aceca03884ac None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:37:56.207789 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db4e4493-5d7d-41a0-be22-aceca03884ac None admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:37:56.227048 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 624/1252] 10.222.0.13 () {56 vars in 1109 bytes} [Wed Aug 31 09:37:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:37:56.228275 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21bc63e0-a495-406a-a6e8-93fedd1a1b68 None admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:37:56.339399 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21bc63e0-a495-406a-a6e8-93fedd1a1b68 None admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:37:56.368159 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=100 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:37:56.377292 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b24f70ae-c8b1-4cb5-9ab7-072214f8505f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:37:56.377934 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24f70ae-c8b1-4cb5-9ab7-072214f8505f None admin] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:37:56.378206 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24f70ae-c8b1-4cb5-9ab7-072214f8505f None admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:37:56.378486 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24f70ae-c8b1-4cb5-9ab7-072214f8505f None admin] PATH_INFO: `/v3/users/placement` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:37:56.387545 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b24f70ae-c8b1-4cb5-9ab7-072214f8505f None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:37:56.389067 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b24f70ae-c8b1-4cb5-9ab7-072214f8505f None admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:37:56.401402 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application [None req-b24f70ae-c8b1-4cb5-9ab7-072214f8505f None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 31 09:37:56.401402 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:37:56.401402 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:37:56.401402 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:37:56.401402 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:37:56.401402 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:37:56.401402 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:37:56.401402 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:37:56.401402 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:37:56.401402 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:37:56.401402 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:37:56.401402 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:37:56.401402 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 31 09:37:56.401402 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 31 09:37:56.401402 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 31 09:37:56.401402 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 31 09:37:56.402621 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:37:56.402621 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:37:56.402621 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 31 09:37:56.402621 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 09:37:56.402621 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 31 09:37:56.402621 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 09:37:56.402621 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:37:56.402621 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:37:56.402621 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:37:56.402621 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:37:56.402621 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:37:56.402621 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application with Lock( Aug 31 09:37:56.402621 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:37:56.402621 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:37:56.402621 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:37:56.402621 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:37:56.402621 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:37:56.402621 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:37:56.403801 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:37:56.403801 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:37:56.403801 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 31 09:37:56.403801 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 31 09:37:56.403801 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 31 09:37:56.403801 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 31 09:37:56.403801 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 31 09:37:56.403801 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 31 09:37:56.403801 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 31 09:37:56.403801 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application Aug 31 09:37:56.403801 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 629/1253] 10.222.0.13 () {56 vars in 1108 bytes} [Wed Aug 31 09:37:56 2022] GET /identity/v3/users/placement => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 09:37:56.408512 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db4e4493-5d7d-41a0-be22-aceca03884ac None admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:37:56.473583 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=100 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:37:56.528275 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db4e4493-5d7d-41a0-be22-aceca03884ac None admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:37:56.564719 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2965c15-600d-4278-a690-89444f100f31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:37:56.566155 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2965c15-600d-4278-a690-89444f100f31 None admin] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:37:56.566434 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2965c15-600d-4278-a690-89444f100f31 None admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:37:56.566702 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2965c15-600d-4278-a690-89444f100f31 None admin] PATH_INFO: `/v3/users` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:37:56.567433 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2965c15-600d-4278-a690-89444f100f31 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=88114) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:37:56.567734 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2965c15-600d-4278-a690-89444f100f31 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:37:56.569154 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2965c15-600d-4278-a690-89444f100f31 None admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:37:56.587472 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 625/1254] 10.222.0.13 () {56 vars in 1143 bytes} [Wed Aug 31 09:37:56 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:37:56.593332 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b24f70ae-c8b1-4cb5-9ab7-072214f8505f None admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:37:56.702701 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b24f70ae-c8b1-4cb5-9ab7-072214f8505f None admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:37:56.742814 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbacfbb3-0e54-4365-8d45-cdb3efad3e86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:37:56.743577 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbacfbb3-0e54-4365-8d45-cdb3efad3e86 None admin] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:37:56.743881 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbacfbb3-0e54-4365-8d45-cdb3efad3e86 None admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:37:56.744147 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbacfbb3-0e54-4365-8d45-cdb3efad3e86 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:37:56.804401 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=100 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:37:56.818634 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbacfbb3-0e54-4365-8d45-cdb3efad3e86 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:37:56.821242 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbacfbb3-0e54-4365-8d45-cdb3efad3e86 None admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:37:56.943751 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 630/1255] 10.222.0.13 () {58 vars in 1282 bytes} [Wed Aug 31 09:37:56 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 351 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 31 09:37:56.950334 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2965c15-600d-4278-a690-89444f100f31 None admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:37:57.059546 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2965c15-600d-4278-a690-89444f100f31 None admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:37:57.074253 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=100 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:37:57.098883 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac6a0c1-6690-48ff-b765-f7b1b8ec4e01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:37:57.099562 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac6a0c1-6690-48ff-b765-f7b1b8ec4e01 None admin] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:37:57.099846 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac6a0c1-6690-48ff-b765-f7b1b8ec4e01 None admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:37:57.100144 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac6a0c1-6690-48ff-b765-f7b1b8ec4e01 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:37:57.110735 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ac6a0c1-6690-48ff-b765-f7b1b8ec4e01 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:37:57.112154 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ac6a0c1-6690-48ff-b765-f7b1b8ec4e01 None admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:37:57.128633 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application [None req-1ac6a0c1-6690-48ff-b765-f7b1b8ec4e01 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 09:37:57.128633 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:37:57.128633 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:37:57.128633 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:37:57.128633 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:37:57.128633 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:37:57.128633 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:37:57.128633 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:37:57.128633 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:37:57.128633 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:37:57.128633 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:37:57.128633 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:37:57.128633 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 09:37:57.128633 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 09:37:57.128633 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 09:37:57.128633 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 09:37:57.129733 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:37:57.129733 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:37:57.129733 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:37:57.129733 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:37:57.129733 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:37:57.129733 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:37:57.129733 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:37:57.129733 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application with Lock( Aug 31 09:37:57.129733 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:37:57.129733 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:37:57.129733 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:37:57.129733 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:37:57.129733 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:37:57.129733 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:37:57.129733 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:37:57.129733 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:37:57.129733 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 09:37:57.130644 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 09:37:57.130644 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 09:37:57.130644 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application Aug 31 09:37:57.130644 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 626/1256] 10.222.0.13 () {56 vars in 1108 bytes} [Wed Aug 31 09:37:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:37:57.136665 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbacfbb3-0e54-4365-8d45-cdb3efad3e86 None admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:37:57.230746 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=100 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:37:57.243451 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbacfbb3-0e54-4365-8d45-cdb3efad3e86 None admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:37:57.279895 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e38465-946d-4043-8ae9-bdd2b176da29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:37:57.280506 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e38465-946d-4043-8ae9-bdd2b176da29 None admin] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:37:57.280785 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e38465-946d-4043-8ae9-bdd2b176da29 None admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:37:57.281058 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e38465-946d-4043-8ae9-bdd2b176da29 None admin] PATH_INFO: `/v3/domains` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:37:57.281735 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02e38465-946d-4043-8ae9-bdd2b176da29 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88113) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:37:57.282039 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02e38465-946d-4043-8ae9-bdd2b176da29 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:37:57.283408 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02e38465-946d-4043-8ae9-bdd2b176da29 None admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:37:57.295150 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 631/1257] 10.222.0.13 () {56 vars in 1109 bytes} [Wed Aug 31 09:37:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:37:57.300912 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac6a0c1-6690-48ff-b765-f7b1b8ec4e01 None admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:37:57.420645 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac6a0c1-6690-48ff-b765-f7b1b8ec4e01 None admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:37:57.458316 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c895038-ff89-4f70-a9c1-a2ae841240f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:37:57.458991 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c895038-ff89-4f70-a9c1-a2ae841240f9 None admin] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:37:57.459270 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c895038-ff89-4f70-a9c1-a2ae841240f9 None admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:37:57.459545 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c895038-ff89-4f70-a9c1-a2ae841240f9 None admin] PATH_INFO: `/v3/projects/service` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:37:57.467531 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c895038-ff89-4f70-a9c1-a2ae841240f9 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:37:57.469070 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c895038-ff89-4f70-a9c1-a2ae841240f9 None admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:37:57.480565 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application [None req-1c895038-ff89-4f70-a9c1-a2ae841240f9 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 09:37:57.480565 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:37:57.480565 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:37:57.480565 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:37:57.480565 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:37:57.480565 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:37:57.480565 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:37:57.480565 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:37:57.480565 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:37:57.480565 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:37:57.480565 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:37:57.480565 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:37:57.480565 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 31 09:37:57.480565 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 31 09:37:57.480565 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 31 09:37:57.481614 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 31 09:37:57.481614 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:37:57.481614 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:37:57.481614 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:37:57.481614 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:37:57.481614 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:37:57.481614 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:37:57.481614 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:37:57.481614 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application with Lock( Aug 31 09:37:57.481614 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:37:57.481614 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:37:57.481614 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:37:57.481614 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:37:57.481614 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:37:57.481614 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:37:57.481614 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:37:57.481614 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:37:57.482602 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 31 09:37:57.482602 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 31 09:37:57.482602 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 31 09:37:57.482602 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 31 09:37:57.482602 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 31 09:37:57.482602 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 31 09:37:57.482602 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 09:37:57.482602 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application Aug 31 09:37:57.482602 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 627/1258] 10.222.0.13 () {56 vars in 1111 bytes} [Wed Aug 31 09:37:57 2022] GET /identity/v3/projects/service => generated 88 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:37:57.488163 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e38465-946d-4043-8ae9-bdd2b176da29 None admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:37:57.608868 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e38465-946d-4043-8ae9-bdd2b176da29 None admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:37:57.645345 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d6cef5-5605-40e6-9820-3b31cd944ae3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:37:57.645986 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d6cef5-5605-40e6-9820-3b31cd944ae3 None admin] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:37:57.646285 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d6cef5-5605-40e6-9820-3b31cd944ae3 None admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:37:57.646530 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d6cef5-5605-40e6-9820-3b31cd944ae3 None admin] PATH_INFO: `/v3/projects` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:37:57.647227 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33d6cef5-5605-40e6-9820-3b31cd944ae3 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88113) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:37:57.648339 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33d6cef5-5605-40e6-9820-3b31cd944ae3 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:37:57.650137 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33d6cef5-5605-40e6-9820-3b31cd944ae3 None admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:37:57.662579 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 632/1259] 10.222.0.13 () {56 vars in 1148 bytes} [Wed Aug 31 09:37:57 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:37:57.668951 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c895038-ff89-4f70-a9c1-a2ae841240f9 None admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:37:57.696273 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=100 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:37:57.779282 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c895038-ff89-4f70-a9c1-a2ae841240f9 None admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:37:57.814004 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eaed3d4-2824-457c-8a6b-36059d9723a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:37:57.814609 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eaed3d4-2824-457c-8a6b-36059d9723a8 None admin] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:37:57.814883 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eaed3d4-2824-457c-8a6b-36059d9723a8 None admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:37:57.815163 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eaed3d4-2824-457c-8a6b-36059d9723a8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:37:57.815869 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eaed3d4-2824-457c-8a6b-36059d9723a8 None admin] RBAC: Adding query filter params (role.id=5d4ee63a3add47fc9a6b8b16d39e0a1f, scope.project.id=49548eb44d62415582af820ab66decee, user.id=1ff3aff5ef444910be6dbb989d0e3194) {{(pid=88114) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:37:57.816173 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eaed3d4-2824-457c-8a6b-36059d9723a8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:37:57.817691 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eaed3d4-2824-457c-8a6b-36059d9723a8 None admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:37:57.823684 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 628/1260] 10.222.0.13 () {56 vars in 1374 bytes} [Wed Aug 31 09:37:57 2022] GET /identity/v3/role_assignments?user.id=1ff3aff5ef444910be6dbb989d0e3194&scope.project.id=49548eb44d62415582af820ab66decee&role.id=5d4ee63a3add47fc9a6b8b16d39e0a1f => generated 259 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:37:59.739475 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7824373c-022e-4ab7-9a75-f18c111cc889 None None] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:37:59.740053 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7824373c-022e-4ab7-9a75-f18c111cc889 None None] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:37:59.740053 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7824373c-022e-4ab7-9a75-f18c111cc889 None None] PATH_INFO: `/` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:37:59.741372 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 633/1261] 10.222.0.13 () {54 vars in 927 bytes} [Wed Aug 31 09:37:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:37:59.747084 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a77408-3b80-438b-aaf4-6ff066e2b4ab None None] REQUEST_METHOD: `POST` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:37:59.747513 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a77408-3b80-438b-aaf4-6ff066e2b4ab None None] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:37:59.747620 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a77408-3b80-438b-aaf4-6ff066e2b4ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:37:59.767905 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.auth.core [None req-61a77408-3b80-438b-aaf4-6ff066e2b4ab None None] MFA Rules not processed for user `fd836cf590a145df9acf19cf3ae2ac56`. Rule list: `[]` (Enabled: `True`). {{(pid=88114) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:37:59.770370 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.fernet_utils [None req-61a77408-3b80-438b-aaf4-6ff066e2b4ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88114) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:37:59.839355 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 629/1262] 10.222.0.13 () {58 vars in 1024 bytes} [Wed Aug 31 09:37:59 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 09:38:00.137100 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7824373c-022e-4ab7-9a75-f18c111cc889 None None] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:38:00.148976 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=100 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:38:00.243773 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7824373c-022e-4ab7-9a75-f18c111cc889 None None] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:38:00.279897 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc83c17-9b98-4964-8f6f-1f9e504fb1a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:38:00.280530 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc83c17-9b98-4964-8f6f-1f9e504fb1a9 None admin] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:38:00.280816 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc83c17-9b98-4964-8f6f-1f9e504fb1a9 None admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:38:00.281091 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc83c17-9b98-4964-8f6f-1f9e504fb1a9 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:38:00.291950 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dc83c17-9b98-4964-8f6f-1f9e504fb1a9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 09:38:00.297246 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application [None req-7dc83c17-9b98-4964-8f6f-1f9e504fb1a9 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 09:38:00.297246 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:38:00.297246 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:38:00.297246 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:38:00.297246 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:38:00.297246 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:38:00.297246 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:38:00.297246 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:38:00.297246 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:38:00.297246 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:38:00.297246 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:38:00.297246 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:38:00.297246 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 09:38:00.297246 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 09:38:00.297246 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 09:38:00.297246 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 09:38:00.298373 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 09:38:00.298373 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 09:38:00.298373 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 09:38:00.298373 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 09:38:00.298373 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:38:00.298373 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:38:00.298373 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:38:00.298373 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:38:00.298373 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:38:00.298373 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:38:00.298373 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:38:00.298373 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application with Lock( Aug 31 09:38:00.298373 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:38:00.298373 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:38:00.298373 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:38:00.298373 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:38:00.299223 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:38:00.299223 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:38:00.299223 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:38:00.299223 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:38:00.299223 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 09:38:00.299223 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 09:38:00.299223 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 09:38:00.299223 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 09:38:00.299223 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 09:38:00.299223 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 09:38:00.299223 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 09:38:00.299223 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application Aug 31 09:38:00.299223 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 634/1263] 10.222.0.13 () {56 vars in 1096 bytes} [Wed Aug 31 09:38:00 2022] GET /identity/v3/roles/admin => generated 83 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:38:00.304817 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a77408-3b80-438b-aaf4-6ff066e2b4ab None None] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:38:00.323596 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=100 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:38:00.406262 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a77408-3b80-438b-aaf4-6ff066e2b4ab None None] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:38:00.440921 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fb5581c-eb1b-46aa-806c-0a5449d58eed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:38:00.441557 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb5581c-eb1b-46aa-806c-0a5449d58eed None admin] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:38:00.441834 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb5581c-eb1b-46aa-806c-0a5449d58eed None admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:38:00.442103 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb5581c-eb1b-46aa-806c-0a5449d58eed None admin] PATH_INFO: `/v3/roles` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:38:00.442796 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fb5581c-eb1b-46aa-806c-0a5449d58eed None admin] RBAC: Adding query filter params (name=admin) {{(pid=88114) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:38:00.443093 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fb5581c-eb1b-46aa-806c-0a5449d58eed None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:38:00.444961 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fb5581c-eb1b-46aa-806c-0a5449d58eed None admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:38:00.453970 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 630/1264] 10.222.0.13 () {56 vars in 1099 bytes} [Wed Aug 31 09:38:00 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 09:38:00.459969 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc83c17-9b98-4964-8f6f-1f9e504fb1a9 None admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:38:00.530245 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=100 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:38:00.563150 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc83c17-9b98-4964-8f6f-1f9e504fb1a9 None admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:38:00.598024 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b17d563d-87c8-4bdd-a820-515038e7d054 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:38:00.598649 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17d563d-87c8-4bdd-a820-515038e7d054 None admin] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:38:00.598925 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17d563d-87c8-4bdd-a820-515038e7d054 None admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:38:00.599194 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17d563d-87c8-4bdd-a820-515038e7d054 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:38:00.609646 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b17d563d-87c8-4bdd-a820-515038e7d054 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:38:00.611114 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b17d563d-87c8-4bdd-a820-515038e7d054 None admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:38:00.625800 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application [None req-b17d563d-87c8-4bdd-a820-515038e7d054 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 09:38:00.625800 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:38:00.625800 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:38:00.625800 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:38:00.625800 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:38:00.625800 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:38:00.625800 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:38:00.625800 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:38:00.625800 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:38:00.625800 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:38:00.625800 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:38:00.625800 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:38:00.625800 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 09:38:00.625800 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 09:38:00.625800 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 09:38:00.625800 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 09:38:00.627041 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:38:00.627041 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:38:00.627041 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:38:00.627041 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:38:00.627041 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:38:00.627041 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:38:00.627041 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:38:00.627041 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application with Lock( Aug 31 09:38:00.627041 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:38:00.627041 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:38:00.627041 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:38:00.627041 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:38:00.627041 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:38:00.627041 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:38:00.627041 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:38:00.627041 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:38:00.627041 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 09:38:00.628119 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 09:38:00.628119 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 09:38:00.628119 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application Aug 31 09:38:00.628119 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 635/1265] 10.222.0.13 () {56 vars in 1108 bytes} [Wed Aug 31 09:38:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:38:00.634076 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fb5581c-eb1b-46aa-806c-0a5449d58eed None admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:38:00.719581 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=100 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:38:00.739627 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fb5581c-eb1b-46aa-806c-0a5449d58eed None admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:38:00.783370 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab26f453-57a4-4d23-b49e-ceeb73503b7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:38:00.784027 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab26f453-57a4-4d23-b49e-ceeb73503b7d None admin] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:38:00.784231 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab26f453-57a4-4d23-b49e-ceeb73503b7d None admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:38:00.784503 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab26f453-57a4-4d23-b49e-ceeb73503b7d None admin] PATH_INFO: `/v3/domains` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:38:00.785140 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab26f453-57a4-4d23-b49e-ceeb73503b7d None admin] RBAC: Adding query filter params (name=Default) {{(pid=88114) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:38:00.785465 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab26f453-57a4-4d23-b49e-ceeb73503b7d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:38:00.786822 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab26f453-57a4-4d23-b49e-ceeb73503b7d None admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:38:00.799401 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 631/1266] 10.222.0.13 () {56 vars in 1109 bytes} [Wed Aug 31 09:38:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:38:00.805794 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b17d563d-87c8-4bdd-a820-515038e7d054 None admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:38:00.912778 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b17d563d-87c8-4bdd-a820-515038e7d054 None admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:38:00.937530 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=100 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:38:00.953299 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d804156b-4a06-4fe2-8637-ae0601e67852 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:38:00.953972 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d804156b-4a06-4fe2-8637-ae0601e67852 None admin] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:38:00.954251 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d804156b-4a06-4fe2-8637-ae0601e67852 None admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:38:00.954524 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d804156b-4a06-4fe2-8637-ae0601e67852 None admin] PATH_INFO: `/v3/users/placement` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:38:00.962766 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d804156b-4a06-4fe2-8637-ae0601e67852 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:38:00.964334 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d804156b-4a06-4fe2-8637-ae0601e67852 None admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:38:00.976533 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application [None req-d804156b-4a06-4fe2-8637-ae0601e67852 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 31 09:38:00.976533 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:38:00.976533 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:38:00.976533 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:38:00.976533 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:38:00.976533 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:38:00.976533 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:38:00.976533 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:38:00.976533 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:38:00.976533 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:38:00.976533 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:38:00.976533 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:38:00.976533 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 31 09:38:00.976533 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 31 09:38:00.976533 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 31 09:38:00.976533 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 31 09:38:00.977898 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:38:00.977898 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:38:00.977898 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 31 09:38:00.977898 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 09:38:00.977898 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 31 09:38:00.977898 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 09:38:00.977898 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:38:00.977898 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:38:00.977898 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:38:00.977898 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:38:00.977898 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:38:00.977898 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application with Lock( Aug 31 09:38:00.977898 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:38:00.977898 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:38:00.977898 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:38:00.977898 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:38:00.977898 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:38:00.977898 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:38:00.979282 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:38:00.979282 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:38:00.979282 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 31 09:38:00.979282 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 31 09:38:00.979282 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 31 09:38:00.979282 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 31 09:38:00.979282 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 31 09:38:00.979282 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 31 09:38:00.979282 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 31 09:38:00.979282 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application Aug 31 09:38:00.979282 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 636/1267] 10.222.0.13 () {56 vars in 1108 bytes} [Wed Aug 31 09:38:00 2022] GET /identity/v3/users/placement => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:38:00.983923 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab26f453-57a4-4d23-b49e-ceeb73503b7d None admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:38:01.093838 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab26f453-57a4-4d23-b49e-ceeb73503b7d None admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:38:01.130024 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-775dd573-016d-4837-8ec8-04fe8c3ca390 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:38:01.130637 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775dd573-016d-4837-8ec8-04fe8c3ca390 None admin] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:38:01.130918 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775dd573-016d-4837-8ec8-04fe8c3ca390 None admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:38:01.131191 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775dd573-016d-4837-8ec8-04fe8c3ca390 None admin] PATH_INFO: `/v3/users` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:38:01.132460 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-775dd573-016d-4837-8ec8-04fe8c3ca390 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=88114) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:38:01.132792 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-775dd573-016d-4837-8ec8-04fe8c3ca390 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:38:01.134279 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-775dd573-016d-4837-8ec8-04fe8c3ca390 None admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:38:01.144025 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=100 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:38:01.153505 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 632/1268] 10.222.0.13 () {56 vars in 1143 bytes} [Wed Aug 31 09:38:00 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:38:01.159289 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d804156b-4a06-4fe2-8637-ae0601e67852 None admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:38:01.261924 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d804156b-4a06-4fe2-8637-ae0601e67852 None admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:38:01.296471 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-335c3f70-b9ea-4713-bbf9-632886d5c6a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:38:01.297097 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335c3f70-b9ea-4713-bbf9-632886d5c6a2 None admin] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:38:01.297370 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335c3f70-b9ea-4713-bbf9-632886d5c6a2 None admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:38:01.297669 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335c3f70-b9ea-4713-bbf9-632886d5c6a2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:38:01.308877 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-335c3f70-b9ea-4713-bbf9-632886d5c6a2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:38:01.310556 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-335c3f70-b9ea-4713-bbf9-632886d5c6a2 None admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:38:01.326270 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application [None req-335c3f70-b9ea-4713-bbf9-632886d5c6a2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 09:38:01.326270 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:38:01.326270 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:38:01.326270 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:38:01.326270 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:38:01.326270 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:38:01.326270 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:38:01.326270 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:38:01.326270 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:38:01.326270 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:38:01.326270 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:38:01.326270 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:38:01.326270 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 09:38:01.326270 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 09:38:01.326270 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 09:38:01.326270 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 09:38:01.327422 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:38:01.327422 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:38:01.327422 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:38:01.327422 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:38:01.327422 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:38:01.327422 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:38:01.327422 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:38:01.327422 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application with Lock( Aug 31 09:38:01.327422 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:38:01.327422 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:38:01.327422 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:38:01.327422 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:38:01.327422 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:38:01.327422 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:38:01.327422 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:38:01.327422 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:38:01.327422 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 09:38:01.328337 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 09:38:01.328337 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 09:38:01.328337 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application Aug 31 09:38:01.328337 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 637/1269] 10.222.0.13 () {56 vars in 1108 bytes} [Wed Aug 31 09:38:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:38:01.333266 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-775dd573-016d-4837-8ec8-04fe8c3ca390 None admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:38:01.441250 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-775dd573-016d-4837-8ec8-04fe8c3ca390 None admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:38:01.476967 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b16d61-7ada-4f0f-b628-037bbe9c4e93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:38:01.477599 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b16d61-7ada-4f0f-b628-037bbe9c4e93 None admin] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:38:01.477873 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b16d61-7ada-4f0f-b628-037bbe9c4e93 None admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:38:01.478139 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b16d61-7ada-4f0f-b628-037bbe9c4e93 None admin] PATH_INFO: `/v3/domains` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:38:01.478783 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0b16d61-7ada-4f0f-b628-037bbe9c4e93 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88114) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:38:01.479076 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0b16d61-7ada-4f0f-b628-037bbe9c4e93 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:38:01.480433 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0b16d61-7ada-4f0f-b628-037bbe9c4e93 None admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:38:01.492248 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 633/1270] 10.222.0.13 () {56 vars in 1109 bytes} [Wed Aug 31 09:38:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:38:01.498037 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-335c3f70-b9ea-4713-bbf9-632886d5c6a2 None admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:38:01.529737 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=100 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:38:01.596980 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-335c3f70-b9ea-4713-bbf9-632886d5c6a2 None admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:38:01.630039 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e251e200-53b2-48a9-b505-c62e4f9bfc2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:38:01.630665 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e251e200-53b2-48a9-b505-c62e4f9bfc2b None admin] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:38:01.630938 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e251e200-53b2-48a9-b505-c62e4f9bfc2b None admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:38:01.631203 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e251e200-53b2-48a9-b505-c62e4f9bfc2b None admin] PATH_INFO: `/v3/projects/service` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:38:01.637725 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e251e200-53b2-48a9-b505-c62e4f9bfc2b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:38:01.639173 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e251e200-53b2-48a9-b505-c62e4f9bfc2b None admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:38:01.651048 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application [None req-e251e200-53b2-48a9-b505-c62e4f9bfc2b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 09:38:01.651048 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:38:01.651048 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:38:01.651048 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:38:01.651048 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:38:01.651048 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:38:01.651048 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:38:01.651048 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:38:01.651048 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:38:01.651048 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:38:01.651048 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:38:01.651048 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:38:01.651048 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 31 09:38:01.651048 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 31 09:38:01.651048 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 31 09:38:01.652118 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 31 09:38:01.652118 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:38:01.652118 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:38:01.652118 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:38:01.652118 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:38:01.652118 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:38:01.652118 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:38:01.652118 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:38:01.652118 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application with Lock( Aug 31 09:38:01.652118 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:38:01.652118 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:38:01.652118 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:38:01.652118 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:38:01.652118 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:38:01.652118 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:38:01.652118 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:38:01.652118 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:38:01.653038 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 31 09:38:01.653038 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 31 09:38:01.653038 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 31 09:38:01.653038 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 31 09:38:01.653038 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 31 09:38:01.653038 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 31 09:38:01.653038 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 09:38:01.653038 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application Aug 31 09:38:01.653038 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 638/1271] 10.222.0.13 () {56 vars in 1111 bytes} [Wed Aug 31 09:38:01 2022] GET /identity/v3/projects/service => generated 88 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:38:01.658013 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b16d61-7ada-4f0f-b628-037bbe9c4e93 None admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:38:01.716284 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=100 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:38:01.758765 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b16d61-7ada-4f0f-b628-037bbe9c4e93 None admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:38:01.792861 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-772e3a72-1469-40ba-bfa1-5bceb98f4bc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:38:01.793534 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772e3a72-1469-40ba-bfa1-5bceb98f4bc6 None admin] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:38:01.793708 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772e3a72-1469-40ba-bfa1-5bceb98f4bc6 None admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:38:01.793976 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772e3a72-1469-40ba-bfa1-5bceb98f4bc6 None admin] PATH_INFO: `/v3/projects` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:38:01.794662 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-772e3a72-1469-40ba-bfa1-5bceb98f4bc6 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88114) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:38:01.794959 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-772e3a72-1469-40ba-bfa1-5bceb98f4bc6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:38:01.796364 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-772e3a72-1469-40ba-bfa1-5bceb98f4bc6 None admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:38:01.808427 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 634/1272] 10.222.0.13 () {56 vars in 1148 bytes} [Wed Aug 31 09:38:01 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:38:01.814452 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e251e200-53b2-48a9-b505-c62e4f9bfc2b None admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:38:01.911416 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=100 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:38:01.922182 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e251e200-53b2-48a9-b505-c62e4f9bfc2b None admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:38:01.956719 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2756c7ac-810e-464c-b759-494d94cdcd3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:38:01.957373 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2756c7ac-810e-464c-b759-494d94cdcd3d None admin] REQUEST_METHOD: `PUT` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:38:01.957672 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2756c7ac-810e-464c-b759-494d94cdcd3d None admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:38:01.957942 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2756c7ac-810e-464c-b759-494d94cdcd3d None admin] PATH_INFO: `/v3/projects/49548eb44d62415582af820ab66decee/users/1ff3aff5ef444910be6dbb989d0e3194/roles/5d4ee63a3add47fc9a6b8b16d39e0a1f` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:38:01.960523 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2756c7ac-810e-464c-b759-494d94cdcd3d None admin] RBAC: Authorizing `identity:create_grant(project_id=49548eb44d62415582af820ab66decee, user_id=1ff3aff5ef444910be6dbb989d0e3194, role_id=5d4ee63a3add47fc9a6b8b16d39e0a1f)` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:38:01.962011 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2756c7ac-810e-464c-b759-494d94cdcd3d None admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:38:01.975137 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 639/1273] 10.222.0.13 () {58 vars in 1439 bytes} [Wed Aug 31 09:38:01 2022] PUT /identity/v3/projects/49548eb44d62415582af820ab66decee/users/1ff3aff5ef444910be6dbb989d0e3194/roles/5d4ee63a3add47fc9a6b8b16d39e0a1f => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:38:03.884089 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b276e5d-786a-46a0-add4-3f788b912886 None None] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:38:03.884665 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b276e5d-786a-46a0-add4-3f788b912886 None None] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:38:03.884758 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b276e5d-786a-46a0-add4-3f788b912886 None None] PATH_INFO: `/` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:38:03.886195 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 635/1274] 10.222.0.13 () {54 vars in 927 bytes} [Wed Aug 31 09:38:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:38:03.892459 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a98f72f-869a-43c5-8383-f570502a98b4 None None] REQUEST_METHOD: `POST` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:38:03.892895 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a98f72f-869a-43c5-8383-f570502a98b4 None None] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:38:03.892999 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a98f72f-869a-43c5-8383-f570502a98b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:38:03.914223 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.auth.core [None req-6a98f72f-869a-43c5-8383-f570502a98b4 None None] MFA Rules not processed for user `fd836cf590a145df9acf19cf3ae2ac56`. Rule list: `[]` (Enabled: `True`). {{(pid=88113) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:38:03.915607 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.fernet_utils [None req-6a98f72f-869a-43c5-8383-f570502a98b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:38:03.984328 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 640/1275] 10.222.0.13 () {58 vars in 1024 bytes} [Wed Aug 31 09:38:03 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 31 09:38:04.257713 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b276e5d-786a-46a0-add4-3f788b912886 None None] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:38:04.359831 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b276e5d-786a-46a0-add4-3f788b912886 None None] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:38:04.377167 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=100 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:38:04.395060 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b54ef4-0c7d-442b-ac7a-1f043ec2dd7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:38:04.395688 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b54ef4-0c7d-442b-ac7a-1f043ec2dd7c None admin] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:38:04.395959 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b54ef4-0c7d-442b-ac7a-1f043ec2dd7c None admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:38:04.396222 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b54ef4-0c7d-442b-ac7a-1f043ec2dd7c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:38:04.406829 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6b54ef4-0c7d-442b-ac7a-1f043ec2dd7c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 09:38:04.411640 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application [None req-c6b54ef4-0c7d-442b-ac7a-1f043ec2dd7c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 09:38:04.411640 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:38:04.411640 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:38:04.411640 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:38:04.411640 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:38:04.411640 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:38:04.411640 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:38:04.411640 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:38:04.411640 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:38:04.411640 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:38:04.411640 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:38:04.411640 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:38:04.411640 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 09:38:04.411640 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 09:38:04.411640 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 09:38:04.411640 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 09:38:04.412746 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 09:38:04.412746 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 09:38:04.412746 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 09:38:04.412746 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 09:38:04.412746 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:38:04.412746 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:38:04.412746 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:38:04.412746 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:38:04.412746 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:38:04.412746 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:38:04.412746 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:38:04.412746 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application with Lock( Aug 31 09:38:04.412746 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:38:04.412746 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:38:04.412746 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:38:04.412746 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:38:04.413830 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:38:04.413830 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:38:04.413830 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:38:04.413830 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:38:04.413830 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 09:38:04.413830 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 09:38:04.413830 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 09:38:04.413830 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 09:38:04.413830 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 09:38:04.413830 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 09:38:04.413830 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 09:38:04.413830 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application Aug 31 09:38:04.413830 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 636/1276] 10.222.0.13 () {56 vars in 1096 bytes} [Wed Aug 31 09:38:04 2022] GET /identity/v3/roles/admin => generated 83 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:38:04.419081 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a98f72f-869a-43c5-8383-f570502a98b4 None None] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:38:04.499317 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:38:04.518517 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a98f72f-869a-43c5-8383-f570502a98b4 None None] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:38:04.552273 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94962a0d-ad9a-4804-83d4-7f3d88a32fd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:38:04.552878 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94962a0d-ad9a-4804-83d4-7f3d88a32fd9 None admin] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:38:04.553152 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94962a0d-ad9a-4804-83d4-7f3d88a32fd9 None admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:38:04.553487 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94962a0d-ad9a-4804-83d4-7f3d88a32fd9 None admin] PATH_INFO: `/v3/roles` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:38:04.554255 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94962a0d-ad9a-4804-83d4-7f3d88a32fd9 None admin] RBAC: Adding query filter params (name=admin) {{(pid=88113) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:38:04.554556 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94962a0d-ad9a-4804-83d4-7f3d88a32fd9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:38:04.555931 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94962a0d-ad9a-4804-83d4-7f3d88a32fd9 None admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:38:04.563882 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 641/1277] 10.222.0.13 () {56 vars in 1099 bytes} [Wed Aug 31 09:38:04 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:38:04.569801 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b54ef4-0c7d-442b-ac7a-1f043ec2dd7c None admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:38:04.667805 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b54ef4-0c7d-442b-ac7a-1f043ec2dd7c None admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:38:04.701051 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b5892b-188c-4ba5-b4e3-234c92a781ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:38:04.701696 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b5892b-188c-4ba5-b4e3-234c92a781ae None admin] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:38:04.701969 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b5892b-188c-4ba5-b4e3-234c92a781ae None admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:38:04.702233 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b5892b-188c-4ba5-b4e3-234c92a781ae None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:38:04.712547 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5b5892b-188c-4ba5-b4e3-234c92a781ae None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:38:04.714010 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5b5892b-188c-4ba5-b4e3-234c92a781ae None admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:38:04.728688 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application [None req-f5b5892b-188c-4ba5-b4e3-234c92a781ae None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 09:38:04.728688 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:38:04.728688 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:38:04.728688 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:38:04.728688 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:38:04.728688 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:38:04.728688 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:38:04.728688 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:38:04.728688 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:38:04.728688 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:38:04.728688 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:38:04.728688 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:38:04.728688 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 09:38:04.728688 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 09:38:04.728688 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 09:38:04.728688 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 09:38:04.729885 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:38:04.729885 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:38:04.729885 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:38:04.729885 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:38:04.729885 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:38:04.729885 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:38:04.729885 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:38:04.729885 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application with Lock( Aug 31 09:38:04.729885 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:38:04.729885 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:38:04.729885 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:38:04.729885 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:38:04.729885 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:38:04.729885 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:38:04.729885 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:38:04.729885 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:38:04.729885 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 09:38:04.731164 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 09:38:04.731164 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 09:38:04.731164 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application Aug 31 09:38:04.731164 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 637/1278] 10.222.0.13 () {56 vars in 1108 bytes} [Wed Aug 31 09:38:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:38:04.735796 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94962a0d-ad9a-4804-83d4-7f3d88a32fd9 None admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:38:04.843554 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94962a0d-ad9a-4804-83d4-7f3d88a32fd9 None admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:38:04.877169 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-694a76c3-851a-4d1b-9237-1334178ef799 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:38:04.877883 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694a76c3-851a-4d1b-9237-1334178ef799 None admin] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:38:04.878161 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694a76c3-851a-4d1b-9237-1334178ef799 None admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:38:04.878493 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694a76c3-851a-4d1b-9237-1334178ef799 None admin] PATH_INFO: `/v3/domains` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:38:04.879317 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-694a76c3-851a-4d1b-9237-1334178ef799 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88113) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:38:04.879624 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-694a76c3-851a-4d1b-9237-1334178ef799 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:38:04.881221 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-694a76c3-851a-4d1b-9237-1334178ef799 None admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:38:04.895256 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 642/1279] 10.222.0.13 () {56 vars in 1109 bytes} [Wed Aug 31 09:38:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:38:04.901763 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b5892b-188c-4ba5-b4e3-234c92a781ae None admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:38:04.936993 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=100 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:38:05.007530 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b5892b-188c-4ba5-b4e3-234c92a781ae None admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:38:05.040792 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f53a347-cc9e-4ddb-8300-cee75fd20e5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:38:05.041426 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f53a347-cc9e-4ddb-8300-cee75fd20e5d None admin] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:38:05.041703 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f53a347-cc9e-4ddb-8300-cee75fd20e5d None admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:38:05.041971 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f53a347-cc9e-4ddb-8300-cee75fd20e5d None admin] PATH_INFO: `/v3/users/placement` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:38:05.049082 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f53a347-cc9e-4ddb-8300-cee75fd20e5d None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:38:05.050611 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f53a347-cc9e-4ddb-8300-cee75fd20e5d None admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:38:05.061569 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application [None req-5f53a347-cc9e-4ddb-8300-cee75fd20e5d None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 31 09:38:05.061569 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:38:05.061569 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:38:05.061569 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:38:05.061569 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:38:05.061569 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:38:05.061569 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:38:05.061569 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:38:05.061569 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:38:05.061569 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:38:05.061569 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:38:05.061569 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:38:05.061569 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 31 09:38:05.061569 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 31 09:38:05.061569 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 31 09:38:05.061569 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 31 09:38:05.062806 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:38:05.062806 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:38:05.062806 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 31 09:38:05.062806 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 09:38:05.062806 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 31 09:38:05.062806 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 09:38:05.062806 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:38:05.062806 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:38:05.062806 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:38:05.062806 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:38:05.062806 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:38:05.062806 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application with Lock( Aug 31 09:38:05.062806 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:38:05.062806 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:38:05.062806 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:38:05.062806 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:38:05.062806 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:38:05.062806 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:38:05.063939 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:38:05.063939 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:38:05.063939 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 31 09:38:05.063939 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 31 09:38:05.063939 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 31 09:38:05.063939 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 31 09:38:05.063939 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 31 09:38:05.063939 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 31 09:38:05.063939 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 31 09:38:05.063939 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application Aug 31 09:38:05.063939 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 638/1280] 10.222.0.13 () {56 vars in 1108 bytes} [Wed Aug 31 09:38:04 2022] GET /identity/v3/users/placement => generated 87 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:38:05.068525 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-694a76c3-851a-4d1b-9237-1334178ef799 None admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:38:05.073035 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=100 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:38:05.167673 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-694a76c3-851a-4d1b-9237-1334178ef799 None admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:38:05.201105 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a2213d7-3c32-4972-8551-af70aed39bee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:38:05.201726 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2213d7-3c32-4972-8551-af70aed39bee None admin] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:38:05.202002 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2213d7-3c32-4972-8551-af70aed39bee None admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:38:05.202269 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2213d7-3c32-4972-8551-af70aed39bee None admin] PATH_INFO: `/v3/users` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:38:05.202982 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a2213d7-3c32-4972-8551-af70aed39bee None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=88113) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:38:05.203281 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a2213d7-3c32-4972-8551-af70aed39bee None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:38:05.204699 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a2213d7-3c32-4972-8551-af70aed39bee None admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:38:05.221373 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 643/1281] 10.222.0.13 () {56 vars in 1143 bytes} [Wed Aug 31 09:38:05 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:38:05.227418 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f53a347-cc9e-4ddb-8300-cee75fd20e5d None admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:38:05.317401 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=100 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:38:05.327265 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f53a347-cc9e-4ddb-8300-cee75fd20e5d None admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:38:05.362048 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad91905-6a48-401e-81a6-39a8162ef568 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:38:05.362656 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad91905-6a48-401e-81a6-39a8162ef568 None admin] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:38:05.362948 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad91905-6a48-401e-81a6-39a8162ef568 None admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:38:05.363199 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad91905-6a48-401e-81a6-39a8162ef568 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:38:05.429873 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ad91905-6a48-401e-81a6-39a8162ef568 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:38:05.431226 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ad91905-6a48-401e-81a6-39a8162ef568 None admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:38:05.526699 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=100 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:38:05.537490 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 639/1282] 10.222.0.13 () {58 vars in 1282 bytes} [Wed Aug 31 09:38:05 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 31 09:38:05.544040 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a2213d7-3c32-4972-8551-af70aed39bee None admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:38:05.608537 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=100 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:38:05.649009 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a2213d7-3c32-4972-8551-af70aed39bee None admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:38:05.681186 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-154d4991-425b-4ef3-b986-2d795a1c1e32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:38:05.681809 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154d4991-425b-4ef3-b986-2d795a1c1e32 None admin] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:38:05.682083 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154d4991-425b-4ef3-b986-2d795a1c1e32 None admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:38:05.682349 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154d4991-425b-4ef3-b986-2d795a1c1e32 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:38:05.692269 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-154d4991-425b-4ef3-b986-2d795a1c1e32 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:38:05.693720 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-154d4991-425b-4ef3-b986-2d795a1c1e32 None admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:38:05.708107 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application [None req-154d4991-425b-4ef3-b986-2d795a1c1e32 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 09:38:05.708107 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:38:05.708107 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:38:05.708107 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:38:05.708107 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:38:05.708107 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:38:05.708107 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:38:05.708107 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:38:05.708107 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:38:05.708107 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:38:05.708107 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:38:05.708107 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:38:05.708107 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 09:38:05.708107 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 09:38:05.708107 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 09:38:05.708107 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 09:38:05.709262 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:38:05.709262 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:38:05.709262 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:38:05.709262 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:38:05.709262 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:38:05.709262 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:38:05.709262 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:38:05.709262 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application with Lock( Aug 31 09:38:05.709262 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:38:05.709262 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:38:05.709262 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:38:05.709262 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:38:05.709262 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:38:05.709262 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:38:05.709262 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:38:05.709262 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:38:05.709262 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 09:38:05.711685 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 09:38:05.711685 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 09:38:05.711685 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application Aug 31 09:38:05.711685 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 644/1283] 10.222.0.13 () {56 vars in 1108 bytes} [Wed Aug 31 09:38:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:38:05.715901 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad91905-6a48-401e-81a6-39a8162ef568 None admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:38:05.816157 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad91905-6a48-401e-81a6-39a8162ef568 None admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:38:05.851262 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c56d538-e4b3-43ae-92d3-ba306efea0dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:38:05.852022 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c56d538-e4b3-43ae-92d3-ba306efea0dc None admin] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:38:05.852325 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c56d538-e4b3-43ae-92d3-ba306efea0dc None admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:38:05.852566 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c56d538-e4b3-43ae-92d3-ba306efea0dc None admin] PATH_INFO: `/v3/domains` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:38:05.853210 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c56d538-e4b3-43ae-92d3-ba306efea0dc None admin] RBAC: Adding query filter params (name=Default) {{(pid=88114) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:38:05.853547 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c56d538-e4b3-43ae-92d3-ba306efea0dc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:38:05.854916 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c56d538-e4b3-43ae-92d3-ba306efea0dc None admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:38:05.866997 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 640/1284] 10.222.0.13 () {56 vars in 1109 bytes} [Wed Aug 31 09:38:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:38:05.872640 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-154d4991-425b-4ef3-b986-2d795a1c1e32 None admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:38:05.979820 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-154d4991-425b-4ef3-b986-2d795a1c1e32 None admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:38:05.996039 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=100 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:38:06.018295 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f828614e-d745-437c-b00f-3cd78211f28c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:38:06.018938 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f828614e-d745-437c-b00f-3cd78211f28c None admin] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:38:06.019213 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f828614e-d745-437c-b00f-3cd78211f28c None admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:38:06.019480 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f828614e-d745-437c-b00f-3cd78211f28c None admin] PATH_INFO: `/v3/projects/service` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:38:06.027297 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f828614e-d745-437c-b00f-3cd78211f28c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:38:06.028764 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f828614e-d745-437c-b00f-3cd78211f28c None admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:38:06.041026 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application [None req-f828614e-d745-437c-b00f-3cd78211f28c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 09:38:06.041026 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:38:06.041026 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:38:06.041026 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:38:06.041026 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:38:06.041026 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:38:06.041026 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:38:06.041026 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:38:06.041026 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:38:06.041026 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:38:06.041026 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:38:06.041026 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:38:06.041026 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 31 09:38:06.041026 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 31 09:38:06.041026 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 31 09:38:06.042138 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 31 09:38:06.042138 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:38:06.042138 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:38:06.042138 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:38:06.042138 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:38:06.042138 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:38:06.042138 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:38:06.042138 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:38:06.042138 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application with Lock( Aug 31 09:38:06.042138 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:38:06.042138 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:38:06.042138 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:38:06.042138 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:38:06.042138 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:38:06.042138 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:38:06.042138 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:38:06.042138 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:38:06.044198 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 31 09:38:06.044198 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 31 09:38:06.044198 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 31 09:38:06.044198 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 31 09:38:06.044198 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 31 09:38:06.044198 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 31 09:38:06.044198 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 09:38:06.044198 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: ERROR keystone.server.flask.application Aug 31 09:38:06.044198 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 645/1285] 10.222.0.13 () {56 vars in 1111 bytes} [Wed Aug 31 09:38:05 2022] GET /identity/v3/projects/service => generated 88 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:38:06.048404 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c56d538-e4b3-43ae-92d3-ba306efea0dc None admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:38:06.103160 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=100 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:38:06.155799 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c56d538-e4b3-43ae-92d3-ba306efea0dc None admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:38:06.190810 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc3e77a-1762-46f8-b5b6-311c27485204 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:38:06.191422 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc3e77a-1762-46f8-b5b6-311c27485204 None admin] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:38:06.191699 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc3e77a-1762-46f8-b5b6-311c27485204 None admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:38:06.191968 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc3e77a-1762-46f8-b5b6-311c27485204 None admin] PATH_INFO: `/v3/projects` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:38:06.192663 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bc3e77a-1762-46f8-b5b6-311c27485204 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88114) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:38:06.192965 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bc3e77a-1762-46f8-b5b6-311c27485204 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:38:06.194516 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bc3e77a-1762-46f8-b5b6-311c27485204 None admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:38:06.207718 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 641/1286] 10.222.0.13 () {56 vars in 1148 bytes} [Wed Aug 31 09:38:06 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:38:06.214930 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f828614e-d745-437c-b00f-3cd78211f28c None admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:38:06.323526 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f828614e-d745-437c-b00f-3cd78211f28c None admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:38:06.363378 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e76e9044-7618-4859-a2bb-ccfa9f98ba7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:38:06.364091 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76e9044-7618-4859-a2bb-ccfa9f98ba7b None admin] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:38:06.364414 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76e9044-7618-4859-a2bb-ccfa9f98ba7b None admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:38:06.364732 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76e9044-7618-4859-a2bb-ccfa9f98ba7b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:38:06.365600 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e76e9044-7618-4859-a2bb-ccfa9f98ba7b None admin] RBAC: Adding query filter params (role.id=5d4ee63a3add47fc9a6b8b16d39e0a1f, scope.project.id=49548eb44d62415582af820ab66decee, user.id=1ff3aff5ef444910be6dbb989d0e3194) {{(pid=88113) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:38:06.366071 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e76e9044-7618-4859-a2bb-ccfa9f98ba7b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:38:06.367650 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e76e9044-7618-4859-a2bb-ccfa9f98ba7b None admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:38:06.374611 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 646/1287] 10.222.0.13 () {56 vars in 1374 bytes} [Wed Aug 31 09:38:06 2022] GET /identity/v3/role_assignments?user.id=1ff3aff5ef444910be6dbb989d0e3194&scope.project.id=49548eb44d62415582af820ab66decee&role.id=5d4ee63a3add47fc9a6b8b16d39e0a1f => generated 608 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:38:08.335208 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e91d9c-dbbc-44dd-bacb-0e1e867e38ca None None] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:38:08.335712 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e91d9c-dbbc-44dd-bacb-0e1e867e38ca None None] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:38:08.335712 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e91d9c-dbbc-44dd-bacb-0e1e867e38ca None None] PATH_INFO: `/` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:38:08.336983 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 642/1288] 10.222.0.13 () {54 vars in 927 bytes} [Wed Aug 31 09:38:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:38:08.342236 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3160063-008b-42a3-a260-966cf1d7e373 None None] REQUEST_METHOD: `POST` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:38:08.342644 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3160063-008b-42a3-a260-966cf1d7e373 None None] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:38:08.342783 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3160063-008b-42a3-a260-966cf1d7e373 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:38:08.360820 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.auth.core [None req-f3160063-008b-42a3-a260-966cf1d7e373 None None] MFA Rules not processed for user `fd836cf590a145df9acf19cf3ae2ac56`. Rule list: `[]` (Enabled: `True`). {{(pid=88113) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:38:08.362038 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.fernet_utils [None req-f3160063-008b-42a3-a260-966cf1d7e373 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:38:08.392083 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=100 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:38:08.430419 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 647/1289] 10.222.0.13 () {58 vars in 1024 bytes} [Wed Aug 31 09:38:08 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 09:38:08.695907 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e91d9c-dbbc-44dd-bacb-0e1e867e38ca None None] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:38:08.797245 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e91d9c-dbbc-44dd-bacb-0e1e867e38ca None None] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:38:08.808288 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=100 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:38:08.832567 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5bbfbb7-e9d4-4807-b76b-417dfc09153c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:38:08.833203 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5bbfbb7-e9d4-4807-b76b-417dfc09153c None admin] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:38:08.833510 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5bbfbb7-e9d4-4807-b76b-417dfc09153c None admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:38:08.833782 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5bbfbb7-e9d4-4807-b76b-417dfc09153c None admin] PATH_INFO: `/v3/services/placement` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:38:08.834368 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5bbfbb7-e9d4-4807-b76b-417dfc09153c None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:38:08.836617 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5bbfbb7-e9d4-4807-b76b-417dfc09153c None admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:38:08.845671 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application [None req-e5bbfbb7-e9d4-4807-b76b-417dfc09153c None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Aug 31 09:38:08.845671 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:38:08.845671 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 31 09:38:08.845671 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 31 09:38:08.845671 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 31 09:38:08.845671 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 31 09:38:08.845671 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 31 09:38:08.845671 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 09:38:08.845671 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 31 09:38:08.845671 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application Aug 31 09:38:08.845671 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 31 09:38:08.845671 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application Aug 31 09:38:08.845671 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:38:08.845671 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:38:08.845671 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:38:08.845671 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:38:08.845671 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:38:08.846930 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:38:08.846930 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:38:08.846930 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:38:08.846930 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:38:08.846930 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:38:08.846930 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:38:08.846930 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 31 09:38:08.846930 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 31 09:38:08.846930 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 31 09:38:08.846930 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 31 09:38:08.846930 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:38:08.846930 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:38:08.846930 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:38:08.846930 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:38:08.846930 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:38:08.846930 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:38:08.848136 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:38:08.848136 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application with Lock( Aug 31 09:38:08.848136 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:38:08.848136 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:38:08.848136 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:38:08.848136 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:38:08.848136 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:38:08.848136 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:38:08.848136 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:38:08.848136 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:38:08.848136 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 31 09:38:08.848136 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 09:38:08.848136 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 31 09:38:08.848136 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application Aug 31 09:38:08.848136 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 643/1290] 10.222.0.13 () {56 vars in 1117 bytes} [Wed Aug 31 09:38:08 2022] GET /identity/v3/services/placement => generated 90 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:38:08.852780 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3160063-008b-42a3-a260-966cf1d7e373 None None] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:38:08.957762 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3160063-008b-42a3-a260-966cf1d7e373 None None] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:38:08.991228 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1641e3b0-2798-43ae-bfe4-5a4a1c3243af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:38:08.991848 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1641e3b0-2798-43ae-bfe4-5a4a1c3243af None admin] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:38:08.992118 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1641e3b0-2798-43ae-bfe4-5a4a1c3243af None admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:38:08.992385 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1641e3b0-2798-43ae-bfe4-5a4a1c3243af None admin] PATH_INFO: `/v3/services` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:38:08.993004 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1641e3b0-2798-43ae-bfe4-5a4a1c3243af None admin] RBAC: Adding query filter params (name=placement) {{(pid=88113) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:38:08.993304 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1641e3b0-2798-43ae-bfe4-5a4a1c3243af None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:38:08.994691 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1641e3b0-2798-43ae-bfe4-5a4a1c3243af None admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:38:08.999365 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 648/1291] 10.222.0.13 () {56 vars in 1116 bytes} [Wed Aug 31 09:38:08 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:38:09.005057 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5bbfbb7-e9d4-4807-b76b-417dfc09153c None admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:38:09.105854 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5bbfbb7-e9d4-4807-b76b-417dfc09153c None admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:38:09.146347 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f7984be-1463-4e29-bc6b-5ea9f0d606df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:38:09.146987 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7984be-1463-4e29-bc6b-5ea9f0d606df None admin] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:38:09.147782 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7984be-1463-4e29-bc6b-5ea9f0d606df None admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:38:09.148299 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7984be-1463-4e29-bc6b-5ea9f0d606df None admin] PATH_INFO: `/v3/services` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:38:09.149107 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f7984be-1463-4e29-bc6b-5ea9f0d606df None admin] RBAC: Adding query filter params (type=placement) {{(pid=88114) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:38:09.149437 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f7984be-1463-4e29-bc6b-5ea9f0d606df None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:38:09.150850 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f7984be-1463-4e29-bc6b-5ea9f0d606df None admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:38:09.156101 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 644/1292] 10.222.0.13 () {56 vars in 1116 bytes} [Wed Aug 31 09:38:09 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:38:11.097467 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf55227-20bd-4363-b1c0-4c3013246082 None None] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:38:11.097990 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf55227-20bd-4363-b1c0-4c3013246082 None None] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:38:11.097990 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf55227-20bd-4363-b1c0-4c3013246082 None None] PATH_INFO: `/` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:38:11.099344 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 649/1293] 10.222.0.13 () {54 vars in 927 bytes} [Wed Aug 31 09:38:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:38:11.104508 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d17cae-674d-47e4-9d0d-342c1ea37167 None None] REQUEST_METHOD: `POST` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:38:11.104908 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d17cae-674d-47e4-9d0d-342c1ea37167 None None] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:38:11.105021 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d17cae-674d-47e4-9d0d-342c1ea37167 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:38:11.122747 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.auth.core [None req-40d17cae-674d-47e4-9d0d-342c1ea37167 None None] MFA Rules not processed for user `fd836cf590a145df9acf19cf3ae2ac56`. Rule list: `[]` (Enabled: `True`). {{(pid=88114) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:38:11.123904 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.fernet_utils [None req-40d17cae-674d-47e4-9d0d-342c1ea37167 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88114) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:38:11.153517 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=100 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:38:11.190122 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 645/1294] 10.222.0.13 () {58 vars in 1024 bytes} [Wed Aug 31 09:38:11 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 09:38:11.451886 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf55227-20bd-4363-b1c0-4c3013246082 None None] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:38:11.483829 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=100 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:38:11.554187 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf55227-20bd-4363-b1c0-4c3013246082 None None] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:38:11.588129 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab4080e-2ed7-49bb-af37-cb78b0579131 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:38:11.588723 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab4080e-2ed7-49bb-af37-cb78b0579131 None admin] REQUEST_METHOD: `POST` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:38:11.588998 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab4080e-2ed7-49bb-af37-cb78b0579131 None admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:38:11.589270 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab4080e-2ed7-49bb-af37-cb78b0579131 None admin] PATH_INFO: `/v3/services` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:38:11.589946 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ab4080e-2ed7-49bb-af37-cb78b0579131 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:38:11.591318 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ab4080e-2ed7-49bb-af37-cb78b0579131 None admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:38:11.605371 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 650/1295] 10.222.0.13 () {60 vars in 1141 bytes} [Wed Aug 31 09:38:11 2022] POST /identity/v3/services => generated 246 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 09:38:13.555996 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11785841-f271-44c2-b578-926d9905ea01 None None] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:38:13.556595 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11785841-f271-44c2-b578-926d9905ea01 None None] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:38:13.556595 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11785841-f271-44c2-b578-926d9905ea01 None None] PATH_INFO: `/` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:38:13.557925 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 646/1296] 10.222.0.13 () {54 vars in 927 bytes} [Wed Aug 31 09:38:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:38:13.563475 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ba7d53-c0d5-4b5b-a698-1814467b4cc7 None None] REQUEST_METHOD: `POST` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:38:13.563931 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ba7d53-c0d5-4b5b-a698-1814467b4cc7 None None] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:38:13.564021 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ba7d53-c0d5-4b5b-a698-1814467b4cc7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:38:13.586396 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.auth.core [None req-82ba7d53-c0d5-4b5b-a698-1814467b4cc7 None None] MFA Rules not processed for user `fd836cf590a145df9acf19cf3ae2ac56`. Rule list: `[]` (Enabled: `True`). {{(pid=88113) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:38:13.587616 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.fernet_utils [None req-82ba7d53-c0d5-4b5b-a698-1814467b4cc7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:38:13.666823 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 651/1297] 10.222.0.13 () {58 vars in 1024 bytes} [Wed Aug 31 09:38:13 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 09:38:13.946497 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11785841-f271-44c2-b578-926d9905ea01 None None] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:38:14.056311 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11785841-f271-44c2-b578-926d9905ea01 None None] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:38:14.096257 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b2ea04e-4b96-4372-99b3-de18ee41f9aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:38:14.096903 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2ea04e-4b96-4372-99b3-de18ee41f9aa None admin] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:38:14.097179 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2ea04e-4b96-4372-99b3-de18ee41f9aa None admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:38:14.097476 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2ea04e-4b96-4372-99b3-de18ee41f9aa None admin] PATH_INFO: `/v3/services/placement` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:38:14.098069 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b2ea04e-4b96-4372-99b3-de18ee41f9aa None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:38:14.101839 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b2ea04e-4b96-4372-99b3-de18ee41f9aa None admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:38:14.117063 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application [None req-2b2ea04e-4b96-4372-99b3-de18ee41f9aa None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Aug 31 09:38:14.117063 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:38:14.117063 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 31 09:38:14.117063 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 31 09:38:14.117063 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 31 09:38:14.117063 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 31 09:38:14.117063 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 31 09:38:14.117063 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 09:38:14.117063 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 31 09:38:14.117063 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application Aug 31 09:38:14.117063 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 31 09:38:14.117063 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application Aug 31 09:38:14.117063 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:38:14.117063 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:38:14.117063 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:38:14.117063 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:38:14.117063 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:38:14.119029 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:38:14.119029 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:38:14.119029 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:38:14.119029 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:38:14.119029 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:38:14.119029 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:38:14.119029 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 31 09:38:14.119029 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 31 09:38:14.119029 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 31 09:38:14.119029 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 31 09:38:14.119029 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:38:14.119029 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:38:14.119029 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:38:14.119029 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:38:14.119029 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:38:14.119029 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:38:14.120741 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:38:14.120741 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application with Lock( Aug 31 09:38:14.120741 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:38:14.120741 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:38:14.120741 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:38:14.120741 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:38:14.120741 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:38:14.120741 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:38:14.120741 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:38:14.120741 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:38:14.120741 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 31 09:38:14.120741 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 09:38:14.120741 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 31 09:38:14.120741 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application Aug 31 09:38:14.120741 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 647/1298] 10.222.0.13 () {56 vars in 1117 bytes} [Wed Aug 31 09:38:13 2022] GET /identity/v3/services/placement => generated 90 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:38:14.126182 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ba7d53-c0d5-4b5b-a698-1814467b4cc7 None None] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:38:14.148470 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:38:14.238882 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ba7d53-c0d5-4b5b-a698-1814467b4cc7 None None] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:38:14.276310 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38794216-3b34-49e7-861a-763533b55a07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:38:14.276922 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38794216-3b34-49e7-861a-763533b55a07 None admin] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:38:14.277196 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38794216-3b34-49e7-861a-763533b55a07 None admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:38:14.277489 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38794216-3b34-49e7-861a-763533b55a07 None admin] PATH_INFO: `/v3/services` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:38:14.278127 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38794216-3b34-49e7-861a-763533b55a07 None admin] RBAC: Adding query filter params (name=placement) {{(pid=88113) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:38:14.278425 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38794216-3b34-49e7-861a-763533b55a07 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:38:14.279798 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38794216-3b34-49e7-861a-763533b55a07 None admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:38:14.285487 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 652/1299] 10.222.0.13 () {56 vars in 1116 bytes} [Wed Aug 31 09:38:14 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 09:38:14.292024 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b2ea04e-4b96-4372-99b3-de18ee41f9aa None admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:38:14.326434 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=100 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:38:14.403231 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b2ea04e-4b96-4372-99b3-de18ee41f9aa None admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:38:14.442546 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-049b357a-6d88-48c4-9f5a-99f4e40d9bba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:38:14.443163 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049b357a-6d88-48c4-9f5a-99f4e40d9bba None admin] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:38:14.443440 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049b357a-6d88-48c4-9f5a-99f4e40d9bba None admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:38:14.443767 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049b357a-6d88-48c4-9f5a-99f4e40d9bba None admin] PATH_INFO: `/v3/endpoints` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:38:14.445370 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-049b357a-6d88-48c4-9f5a-99f4e40d9bba None admin] RBAC: Adding query filter params (interface=public, service_id=9b582f8b4e36488a8fb2e95dd84680ed, region_id=RegionOne) {{(pid=88114) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:38:14.445764 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-049b357a-6d88-48c4-9f5a-99f4e40d9bba None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:38:14.447154 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-049b357a-6d88-48c4-9f5a-99f4e40d9bba None admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:38:14.453129 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 648/1300] 10.222.0.13 () {56 vars in 1251 bytes} [Wed Aug 31 09:38:14 2022] GET /identity/v3/endpoints?service_id=9b582f8b4e36488a8fb2e95dd84680ed&interface=public®ion_id=RegionOne => generated 194 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 09:38:14.462553 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38794216-3b34-49e7-861a-763533b55a07 None admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:38:14.556302 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=100 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:38:14.587424 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38794216-3b34-49e7-861a-763533b55a07 None admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:38:14.643946 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e1b320a-b4ac-43d4-b81c-0dbcaa9b4d2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:38:14.644965 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1b320a-b4ac-43d4-b81c-0dbcaa9b4d2c None admin] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:38:14.645474 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1b320a-b4ac-43d4-b81c-0dbcaa9b4d2c None admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:38:14.645953 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1b320a-b4ac-43d4-b81c-0dbcaa9b4d2c None admin] PATH_INFO: `/v3/services` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:38:14.647735 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e1b320a-b4ac-43d4-b81c-0dbcaa9b4d2c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:38:14.650244 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e1b320a-b4ac-43d4-b81c-0dbcaa9b4d2c None admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:38:14.664509 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 653/1301] 10.222.0.13 () {56 vars in 1087 bytes} [Wed Aug 31 09:38:14 2022] GET /identity/v3/services => generated 2191 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 09:38:16.784164 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4927c2-46be-481b-a1ff-97eeee2062a1 None None] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:38:16.784769 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4927c2-46be-481b-a1ff-97eeee2062a1 None None] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:38:16.785082 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4927c2-46be-481b-a1ff-97eeee2062a1 None None] PATH_INFO: `/` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:38:16.787532 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 649/1302] 10.222.0.13 () {54 vars in 927 bytes} [Wed Aug 31 09:38:16 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:38:16.793846 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e7aa98-0a51-4818-90b5-91ef83e866b7 None None] REQUEST_METHOD: `POST` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:38:16.794357 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e7aa98-0a51-4818-90b5-91ef83e866b7 None None] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:38:16.794357 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e7aa98-0a51-4818-90b5-91ef83e866b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:38:16.813589 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.auth.core [None req-e4e7aa98-0a51-4818-90b5-91ef83e866b7 None None] MFA Rules not processed for user `fd836cf590a145df9acf19cf3ae2ac56`. Rule list: `[]` (Enabled: `True`). {{(pid=88113) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:38:16.814887 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.fernet_utils [None req-e4e7aa98-0a51-4818-90b5-91ef83e866b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:38:16.905318 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 654/1303] 10.222.0.13 () {58 vars in 1024 bytes} [Wed Aug 31 09:38:16 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 09:38:17.198661 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b4927c2-46be-481b-a1ff-97eeee2062a1 None None] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:38:17.304394 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=100 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:38:17.312189 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b4927c2-46be-481b-a1ff-97eeee2062a1 None None] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:38:17.350457 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c4433c-df21-43fd-ade0-8f4adeb1c295 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:38:17.351147 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c4433c-df21-43fd-ade0-8f4adeb1c295 None admin] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:38:17.351466 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c4433c-df21-43fd-ade0-8f4adeb1c295 None admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:38:17.351777 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c4433c-df21-43fd-ade0-8f4adeb1c295 None admin] PATH_INFO: `/v3/services/placement` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:38:17.352406 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66c4433c-df21-43fd-ade0-8f4adeb1c295 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:38:17.353922 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66c4433c-df21-43fd-ade0-8f4adeb1c295 None admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:38:17.363527 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application [None req-66c4433c-df21-43fd-ade0-8f4adeb1c295 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Aug 31 09:38:17.363527 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:38:17.363527 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 31 09:38:17.363527 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 31 09:38:17.363527 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 31 09:38:17.363527 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 31 09:38:17.363527 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 31 09:38:17.363527 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 09:38:17.363527 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 31 09:38:17.363527 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application Aug 31 09:38:17.363527 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 31 09:38:17.363527 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application Aug 31 09:38:17.363527 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 09:38:17.363527 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 09:38:17.363527 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 09:38:17.363527 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 09:38:17.363527 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 09:38:17.365577 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 09:38:17.365577 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 09:38:17.365577 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 09:38:17.365577 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 09:38:17.365577 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 09:38:17.365577 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 09:38:17.365577 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 31 09:38:17.365577 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 31 09:38:17.365577 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 31 09:38:17.365577 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 31 09:38:17.365577 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 09:38:17.365577 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 09:38:17.365577 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 09:38:17.365577 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 09:38:17.365577 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 09:38:17.365577 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 09:38:17.367193 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 09:38:17.367193 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application with Lock( Aug 31 09:38:17.367193 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 09:38:17.367193 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self._enter() Aug 31 09:38:17.367193 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 09:38:17.367193 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 09:38:17.367193 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 09:38:17.367193 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application return self.creator() Aug 31 09:38:17.367193 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 09:38:17.367193 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application created_value = creator( Aug 31 09:38:17.367193 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 31 09:38:17.367193 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 09:38:17.367193 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 31 09:38:17.367193 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: ERROR keystone.server.flask.application Aug 31 09:38:17.369586 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 650/1304] 10.222.0.13 () {56 vars in 1117 bytes} [Wed Aug 31 09:38:17 2022] GET /identity/v3/services/placement => generated 90 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 09:38:17.371130 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e7aa98-0a51-4818-90b5-91ef83e866b7 None None] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:38:17.454405 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=100 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:38:17.480860 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e7aa98-0a51-4818-90b5-91ef83e866b7 None None] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:38:17.521217 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f00b4c-10d8-44c5-87a7-306612ddbfac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:38:17.522320 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f00b4c-10d8-44c5-87a7-306612ddbfac None admin] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:38:17.522610 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f00b4c-10d8-44c5-87a7-306612ddbfac None admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:38:17.522983 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f00b4c-10d8-44c5-87a7-306612ddbfac None admin] PATH_INFO: `/v3/services` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:38:17.523662 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0f00b4c-10d8-44c5-87a7-306612ddbfac None admin] RBAC: Adding query filter params (name=placement) {{(pid=88113) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:38:17.523965 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0f00b4c-10d8-44c5-87a7-306612ddbfac None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:38:17.525362 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0f00b4c-10d8-44c5-87a7-306612ddbfac None admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:38:17.531200 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 655/1305] 10.222.0.13 () {56 vars in 1116 bytes} [Wed Aug 31 09:38:17 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:38:17.537165 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c4433c-df21-43fd-ade0-8f4adeb1c295 None admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:38:17.666218 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c4433c-df21-43fd-ade0-8f4adeb1c295 None admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:38:17.730991 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6238dfd6-e8a3-4c5d-80fa-3b41eda23b26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:38:17.731925 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6238dfd6-e8a3-4c5d-80fa-3b41eda23b26 None admin] REQUEST_METHOD: `POST` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:38:17.732416 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6238dfd6-e8a3-4c5d-80fa-3b41eda23b26 None admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:38:17.732902 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6238dfd6-e8a3-4c5d-80fa-3b41eda23b26 None admin] PATH_INFO: `/v3/endpoints` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:38:17.734161 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6238dfd6-e8a3-4c5d-80fa-3b41eda23b26 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:38:17.736720 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6238dfd6-e8a3-4c5d-80fa-3b41eda23b26 None admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:38:17.771595 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 651/1306] 10.222.0.13 () {60 vars in 1144 bytes} [Wed Aug 31 09:38:17 2022] POST /identity/v3/endpoints => generated 331 bytes in 235 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 09:38:27.528688 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=30 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:38:27.762889 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:39:27.332673 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdfb2f8d-e45e-42c4-bfb8-ed629912eb20 None None] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:39:27.334037 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdfb2f8d-e45e-42c4-bfb8-ed629912eb20 None None] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:39:27.334536 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdfb2f8d-e45e-42c4-bfb8-ed629912eb20 None None] PATH_INFO: `/` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:39:27.336143 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 656/1307] 10.222.0.13 () {54 vars in 927 bytes} [Wed Aug 31 09:39:27 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:39:27.341881 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a996017-7382-4a3c-a234-b8fa057ba857 None None] REQUEST_METHOD: `POST` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:39:27.342796 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a996017-7382-4a3c-a234-b8fa057ba857 None None] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:39:27.343283 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a996017-7382-4a3c-a234-b8fa057ba857 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:39:27.488552 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.auth.core [None req-0a996017-7382-4a3c-a234-b8fa057ba857 None None] MFA Rules not processed for user `fd836cf590a145df9acf19cf3ae2ac56`. Rule list: `[]` (Enabled: `True`). {{(pid=88114) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:39:27.496551 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.fernet_utils [None req-0a996017-7382-4a3c-a234-b8fa057ba857 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88114) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:39:27.814133 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 652/1308] 10.222.0.13 () {58 vars in 1024 bytes} [Wed Aug 31 09:39:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 473 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:39:28.126382 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb11b9f-7463-4550-a289-87c02fd54a9b None None] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:39:28.127076 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb11b9f-7463-4550-a289-87c02fd54a9b None None] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:39:28.127570 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb11b9f-7463-4550-a289-87c02fd54a9b None None] PATH_INFO: `/` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:39:28.130949 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 657/1309] 10.222.0.13 () {54 vars in 965 bytes} [Wed Aug 31 09:39:28 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:39:28.137913 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5511824f-9847-46ba-a360-d69e9dee17e5 None None] REQUEST_METHOD: `POST` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:39:28.138219 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5511824f-9847-46ba-a360-d69e9dee17e5 None None] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:39:28.138459 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5511824f-9847-46ba-a360-d69e9dee17e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:39:28.249519 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.auth.core [None req-5511824f-9847-46ba-a360-d69e9dee17e5 None None] MFA Rules not processed for user `a9bdaacd597442c2bcde2c555f2cc140`. Rule list: `[]` (Enabled: `True`). {{(pid=88114) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:39:28.251087 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.fernet_utils [None req-5511824f-9847-46ba-a360-d69e9dee17e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88114) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:39:28.282114 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=100 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:39:28.370202 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 653/1310] 10.222.0.13 () {58 vars in 1062 bytes} [Wed Aug 31 09:39:28 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 233 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 09:39:28.379831 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb11b9f-7463-4550-a289-87c02fd54a9b None None] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:39:28.381622 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.fernet_utils [None req-feb11b9f-7463-4550-a289-87c02fd54a9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:39:28.506226 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb11b9f-7463-4550-a289-87c02fd54a9b None None] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:39:28.589097 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9efb2634-b718-4bd0-b27e-7c9b68582b05 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9bdaacd597442c2bcde2c555f2cc140', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '49548eb44d62415582af820ab66decee', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:39:28.589705 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9efb2634-b718-4bd0-b27e-7c9b68582b05 service neutron] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:39:28.590104 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9efb2634-b718-4bd0-b27e-7c9b68582b05 service neutron] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:39:28.590484 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9efb2634-b718-4bd0-b27e-7c9b68582b05 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:39:28.592191 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.fernet_utils [None req-9efb2634-b718-4bd0-b27e-7c9b68582b05 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:39:28.657323 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=100 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:39:28.717113 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9efb2634-b718-4bd0-b27e-7c9b68582b05 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:39:28.718757 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9efb2634-b718-4bd0-b27e-7c9b68582b05 service neutron] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:39:28.739846 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 658/1311] 10.222.0.13 () {60 vars in 1365 bytes} [Wed Aug 31 09:39:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 361 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:39:31.130470 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8119046-f829-45c9-b7cb-5d9ec288f56e None None] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:39:31.131041 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8119046-f829-45c9-b7cb-5d9ec288f56e None None] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:39:31.131041 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8119046-f829-45c9-b7cb-5d9ec288f56e None None] PATH_INFO: `/` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:39:31.132403 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 654/1312] 10.222.0.13 () {54 vars in 927 bytes} [Wed Aug 31 09:39:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:39:31.138672 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa9c89b-88a9-428b-8a31-1c8c08de01ee None None] REQUEST_METHOD: `POST` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:39:31.139104 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa9c89b-88a9-428b-8a31-1c8c08de01ee None None] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:39:31.139235 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa9c89b-88a9-428b-8a31-1c8c08de01ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:39:31.226599 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.auth.core [None req-8aa9c89b-88a9-428b-8a31-1c8c08de01ee None None] MFA Rules not processed for user `fd836cf590a145df9acf19cf3ae2ac56`. Rule list: `[]` (Enabled: `True`). {{(pid=88113) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:39:31.227753 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.fernet_utils [None req-8aa9c89b-88a9-428b-8a31-1c8c08de01ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:39:31.342070 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 659/1313] 10.222.0.13 () {58 vars in 1024 bytes} [Wed Aug 31 09:39:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:39:31.660892 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd95b59-cd81-4dd0-b9b3-882daf919fe6 None None] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:39:31.661171 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd95b59-cd81-4dd0-b9b3-882daf919fe6 None None] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:39:31.661474 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd95b59-cd81-4dd0-b9b3-882daf919fe6 None None] PATH_INFO: `/` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:39:31.662800 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 655/1314] 10.222.0.13 () {54 vars in 965 bytes} [Wed Aug 31 09:39:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:39:31.669246 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13736ed-c92b-47be-9de1-48282ce1e70e None None] REQUEST_METHOD: `POST` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:39:31.669623 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13736ed-c92b-47be-9de1-48282ce1e70e None None] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:39:31.669951 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13736ed-c92b-47be-9de1-48282ce1e70e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:39:31.683330 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=100 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:39:31.754899 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.auth.core [None req-b13736ed-c92b-47be-9de1-48282ce1e70e None None] MFA Rules not processed for user `a9bdaacd597442c2bcde2c555f2cc140`. Rule list: `[]` (Enabled: `True`). {{(pid=88113) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:39:31.756123 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.fernet_utils [None req-b13736ed-c92b-47be-9de1-48282ce1e70e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:39:31.836409 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 660/1315] 10.222.0.13 () {58 vars in 1062 bytes} [Wed Aug 31 09:39:31 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:39:31.845516 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd95b59-cd81-4dd0-b9b3-882daf919fe6 None None] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:39:31.847807 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.fernet_utils [None req-cdd95b59-cd81-4dd0-b9b3-882daf919fe6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88114) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:39:31.958842 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd95b59-cd81-4dd0-b9b3-882daf919fe6 None None] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:39:32.037819 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=100 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:39:32.040680 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f485e58c-8dd9-4fc9-a823-5b2cf4eab2e1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9bdaacd597442c2bcde2c555f2cc140', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '49548eb44d62415582af820ab66decee', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:39:32.041508 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f485e58c-8dd9-4fc9-a823-5b2cf4eab2e1 service neutron] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:39:32.041855 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f485e58c-8dd9-4fc9-a823-5b2cf4eab2e1 service neutron] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:39:32.042126 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f485e58c-8dd9-4fc9-a823-5b2cf4eab2e1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:39:32.044133 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.fernet_utils [None req-f485e58c-8dd9-4fc9-a823-5b2cf4eab2e1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88114) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:39:32.160903 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f485e58c-8dd9-4fc9-a823-5b2cf4eab2e1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:39:32.162364 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f485e58c-8dd9-4fc9-a823-5b2cf4eab2e1 service neutron] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:39:32.181870 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 656/1316] 10.222.0.13 () {60 vars in 1365 bytes} [Wed Aug 31 09:39:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 337 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:39:34.752281 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb29420-6ff2-49dd-9e80-fc08ec2618d4 None None] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:39:34.752281 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb29420-6ff2-49dd-9e80-fc08ec2618d4 None None] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:39:34.752281 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb29420-6ff2-49dd-9e80-fc08ec2618d4 None None] PATH_INFO: `/` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:39:34.752281 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 661/1317] 10.222.0.13 () {54 vars in 927 bytes} [Wed Aug 31 09:39:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:39:34.755014 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201b540a-d9d5-4cde-b9b3-489082d3aeed None None] REQUEST_METHOD: `POST` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:39:34.755649 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201b540a-d9d5-4cde-b9b3-489082d3aeed None None] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:39:34.755941 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201b540a-d9d5-4cde-b9b3-489082d3aeed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:39:34.826799 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.auth.core [None req-201b540a-d9d5-4cde-b9b3-489082d3aeed None None] MFA Rules not processed for user `1dc7ae39ad83425fb2f7ddb6c98b0a20`. Rule list: `[]` (Enabled: `True`). {{(pid=88114) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:39:34.828000 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.fernet_utils [None req-201b540a-d9d5-4cde-b9b3-489082d3aeed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88114) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:39:34.882113 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=100 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:39:34.931861 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 657/1318] 10.222.0.13 () {58 vars in 1024 bytes} [Wed Aug 31 09:39:34 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:39:35.217005 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb29420-6ff2-49dd-9e80-fc08ec2618d4 None None] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:39:35.218608 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.fernet_utils [None req-9bb29420-6ff2-49dd-9e80-fc08ec2618d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:39:35.323237 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb29420-6ff2-49dd-9e80-fc08ec2618d4 None None] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:39:35.326086 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=100 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:39:35.398029 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd340e6-1749-4cca-98ab-d723472e961e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9bdaacd597442c2bcde2c555f2cc140', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '49548eb44d62415582af820ab66decee', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:39:35.398636 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd340e6-1749-4cca-98ab-d723472e961e service neutron] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:39:35.398909 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd340e6-1749-4cca-98ab-d723472e961e service neutron] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:39:35.399177 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd340e6-1749-4cca-98ab-d723472e961e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:39:35.400779 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.fernet_utils [None req-3fd340e6-1749-4cca-98ab-d723472e961e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:39:35.522373 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fd340e6-1749-4cca-98ab-d723472e961e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:39:35.523735 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fd340e6-1749-4cca-98ab-d723472e961e service neutron] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:39:35.544820 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 662/1319] 10.222.0.13 () {60 vars in 1365 bytes} [Wed Aug 31 09:39:35 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 328 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:39:38.005643 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8c7c82-8462-4033-a093-b43188d38b71 None None] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:39:38.006171 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8c7c82-8462-4033-a093-b43188d38b71 None None] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:39:38.006171 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8c7c82-8462-4033-a093-b43188d38b71 None None] PATH_INFO: `/` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:39:38.007501 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 658/1320] 10.222.0.13 () {54 vars in 927 bytes} [Wed Aug 31 09:39:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:39:38.012769 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62703399-c40e-4a8f-bd82-bb721d6e6db5 None None] REQUEST_METHOD: `POST` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:39:38.013174 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62703399-c40e-4a8f-bd82-bb721d6e6db5 None None] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:39:38.013311 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62703399-c40e-4a8f-bd82-bb721d6e6db5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:39:38.054019 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=100 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:39:38.083712 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.auth.core [None req-62703399-c40e-4a8f-bd82-bb721d6e6db5 None None] MFA Rules not processed for user `fd836cf590a145df9acf19cf3ae2ac56`. Rule list: `[]` (Enabled: `True`). {{(pid=88113) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:39:38.084914 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.fernet_utils [None req-62703399-c40e-4a8f-bd82-bb721d6e6db5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:39:38.179216 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 663/1321] 10.222.0.13 () {58 vars in 1024 bytes} [Wed Aug 31 09:39:38 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:39:38.463467 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea8c7c82-8462-4033-a093-b43188d38b71 None None] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:39:38.464981 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.fernet_utils [None req-ea8c7c82-8462-4033-a093-b43188d38b71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88114) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:39:38.569010 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea8c7c82-8462-4033-a093-b43188d38b71 None None] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:39:38.641703 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69dda3b6-21a3-4d64-b365-af1c5231af90 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9bdaacd597442c2bcde2c555f2cc140', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '49548eb44d62415582af820ab66decee', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:39:38.642309 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69dda3b6-21a3-4d64-b365-af1c5231af90 service neutron] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:39:38.642588 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69dda3b6-21a3-4d64-b365-af1c5231af90 service neutron] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:39:38.642864 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69dda3b6-21a3-4d64-b365-af1c5231af90 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:39:38.645059 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.fernet_utils [None req-69dda3b6-21a3-4d64-b365-af1c5231af90 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88114) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:39:38.652977 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=100 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:39:38.758167 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69dda3b6-21a3-4d64-b365-af1c5231af90 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:39:38.759521 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69dda3b6-21a3-4d64-b365-af1c5231af90 service neutron] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:39:38.778890 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 659/1322] 10.222.0.13 () {60 vars in 1365 bytes} [Wed Aug 31 09:39:38 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:39:41.418849 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895d88c4-6575-4935-a1c9-06d6b3ce190e None None] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:39:41.419388 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895d88c4-6575-4935-a1c9-06d6b3ce190e None None] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:39:41.419388 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895d88c4-6575-4935-a1c9-06d6b3ce190e None None] PATH_INFO: `/` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:39:41.420731 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 664/1323] 10.222.0.13 () {54 vars in 927 bytes} [Wed Aug 31 09:39:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:39:41.426396 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6442ef1f-78ac-4829-8137-ea1034b8d2e1 None None] REQUEST_METHOD: `POST` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:39:41.426934 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6442ef1f-78ac-4829-8137-ea1034b8d2e1 None None] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:39:41.426934 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6442ef1f-78ac-4829-8137-ea1034b8d2e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:39:41.494418 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.auth.core [None req-6442ef1f-78ac-4829-8137-ea1034b8d2e1 None None] MFA Rules not processed for user `fd836cf590a145df9acf19cf3ae2ac56`. Rule list: `[]` (Enabled: `True`). {{(pid=88114) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:39:41.495626 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.fernet_utils [None req-6442ef1f-78ac-4829-8137-ea1034b8d2e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88114) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:39:41.576212 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=100 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:39:41.619228 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 660/1324] 10.222.0.13 () {58 vars in 1024 bytes} [Wed Aug 31 09:39:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:39:41.911130 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-895d88c4-6575-4935-a1c9-06d6b3ce190e None None] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:39:41.912962 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.fernet_utils [None req-895d88c4-6575-4935-a1c9-06d6b3ce190e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:39:42.028867 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-895d88c4-6575-4935-a1c9-06d6b3ce190e None None] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:39:42.042633 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=100 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:39:42.107617 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27feb7ef-2ba1-4526-8eb5-ac5e3427e89a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9bdaacd597442c2bcde2c555f2cc140', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '49548eb44d62415582af820ab66decee', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:39:42.108250 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27feb7ef-2ba1-4526-8eb5-ac5e3427e89a service neutron] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:39:42.108530 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27feb7ef-2ba1-4526-8eb5-ac5e3427e89a service neutron] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:39:42.108798 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27feb7ef-2ba1-4526-8eb5-ac5e3427e89a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:39:42.110469 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.fernet_utils [None req-27feb7ef-2ba1-4526-8eb5-ac5e3427e89a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:39:42.221547 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27feb7ef-2ba1-4526-8eb5-ac5e3427e89a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:39:42.223006 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27feb7ef-2ba1-4526-8eb5-ac5e3427e89a service neutron] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:39:42.241855 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 665/1325] 10.222.0.13 () {60 vars in 1365 bytes} [Wed Aug 31 09:39:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 331 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:39:44.243394 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f8ffc5-ee55-4f98-a692-2c51f7a7a2f9 None None] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:39:44.243942 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f8ffc5-ee55-4f98-a692-2c51f7a7a2f9 None None] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:39:44.243942 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f8ffc5-ee55-4f98-a692-2c51f7a7a2f9 None None] PATH_INFO: `/` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:39:44.245215 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 661/1326] 10.222.0.13 () {54 vars in 927 bytes} [Wed Aug 31 09:39:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:39:44.250459 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad775d7-d152-4b5a-a955-d6b0ebf46908 None None] REQUEST_METHOD: `POST` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:39:44.250905 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad775d7-d152-4b5a-a955-d6b0ebf46908 None None] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:39:44.251003 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad775d7-d152-4b5a-a955-d6b0ebf46908 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:39:44.301597 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=100 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:39:44.321615 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.auth.core [None req-0ad775d7-d152-4b5a-a955-d6b0ebf46908 None None] MFA Rules not processed for user `fd836cf590a145df9acf19cf3ae2ac56`. Rule list: `[]` (Enabled: `True`). {{(pid=88113) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:39:44.322853 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.fernet_utils [None req-0ad775d7-d152-4b5a-a955-d6b0ebf46908 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:39:44.421251 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 666/1327] 10.222.0.13 () {58 vars in 1024 bytes} [Wed Aug 31 09:39:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:39:44.699811 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f8ffc5-ee55-4f98-a692-2c51f7a7a2f9 None None] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:39:44.702150 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.fernet_utils [None req-36f8ffc5-ee55-4f98-a692-2c51f7a7a2f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88114) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:39:44.817999 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f8ffc5-ee55-4f98-a692-2c51f7a7a2f9 None None] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:39:44.901338 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde59a4c-7ecc-40e9-965b-9cb237c3e48a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9bdaacd597442c2bcde2c555f2cc140', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '49548eb44d62415582af820ab66decee', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:39:44.901885 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde59a4c-7ecc-40e9-965b-9cb237c3e48a service neutron] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:39:44.902156 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde59a4c-7ecc-40e9-965b-9cb237c3e48a service neutron] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:39:44.902422 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde59a4c-7ecc-40e9-965b-9cb237c3e48a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:39:44.904062 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.fernet_utils [None req-dde59a4c-7ecc-40e9-965b-9cb237c3e48a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88114) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:39:44.935212 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=100 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:39:45.037247 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dde59a4c-7ecc-40e9-965b-9cb237c3e48a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:39:45.038737 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dde59a4c-7ecc-40e9-965b-9cb237c3e48a service neutron] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:39:45.061488 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 662/1328] 10.222.0.13 () {60 vars in 1365 bytes} [Wed Aug 31 09:39:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 362 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:39:47.109326 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72dc58f7-3978-4e3a-9829-4686311cb16d None None] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:39:47.109872 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72dc58f7-3978-4e3a-9829-4686311cb16d None None] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:39:47.109872 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72dc58f7-3978-4e3a-9829-4686311cb16d None None] PATH_INFO: `/` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:39:47.111184 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 667/1329] 10.222.0.13 () {54 vars in 927 bytes} [Wed Aug 31 09:39:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 31 09:39:47.117673 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d97bcc8-3259-49d7-90f7-9ae463e2b3f3 None None] REQUEST_METHOD: `POST` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:39:47.118166 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d97bcc8-3259-49d7-90f7-9ae463e2b3f3 None None] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:39:47.118565 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d97bcc8-3259-49d7-90f7-9ae463e2b3f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:39:47.198306 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.auth.core [None req-9d97bcc8-3259-49d7-90f7-9ae463e2b3f3 None None] MFA Rules not processed for user `1dc7ae39ad83425fb2f7ddb6c98b0a20`. Rule list: `[]` (Enabled: `True`). {{(pid=88114) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:39:47.199513 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.fernet_utils [None req-9d97bcc8-3259-49d7-90f7-9ae463e2b3f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88114) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:39:47.283086 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=100 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:39:47.322126 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 663/1330] 10.222.0.13 () {58 vars in 1024 bytes} [Wed Aug 31 09:39:47 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:39:47.608639 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72dc58f7-3978-4e3a-9829-4686311cb16d None None] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:39:47.609704 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.fernet_utils [None req-72dc58f7-3978-4e3a-9829-4686311cb16d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:39:47.762330 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72dc58f7-3978-4e3a-9829-4686311cb16d None None] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:39:47.783661 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=100 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:39:47.840957 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-005a919d-762a-426e-af77-1b88d1238e58 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9bdaacd597442c2bcde2c555f2cc140', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '49548eb44d62415582af820ab66decee', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:39:47.841617 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005a919d-762a-426e-af77-1b88d1238e58 service neutron] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:39:47.841898 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005a919d-762a-426e-af77-1b88d1238e58 service neutron] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:39:47.842168 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005a919d-762a-426e-af77-1b88d1238e58 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:39:47.843776 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.fernet_utils [None req-005a919d-762a-426e-af77-1b88d1238e58 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:39:47.953432 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-005a919d-762a-426e-af77-1b88d1238e58 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:39:47.954924 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-005a919d-762a-426e-af77-1b88d1238e58 service neutron] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:39:47.973325 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 668/1331] 10.222.0.13 () {60 vars in 1365 bytes} [Wed Aug 31 09:39:47 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 366 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 09:39:50.157881 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d46b5e0-fd2b-4bc4-a88e-65d70b39a7a6 None None] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:39:50.158416 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d46b5e0-fd2b-4bc4-a88e-65d70b39a7a6 None None] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:39:50.158511 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d46b5e0-fd2b-4bc4-a88e-65d70b39a7a6 None None] PATH_INFO: `/` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:39:50.159813 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 664/1332] 10.222.0.13 () {54 vars in 927 bytes} [Wed Aug 31 09:39:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:39:50.165536 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490c626b-c7b6-4e03-8cf3-8dd543d6eac6 None None] REQUEST_METHOD: `POST` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:39:50.166016 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490c626b-c7b6-4e03-8cf3-8dd543d6eac6 None None] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:39:50.166114 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490c626b-c7b6-4e03-8cf3-8dd543d6eac6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:39:50.230641 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=100 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:39:50.233409 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.auth.core [None req-490c626b-c7b6-4e03-8cf3-8dd543d6eac6 None None] MFA Rules not processed for user `fd836cf590a145df9acf19cf3ae2ac56`. Rule list: `[]` (Enabled: `True`). {{(pid=88113) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:39:50.235171 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.fernet_utils [None req-490c626b-c7b6-4e03-8cf3-8dd543d6eac6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:39:50.337856 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 669/1333] 10.222.0.13 () {58 vars in 1024 bytes} [Wed Aug 31 09:39:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:39:50.631856 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d46b5e0-fd2b-4bc4-a88e-65d70b39a7a6 None None] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:39:50.633438 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.fernet_utils [None req-4d46b5e0-fd2b-4bc4-a88e-65d70b39a7a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88114) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:39:50.736674 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d46b5e0-fd2b-4bc4-a88e-65d70b39a7a6 None None] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:39:50.811020 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd26ae9a-12c0-4aba-989b-4cb752dc69f8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9bdaacd597442c2bcde2c555f2cc140', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '49548eb44d62415582af820ab66decee', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:39:50.811630 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd26ae9a-12c0-4aba-989b-4cb752dc69f8 service neutron] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:39:50.811906 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd26ae9a-12c0-4aba-989b-4cb752dc69f8 service neutron] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:39:50.812174 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd26ae9a-12c0-4aba-989b-4cb752dc69f8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:39:50.813791 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.fernet_utils [None req-cd26ae9a-12c0-4aba-989b-4cb752dc69f8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88114) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:39:50.840462 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=100 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:39:50.929150 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd26ae9a-12c0-4aba-989b-4cb752dc69f8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:39:50.930537 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd26ae9a-12c0-4aba-989b-4cb752dc69f8 service neutron] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:39:50.950355 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 665/1334] 10.222.0.13 () {60 vars in 1365 bytes} [Wed Aug 31 09:39:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:39:52.922242 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff145657-a829-4be8-ac7f-48c2bd8ac5ff None None] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:39:52.922823 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff145657-a829-4be8-ac7f-48c2bd8ac5ff None None] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:39:52.922823 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff145657-a829-4be8-ac7f-48c2bd8ac5ff None None] PATH_INFO: `/` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:39:52.924172 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 670/1335] 10.222.0.13 () {54 vars in 927 bytes} [Wed Aug 31 09:39:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:39:52.929584 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cecd04d-3360-4c4d-a9f2-0ae8133ff336 None None] REQUEST_METHOD: `POST` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:39:52.930003 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cecd04d-3360-4c4d-a9f2-0ae8133ff336 None None] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:39:52.930245 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cecd04d-3360-4c4d-a9f2-0ae8133ff336 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:39:52.999483 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.auth.core [None req-7cecd04d-3360-4c4d-a9f2-0ae8133ff336 None None] MFA Rules not processed for user `fd836cf590a145df9acf19cf3ae2ac56`. Rule list: `[]` (Enabled: `True`). {{(pid=88114) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:39:53.000768 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.fernet_utils [None req-7cecd04d-3360-4c4d-a9f2-0ae8133ff336 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88114) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:39:53.093173 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=100 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:39:53.118447 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 666/1336] 10.222.0.13 () {58 vars in 1024 bytes} [Wed Aug 31 09:39:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:39:53.402556 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff145657-a829-4be8-ac7f-48c2bd8ac5ff None None] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:39:53.404077 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.fernet_utils [None req-ff145657-a829-4be8-ac7f-48c2bd8ac5ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:39:53.518158 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff145657-a829-4be8-ac7f-48c2bd8ac5ff None None] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:39:53.549903 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=100 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:39:53.600110 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4749a9f2-6db5-43dd-9a8d-a5381b5d4b04 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9bdaacd597442c2bcde2c555f2cc140', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '49548eb44d62415582af820ab66decee', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:39:53.601803 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4749a9f2-6db5-43dd-9a8d-a5381b5d4b04 service neutron] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:39:53.601803 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4749a9f2-6db5-43dd-9a8d-a5381b5d4b04 service neutron] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:39:53.601803 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4749a9f2-6db5-43dd-9a8d-a5381b5d4b04 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:39:53.603315 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.fernet_utils [None req-4749a9f2-6db5-43dd-9a8d-a5381b5d4b04 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:39:53.719462 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4749a9f2-6db5-43dd-9a8d-a5381b5d4b04 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:39:53.721099 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4749a9f2-6db5-43dd-9a8d-a5381b5d4b04 service neutron] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:39:53.739288 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 671/1337] 10.222.0.13 () {60 vars in 1365 bytes} [Wed Aug 31 09:39:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 337 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:39:56.056705 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e67f39-7882-4438-ab14-e2ccadf581ef None None] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:39:56.057438 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e67f39-7882-4438-ab14-e2ccadf581ef None None] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:39:56.057438 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e67f39-7882-4438-ab14-e2ccadf581ef None None] PATH_INFO: `/` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:39:56.058730 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 667/1338] 10.222.0.13 () {54 vars in 927 bytes} [Wed Aug 31 09:39:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 31 09:39:56.065212 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46231372-9e84-4285-a745-2b6c71dbf105 None None] REQUEST_METHOD: `POST` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:39:56.065844 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46231372-9e84-4285-a745-2b6c71dbf105 None None] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:39:56.066211 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46231372-9e84-4285-a745-2b6c71dbf105 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:39:56.137695 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.auth.core [None req-46231372-9e84-4285-a745-2b6c71dbf105 None None] MFA Rules not processed for user `fd836cf590a145df9acf19cf3ae2ac56`. Rule list: `[]` (Enabled: `True`). {{(pid=88113) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:39:56.138930 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.fernet_utils [None req-46231372-9e84-4285-a745-2b6c71dbf105 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:39:56.151706 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=100 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:39:56.249851 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 672/1339] 10.222.0.13 () {58 vars in 1024 bytes} [Wed Aug 31 09:39:56 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:39:56.532510 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e67f39-7882-4438-ab14-e2ccadf581ef None None] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:39:56.534605 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.fernet_utils [None req-96e67f39-7882-4438-ab14-e2ccadf581ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88114) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:39:56.640256 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e67f39-7882-4438-ab14-e2ccadf581ef None None] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:39:56.717332 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1781eec-f1fd-4f8d-886a-183869d2e2aa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9bdaacd597442c2bcde2c555f2cc140', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '49548eb44d62415582af820ab66decee', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:39:56.717965 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1781eec-f1fd-4f8d-886a-183869d2e2aa service neutron] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:39:56.718240 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1781eec-f1fd-4f8d-886a-183869d2e2aa service neutron] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:39:56.718509 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1781eec-f1fd-4f8d-886a-183869d2e2aa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:39:56.720090 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.fernet_utils [None req-c1781eec-f1fd-4f8d-886a-183869d2e2aa service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88114) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:39:56.757045 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=100 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:39:56.832420 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1781eec-f1fd-4f8d-886a-183869d2e2aa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:39:56.834187 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1781eec-f1fd-4f8d-886a-183869d2e2aa service neutron] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:39:56.854665 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 668/1340] 10.222.0.13 () {60 vars in 1365 bytes} [Wed Aug 31 09:39:56 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:40:00.794182 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad22fd9-0855-4f1d-80cd-cdec422e9da6 None None] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:40:00.794749 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad22fd9-0855-4f1d-80cd-cdec422e9da6 None None] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:40:00.794749 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad22fd9-0855-4f1d-80cd-cdec422e9da6 None None] PATH_INFO: `/` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:40:00.796092 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 673/1341] 10.222.0.13 () {54 vars in 927 bytes} [Wed Aug 31 09:40:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:40:00.802784 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9aab63-64a8-42a6-a6da-9a2d35b460cb None None] REQUEST_METHOD: `POST` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:40:00.803343 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9aab63-64a8-42a6-a6da-9a2d35b460cb None None] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:40:00.803343 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9aab63-64a8-42a6-a6da-9a2d35b460cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:40:00.880450 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.auth.core [None req-bd9aab63-64a8-42a6-a6da-9a2d35b460cb None None] MFA Rules not processed for user `fd836cf590a145df9acf19cf3ae2ac56`. Rule list: `[]` (Enabled: `True`). {{(pid=88114) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:40:00.881705 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.fernet_utils [None req-bd9aab63-64a8-42a6-a6da-9a2d35b460cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88114) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:40:00.997778 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=100 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:40:01.010586 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 669/1342] 10.222.0.13 () {58 vars in 1024 bytes} [Wed Aug 31 09:40:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:40:01.301410 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad22fd9-0855-4f1d-80cd-cdec422e9da6 None None] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:40:01.303184 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.fernet_utils [None req-0ad22fd9-0855-4f1d-80cd-cdec422e9da6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:40:01.418817 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad22fd9-0855-4f1d-80cd-cdec422e9da6 None None] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:40:01.460635 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=100 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:40:01.501346 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-581f6d6e-d761-4500-8107-5a7b84e9a2e0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9bdaacd597442c2bcde2c555f2cc140', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '49548eb44d62415582af820ab66decee', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:40:01.502628 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581f6d6e-d761-4500-8107-5a7b84e9a2e0 service neutron] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:40:01.502903 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581f6d6e-d761-4500-8107-5a7b84e9a2e0 service neutron] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:40:01.503354 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581f6d6e-d761-4500-8107-5a7b84e9a2e0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:40:01.505031 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.fernet_utils [None req-581f6d6e-d761-4500-8107-5a7b84e9a2e0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:40:01.645883 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-581f6d6e-d761-4500-8107-5a7b84e9a2e0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:40:01.647274 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-581f6d6e-d761-4500-8107-5a7b84e9a2e0 service neutron] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:40:01.672485 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 674/1343] 10.222.0.13 () {60 vars in 1365 bytes} [Wed Aug 31 09:40:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 372 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:40:04.403935 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d771043-6c2d-4926-a66b-48ef365d7cfd None None] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:40:04.404758 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d771043-6c2d-4926-a66b-48ef365d7cfd None None] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:40:04.404758 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d771043-6c2d-4926-a66b-48ef365d7cfd None None] PATH_INFO: `/` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:40:04.405758 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 670/1344] 10.222.0.13 () {54 vars in 927 bytes} [Wed Aug 31 09:40:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 31 09:40:04.413005 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094489a9-9ede-45b5-acaf-1234756b9842 None None] REQUEST_METHOD: `POST` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:40:04.413689 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094489a9-9ede-45b5-acaf-1234756b9842 None None] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:40:04.413967 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094489a9-9ede-45b5-acaf-1234756b9842 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:40:04.533967 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.auth.core [None req-094489a9-9ede-45b5-acaf-1234756b9842 None None] MFA Rules not processed for user `fd836cf590a145df9acf19cf3ae2ac56`. Rule list: `[]` (Enabled: `True`). {{(pid=88113) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:40:04.535346 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.fernet_utils [None req-094489a9-9ede-45b5-acaf-1234756b9842 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:40:04.558456 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=100 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:40:04.676459 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 675/1345] 10.222.0.13 () {58 vars in 1024 bytes} [Wed Aug 31 09:40:04 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 265 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 09:40:04.984492 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d771043-6c2d-4926-a66b-48ef365d7cfd None None] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:40:04.987493 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.fernet_utils [None req-9d771043-6c2d-4926-a66b-48ef365d7cfd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88114) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:40:05.115797 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d771043-6c2d-4926-a66b-48ef365d7cfd None None] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:40:05.230066 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c73cc24-5938-4d0a-b67d-5b9eaedb1f6e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9bdaacd597442c2bcde2c555f2cc140', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '49548eb44d62415582af820ab66decee', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:40:05.231008 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c73cc24-5938-4d0a-b67d-5b9eaedb1f6e service neutron] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:40:05.231486 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c73cc24-5938-4d0a-b67d-5b9eaedb1f6e service neutron] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:40:05.231957 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c73cc24-5938-4d0a-b67d-5b9eaedb1f6e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:40:05.234942 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.fernet_utils [None req-7c73cc24-5938-4d0a-b67d-5b9eaedb1f6e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88114) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:40:05.310781 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=100 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:40:05.401958 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c73cc24-5938-4d0a-b67d-5b9eaedb1f6e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:40:05.403551 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c73cc24-5938-4d0a-b67d-5b9eaedb1f6e service neutron] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:40:05.425173 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 671/1346] 10.222.0.13 () {60 vars in 1365 bytes} [Wed Aug 31 09:40:04 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 441 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:40:10.393925 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db87c692-1bfe-4e8e-9132-023438f9fb4a None None] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:40:10.394792 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db87c692-1bfe-4e8e-9132-023438f9fb4a None None] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:40:10.394792 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db87c692-1bfe-4e8e-9132-023438f9fb4a None None] PATH_INFO: `/` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:40:10.396375 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 676/1347] 10.222.0.13 () {54 vars in 927 bytes} [Wed Aug 31 09:40:10 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:40:10.402923 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70659c7-1e95-4cfd-ab94-527d2c77ff53 None None] REQUEST_METHOD: `POST` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:40:10.403753 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70659c7-1e95-4cfd-ab94-527d2c77ff53 None None] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:40:10.403753 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70659c7-1e95-4cfd-ab94-527d2c77ff53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:40:10.505252 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.auth.core [None req-a70659c7-1e95-4cfd-ab94-527d2c77ff53 None None] MFA Rules not processed for user `fd836cf590a145df9acf19cf3ae2ac56`. Rule list: `[]` (Enabled: `True`). {{(pid=88114) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:40:10.507435 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.fernet_utils [None req-a70659c7-1e95-4cfd-ab94-527d2c77ff53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88114) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:40:10.649789 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=100 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:40:10.653146 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 672/1348] 10.222.0.13 () {58 vars in 1024 bytes} [Wed Aug 31 09:40:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 251 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:40:10.960281 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db87c692-1bfe-4e8e-9132-023438f9fb4a None None] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:40:10.962132 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.fernet_utils [None req-db87c692-1bfe-4e8e-9132-023438f9fb4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:40:11.093004 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db87c692-1bfe-4e8e-9132-023438f9fb4a None None] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:40:11.156808 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=100 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:40:11.197912 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9615994-3277-4efe-807d-4c422f7aa434 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9bdaacd597442c2bcde2c555f2cc140', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '49548eb44d62415582af820ab66decee', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:40:11.198550 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9615994-3277-4efe-807d-4c422f7aa434 service neutron] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:40:11.198824 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9615994-3277-4efe-807d-4c422f7aa434 service neutron] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:40:11.199081 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9615994-3277-4efe-807d-4c422f7aa434 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:40:11.200744 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.fernet_utils [None req-f9615994-3277-4efe-807d-4c422f7aa434 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:40:11.349554 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9615994-3277-4efe-807d-4c422f7aa434 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:40:11.351098 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9615994-3277-4efe-807d-4c422f7aa434 service neutron] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:40:11.372506 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 677/1349] 10.222.0.13 () {60 vars in 1365 bytes} [Wed Aug 31 09:40:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 413 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:40:16.667057 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b7027e-ff53-492d-883e-f7a4dc7aff13 None None] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:40:16.668090 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b7027e-ff53-492d-883e-f7a4dc7aff13 None None] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:40:16.668522 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b7027e-ff53-492d-883e-f7a4dc7aff13 None None] PATH_INFO: `/` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:40:16.670374 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 673/1350] 10.222.0.13 () {54 vars in 921 bytes} [Wed Aug 31 09:40:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:40:16.675888 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f66cc3-34d3-47c1-9773-10d359231c16 None None] REQUEST_METHOD: `POST` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:40:16.676876 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f66cc3-34d3-47c1-9773-10d359231c16 None None] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:40:16.677421 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f66cc3-34d3-47c1-9773-10d359231c16 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:40:16.773648 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.auth.core [None req-04f66cc3-34d3-47c1-9773-10d359231c16 None None] MFA Rules not processed for user `1ff3aff5ef444910be6dbb989d0e3194`. Rule list: `[]` (Enabled: `True`). {{(pid=88113) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:40:16.775070 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.fernet_utils [None req-04f66cc3-34d3-47c1-9773-10d359231c16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:40:16.795814 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=100 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:40:16.913789 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 678/1351] 10.222.0.13 () {58 vars in 1018 bytes} [Wed Aug 31 09:40:16 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 239 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:40:21.400362 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81366dd8-eed7-47a2-8d4d-84953ee25e81 None None] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:40:21.400888 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81366dd8-eed7-47a2-8d4d-84953ee25e81 None None] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:40:21.400991 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81366dd8-eed7-47a2-8d4d-84953ee25e81 None None] PATH_INFO: `/` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:40:21.402350 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 674/1352] 10.222.0.13 () {54 vars in 921 bytes} [Wed Aug 31 09:40:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:40:21.411921 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5974451b-68f4-4cac-8deb-cb6a7835318a None None] REQUEST_METHOD: `POST` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:40:21.412407 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5974451b-68f4-4cac-8deb-cb6a7835318a None None] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:40:21.412496 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5974451b-68f4-4cac-8deb-cb6a7835318a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:40:21.514406 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.auth.core [None req-5974451b-68f4-4cac-8deb-cb6a7835318a None None] MFA Rules not processed for user `1ff3aff5ef444910be6dbb989d0e3194`. Rule list: `[]` (Enabled: `True`). {{(pid=88113) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:40:21.515808 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.fernet_utils [None req-5974451b-68f4-4cac-8deb-cb6a7835318a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:40:21.603891 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=100 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:40:21.646594 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 679/1353] 10.222.0.13 () {58 vars in 1018 bytes} [Wed Aug 31 09:40:21 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 236 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:40:23.269719 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a68159-f857-40f1-841b-408096a794ac None None] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:40:23.270297 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a68159-f857-40f1-841b-408096a794ac None None] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:40:23.270558 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a68159-f857-40f1-841b-408096a794ac None None] PATH_INFO: `/` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:40:23.272071 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 675/1354] 10.222.0.13 () {54 vars in 921 bytes} [Wed Aug 31 09:40:23 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:40:23.277914 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c298e76-2d0c-45e4-bfdc-46f7c5b59064 None None] REQUEST_METHOD: `POST` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:40:23.278510 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c298e76-2d0c-45e4-bfdc-46f7c5b59064 None None] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:40:23.278605 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c298e76-2d0c-45e4-bfdc-46f7c5b59064 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:40:23.374379 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.auth.core [None req-8c298e76-2d0c-45e4-bfdc-46f7c5b59064 None None] MFA Rules not processed for user `1ff3aff5ef444910be6dbb989d0e3194`. Rule list: `[]` (Enabled: `True`). {{(pid=88113) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:40:23.375611 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.fernet_utils [None req-8c298e76-2d0c-45e4-bfdc-46f7c5b59064 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:40:23.502319 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 680/1355] 10.222.0.13 () {58 vars in 1018 bytes} [Wed Aug 31 09:40:23 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 226 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:40:26.924387 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4c70c4-3b2b-4f3c-bb2c-cce2f83af082 None None] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:40:26.925065 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4c70c4-3b2b-4f3c-bb2c-cce2f83af082 None None] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:40:26.925065 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4c70c4-3b2b-4f3c-bb2c-cce2f83af082 None None] PATH_INFO: `/` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:40:26.926177 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 676/1356] 10.222.0.13 () {54 vars in 927 bytes} [Wed Aug 31 09:40:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:40:26.932798 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d63ede-24a8-4e99-9cfa-316e3956adf8 None None] REQUEST_METHOD: `POST` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:40:26.933308 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d63ede-24a8-4e99-9cfa-316e3956adf8 None None] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:40:26.933627 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d63ede-24a8-4e99-9cfa-316e3956adf8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:40:26.975616 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=100 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:40:27.012270 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.auth.core [None req-67d63ede-24a8-4e99-9cfa-316e3956adf8 None None] MFA Rules not processed for user `fd836cf590a145df9acf19cf3ae2ac56`. Rule list: `[]` (Enabled: `True`). {{(pid=88113) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:40:27.013563 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.fernet_utils [None req-67d63ede-24a8-4e99-9cfa-316e3956adf8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:40:27.132894 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 681/1357] 10.222.0.13 () {58 vars in 1024 bytes} [Wed Aug 31 09:40:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:40:27.480849 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006f9420-1c4c-4172-8995-f130c6287d3e None None] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:40:27.481647 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006f9420-1c4c-4172-8995-f130c6287d3e None None] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:40:27.482197 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006f9420-1c4c-4172-8995-f130c6287d3e None None] PATH_INFO: `/` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:40:27.484848 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 677/1358] 10.222.0.13 () {54 vars in 962 bytes} [Wed Aug 31 09:40:27 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:40:27.492936 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437c6724-2e65-4bc7-b9d5-6f058a82882c None None] REQUEST_METHOD: `POST` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:40:27.493351 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437c6724-2e65-4bc7-b9d5-6f058a82882c None None] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:40:27.493741 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437c6724-2e65-4bc7-b9d5-6f058a82882c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:40:27.594873 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.auth.core [None req-437c6724-2e65-4bc7-b9d5-6f058a82882c None None] MFA Rules not processed for user `dc076493239d4ccdbdc37c0369c49529`. Rule list: `[]` (Enabled: `True`). {{(pid=88113) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:40:27.596424 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.fernet_utils [None req-437c6724-2e65-4bc7-b9d5-6f058a82882c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:40:27.651969 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=100 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:40:27.755260 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 682/1359] 10.222.0.13 () {58 vars in 1059 bytes} [Wed Aug 31 09:40:27 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 263 msecs (HTTP/1.1 201) 6 headers in 385 bytes (3 switches on core 0) Aug 31 09:40:27.763518 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-006f9420-1c4c-4172-8995-f130c6287d3e None None] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:40:27.765154 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.fernet_utils [None req-006f9420-1c4c-4172-8995-f130c6287d3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88114) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:40:27.935354 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-006f9420-1c4c-4172-8995-f130c6287d3e None None] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:40:28.017600 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=100 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:40:28.063176 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5763bd5-2485-4f23-b231-86c13ef00a27 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc076493239d4ccdbdc37c0369c49529', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '49548eb44d62415582af820ab66decee', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:40:28.064340 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5763bd5-2485-4f23-b231-86c13ef00a27 service nova] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:40:28.064618 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5763bd5-2485-4f23-b231-86c13ef00a27 service nova] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:40:28.064897 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5763bd5-2485-4f23-b231-86c13ef00a27 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:40:28.066565 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.fernet_utils [None req-c5763bd5-2485-4f23-b231-86c13ef00a27 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88114) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:40:28.172744 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5763bd5-2485-4f23-b231-86c13ef00a27 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:40:28.174657 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5763bd5-2485-4f23-b231-86c13ef00a27 service nova] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:40:28.196733 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 678/1360] 10.222.0.13 () {60 vars in 1365 bytes} [Wed Aug 31 09:40:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 434 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:40:30.510051 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d0c0fe-53e7-4508-9c88-52f28af2cdc1 None None] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:40:30.510539 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d0c0fe-53e7-4508-9c88-52f28af2cdc1 None None] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:40:30.510692 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d0c0fe-53e7-4508-9c88-52f28af2cdc1 None None] PATH_INFO: `/` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:40:30.516471 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 683/1361] 10.222.0.13 () {54 vars in 927 bytes} [Wed Aug 31 09:40:30 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:40:30.518251 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0223fa46-d759-41a7-940f-92857efdfb0f None None] REQUEST_METHOD: `POST` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:40:30.519056 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0223fa46-d759-41a7-940f-92857efdfb0f None None] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:40:30.519493 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0223fa46-d759-41a7-940f-92857efdfb0f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:40:30.613815 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.auth.core [None req-0223fa46-d759-41a7-940f-92857efdfb0f None None] MFA Rules not processed for user `fd836cf590a145df9acf19cf3ae2ac56`. Rule list: `[]` (Enabled: `True`). {{(pid=88114) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:40:30.615089 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.fernet_utils [None req-0223fa46-d759-41a7-940f-92857efdfb0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88114) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:40:30.669726 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=100 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:40:30.757643 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 679/1362] 10.222.0.13 () {58 vars in 1024 bytes} [Wed Aug 31 09:40:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 240 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:40:31.058358 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d0c0fe-53e7-4508-9c88-52f28af2cdc1 None None] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:40:31.060652 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.fernet_utils [None req-90d0c0fe-53e7-4508-9c88-52f28af2cdc1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:40:31.282684 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d0c0fe-53e7-4508-9c88-52f28af2cdc1 None None] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:40:31.294223 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=100 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:40:31.468442 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd5609d8-1350-4614-b73b-2c4ff501c520 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc076493239d4ccdbdc37c0369c49529', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '49548eb44d62415582af820ab66decee', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:40:31.469203 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5609d8-1350-4614-b73b-2c4ff501c520 service nova] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:40:31.469859 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5609d8-1350-4614-b73b-2c4ff501c520 service nova] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:40:31.470433 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5609d8-1350-4614-b73b-2c4ff501c520 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:40:31.472533 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.fernet_utils [None req-dd5609d8-1350-4614-b73b-2c4ff501c520 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:40:31.608276 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd5609d8-1350-4614-b73b-2c4ff501c520 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:40:31.612000 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd5609d8-1350-4614-b73b-2c4ff501c520 service nova] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:40:31.634496 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 684/1363] 10.222.0.13 () {60 vars in 1365 bytes} [Wed Aug 31 09:40:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 577 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:40:33.992281 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62e50ce-19ca-472d-a944-e70bc535559b None None] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:40:33.992978 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62e50ce-19ca-472d-a944-e70bc535559b None None] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:40:33.992978 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62e50ce-19ca-472d-a944-e70bc535559b None None] PATH_INFO: `/` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:40:33.994149 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 680/1364] 10.222.0.13 () {54 vars in 927 bytes} [Wed Aug 31 09:40:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:40:34.000334 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a61cce8-dc92-4abe-a47e-a82448c19d06 None None] REQUEST_METHOD: `POST` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:40:34.001028 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a61cce8-dc92-4abe-a47e-a82448c19d06 None None] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:40:34.001028 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a61cce8-dc92-4abe-a47e-a82448c19d06 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:40:34.018316 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=100 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:40:34.077002 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.auth.core [None req-3a61cce8-dc92-4abe-a47e-a82448c19d06 None None] MFA Rules not processed for user `fd836cf590a145df9acf19cf3ae2ac56`. Rule list: `[]` (Enabled: `True`). {{(pid=88113) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:40:34.078501 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.fernet_utils [None req-3a61cce8-dc92-4abe-a47e-a82448c19d06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:40:34.189473 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 685/1365] 10.222.0.13 () {58 vars in 1024 bytes} [Wed Aug 31 09:40:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 09:40:34.486627 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7f439c-dad8-4c70-8c1f-74f46a6b048d None None] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:40:34.486907 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7f439c-dad8-4c70-8c1f-74f46a6b048d None None] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:40:34.487163 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7f439c-dad8-4c70-8c1f-74f46a6b048d None None] PATH_INFO: `/` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:40:34.488950 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 681/1366] 10.222.0.13 () {54 vars in 962 bytes} [Wed Aug 31 09:40:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:40:34.496028 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d4de50-65df-424b-9423-e60b88230057 None None] REQUEST_METHOD: `POST` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:40:34.496405 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d4de50-65df-424b-9423-e60b88230057 None None] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:40:34.496789 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d4de50-65df-424b-9423-e60b88230057 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:40:34.592469 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.auth.core [None req-a0d4de50-65df-424b-9423-e60b88230057 None None] MFA Rules not processed for user `dc076493239d4ccdbdc37c0369c49529`. Rule list: `[]` (Enabled: `True`). {{(pid=88113) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:40:34.593764 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.fernet_utils [None req-a0d4de50-65df-424b-9423-e60b88230057 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:40:34.637217 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=100 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:40:34.736931 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 686/1367] 10.222.0.13 () {58 vars in 1059 bytes} [Wed Aug 31 09:40:34 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 242 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:40:34.746011 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca7f439c-dad8-4c70-8c1f-74f46a6b048d None None] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:40:34.748314 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.fernet_utils [None req-ca7f439c-dad8-4c70-8c1f-74f46a6b048d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88114) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:40:34.948903 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca7f439c-dad8-4c70-8c1f-74f46a6b048d None None] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:40:34.979434 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=100 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:40:35.088137 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ebcd0d-769a-408a-ae70-6edd54731a4b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc076493239d4ccdbdc37c0369c49529', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '49548eb44d62415582af820ab66decee', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:40:35.088771 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ebcd0d-769a-408a-ae70-6edd54731a4b service nova] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:40:35.089229 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ebcd0d-769a-408a-ae70-6edd54731a4b service nova] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:40:35.089543 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ebcd0d-769a-408a-ae70-6edd54731a4b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:40:35.091277 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.fernet_utils [None req-75ebcd0d-769a-408a-ae70-6edd54731a4b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88114) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:40:35.191308 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75ebcd0d-769a-408a-ae70-6edd54731a4b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:40:35.192923 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75ebcd0d-769a-408a-ae70-6edd54731a4b service nova] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:40:35.213155 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 682/1368] 10.222.0.13 () {60 vars in 1365 bytes} [Wed Aug 31 09:40:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 468 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:40:35.694604 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c8e4d8-04b4-401e-9f58-9f406487ef62 None None] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:40:35.695177 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c8e4d8-04b4-401e-9f58-9f406487ef62 None None] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:40:35.695177 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c8e4d8-04b4-401e-9f58-9f406487ef62 None None] PATH_INFO: `/` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:40:35.696663 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 687/1369] 10.222.0.13 () {54 vars in 927 bytes} [Wed Aug 31 09:40:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:40:35.702045 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fbc8c29-1620-4511-8624-3479d9c2c8ce None None] REQUEST_METHOD: `POST` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:40:35.702338 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fbc8c29-1620-4511-8624-3479d9c2c8ce None None] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:40:35.702617 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fbc8c29-1620-4511-8624-3479d9c2c8ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:40:35.736878 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=100 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:40:35.774762 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.auth.core [None req-6fbc8c29-1620-4511-8624-3479d9c2c8ce None None] MFA Rules not processed for user `fd836cf590a145df9acf19cf3ae2ac56`. Rule list: `[]` (Enabled: `True`). {{(pid=88114) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:40:35.775973 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.fernet_utils [None req-6fbc8c29-1620-4511-8624-3479d9c2c8ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88114) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:40:35.884987 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 683/1370] 10.222.0.13 () {58 vars in 1024 bytes} [Wed Aug 31 09:40:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:40:36.453285 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449700d2-a0a3-4911-95d4-c95ee6770a4d None None] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:40:36.453557 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449700d2-a0a3-4911-95d4-c95ee6770a4d None None] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:40:36.454369 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449700d2-a0a3-4911-95d4-c95ee6770a4d None None] PATH_INFO: `/` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:40:36.455963 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 688/1371] 10.222.0.13 () {54 vars in 964 bytes} [Wed Aug 31 09:40:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:40:36.460871 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2f4255-4a41-4da2-9e22-1bd318272745 None None] REQUEST_METHOD: `POST` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:40:36.461331 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2f4255-4a41-4da2-9e22-1bd318272745 None None] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:40:36.461483 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2f4255-4a41-4da2-9e22-1bd318272745 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:40:36.545324 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.auth.core [None req-2f2f4255-4a41-4da2-9e22-1bd318272745 None None] MFA Rules not processed for user `0d52b69c8e94441dacfcff944326f169`. Rule list: `[]` (Enabled: `True`). {{(pid=88114) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:40:36.546590 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.fernet_utils [None req-2f2f4255-4a41-4da2-9e22-1bd318272745 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88114) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:40:36.603396 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=100 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:40:36.634563 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 684/1372] 10.222.0.13 () {58 vars in 1061 bytes} [Wed Aug 31 09:40:36 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:40:36.642263 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-449700d2-a0a3-4911-95d4-c95ee6770a4d None None] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:40:36.643759 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.fernet_utils [None req-449700d2-a0a3-4911-95d4-c95ee6770a4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:40:36.759014 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-449700d2-a0a3-4911-95d4-c95ee6770a4d None None] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:40:36.804686 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=100 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:40:36.844119 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c0be141-90b6-49ed-b5cb-974ca91a6258 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d52b69c8e94441dacfcff944326f169', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '49548eb44d62415582af820ab66decee', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:40:36.845951 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0be141-90b6-49ed-b5cb-974ca91a6258 service cinder] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:40:36.846300 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0be141-90b6-49ed-b5cb-974ca91a6258 service cinder] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:40:36.846646 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0be141-90b6-49ed-b5cb-974ca91a6258 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:40:36.848573 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.fernet_utils [None req-6c0be141-90b6-49ed-b5cb-974ca91a6258 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:40:36.997088 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c0be141-90b6-49ed-b5cb-974ca91a6258 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:40:36.998594 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c0be141-90b6-49ed-b5cb-974ca91a6258 service cinder] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:40:37.024341 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 689/1373] 10.222.0.13 () {60 vars in 1365 bytes} [Wed Aug 31 09:40:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 380 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:40:37.272098 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4a00ad-e583-4f15-812f-c7e12f65c5db None None] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:40:37.272311 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4a00ad-e583-4f15-812f-c7e12f65c5db None None] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:40:37.272573 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4a00ad-e583-4f15-812f-c7e12f65c5db None None] PATH_INFO: `/` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:40:37.274016 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 685/1374] 10.222.0.13 () {54 vars in 927 bytes} [Wed Aug 31 09:40:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 31 09:40:37.279359 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d135725-d462-49d5-a36f-125551053c48 None None] REQUEST_METHOD: `POST` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:40:37.279653 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d135725-d462-49d5-a36f-125551053c48 None None] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:40:37.280043 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d135725-d462-49d5-a36f-125551053c48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:40:37.356293 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.auth.core [None req-1d135725-d462-49d5-a36f-125551053c48 None None] MFA Rules not processed for user `fd836cf590a145df9acf19cf3ae2ac56`. Rule list: `[]` (Enabled: `True`). {{(pid=88113) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:40:37.357687 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.fernet_utils [None req-1d135725-d462-49d5-a36f-125551053c48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:40:37.381628 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=100 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:40:37.465910 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 690/1375] 10.222.0.13 () {58 vars in 1024 bytes} [Wed Aug 31 09:40:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:40:37.762249 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca4a00ad-e583-4f15-812f-c7e12f65c5db None None] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:40:37.763682 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.fernet_utils [None req-ca4a00ad-e583-4f15-812f-c7e12f65c5db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88114) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:40:37.932301 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca4a00ad-e583-4f15-812f-c7e12f65c5db None None] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:40:37.989077 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=100 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:40:38.072553 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de51320-20bf-4b6c-ae06-e869d96c14eb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc076493239d4ccdbdc37c0369c49529', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '49548eb44d62415582af820ab66decee', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:40:38.073633 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de51320-20bf-4b6c-ae06-e869d96c14eb service nova] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:40:38.074109 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de51320-20bf-4b6c-ae06-e869d96c14eb service nova] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:40:38.074594 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de51320-20bf-4b6c-ae06-e869d96c14eb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:40:38.078489 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.fernet_utils [None req-7de51320-20bf-4b6c-ae06-e869d96c14eb service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88114) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:40:38.196027 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7de51320-20bf-4b6c-ae06-e869d96c14eb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:40:38.197537 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7de51320-20bf-4b6c-ae06-e869d96c14eb service nova] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:40:38.215861 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 686/1376] 10.222.0.13 () {60 vars in 1365 bytes} [Wed Aug 31 09:40:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 456 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:40:40.220948 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee7b753-7a42-4a43-9cc1-a155b2da5e2c None None] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:40:40.221521 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee7b753-7a42-4a43-9cc1-a155b2da5e2c None None] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:40:40.221521 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee7b753-7a42-4a43-9cc1-a155b2da5e2c None None] PATH_INFO: `/` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:40:40.223215 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 691/1377] 10.222.0.13 () {54 vars in 927 bytes} [Wed Aug 31 09:40:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:40:40.228535 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d9879a-c88b-48d1-a018-b73009f8ed21 None None] REQUEST_METHOD: `POST` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:40:40.228995 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d9879a-c88b-48d1-a018-b73009f8ed21 None None] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:40:40.228995 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d9879a-c88b-48d1-a018-b73009f8ed21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:40:40.322026 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.auth.core [None req-92d9879a-c88b-48d1-a018-b73009f8ed21 None None] MFA Rules not processed for user `fd836cf590a145df9acf19cf3ae2ac56`. Rule list: `[]` (Enabled: `True`). {{(pid=88114) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:40:40.323157 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.fernet_utils [None req-92d9879a-c88b-48d1-a018-b73009f8ed21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88114) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:40:40.330893 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=100 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:40:40.427964 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 687/1378] 10.222.0.13 () {58 vars in 1024 bytes} [Wed Aug 31 09:40:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:40:40.741327 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee7b753-7a42-4a43-9cc1-a155b2da5e2c None None] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:40:40.742993 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.fernet_utils [None req-8ee7b753-7a42-4a43-9cc1-a155b2da5e2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:40:40.933102 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee7b753-7a42-4a43-9cc1-a155b2da5e2c None None] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:40:40.937816 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=100 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:40:41.079806 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da61564-8288-4127-a46b-585125f82c2b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc076493239d4ccdbdc37c0369c49529', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '49548eb44d62415582af820ab66decee', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:40:41.080957 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da61564-8288-4127-a46b-585125f82c2b service nova] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:40:41.081523 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da61564-8288-4127-a46b-585125f82c2b service nova] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:40:41.081977 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da61564-8288-4127-a46b-585125f82c2b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:40:41.084008 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.fernet_utils [None req-6da61564-8288-4127-a46b-585125f82c2b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:40:41.189026 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6da61564-8288-4127-a46b-585125f82c2b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:40:41.190895 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6da61564-8288-4127-a46b-585125f82c2b service nova] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:40:41.211815 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 692/1379] 10.222.0.13 () {60 vars in 1365 bytes} [Wed Aug 31 09:40:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 471 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 09:40:43.278153 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89dc0cd0-fa9b-4d32-a3b9-eed2bced6a2a None None] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:40:43.278737 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89dc0cd0-fa9b-4d32-a3b9-eed2bced6a2a None None] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:40:43.278737 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89dc0cd0-fa9b-4d32-a3b9-eed2bced6a2a None None] PATH_INFO: `/` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:40:43.280213 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 688/1380] 10.222.0.13 () {54 vars in 927 bytes} [Wed Aug 31 09:40:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 31 09:40:43.285813 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a533446-0aef-4519-a10c-fb14f669a761 None None] REQUEST_METHOD: `POST` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:40:43.286341 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a533446-0aef-4519-a10c-fb14f669a761 None None] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:40:43.286341 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a533446-0aef-4519-a10c-fb14f669a761 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:40:43.298266 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=100 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:40:43.368791 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.auth.core [None req-9a533446-0aef-4519-a10c-fb14f669a761 None None] MFA Rules not processed for user `fd836cf590a145df9acf19cf3ae2ac56`. Rule list: `[]` (Enabled: `True`). {{(pid=88113) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:40:43.370037 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.fernet_utils [None req-9a533446-0aef-4519-a10c-fb14f669a761 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:40:43.497432 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 693/1381] 10.222.0.13 () {58 vars in 1024 bytes} [Wed Aug 31 09:40:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:40:43.868278 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89dc0cd0-fa9b-4d32-a3b9-eed2bced6a2a None None] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:40:43.869944 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.fernet_utils [None req-89dc0cd0-fa9b-4d32-a3b9-eed2bced6a2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88114) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:40:44.043361 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=100 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:40:44.055880 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89dc0cd0-fa9b-4d32-a3b9-eed2bced6a2a None None] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:40:44.201553 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b964fb-bd55-402c-807e-5ab72e7f91e4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc076493239d4ccdbdc37c0369c49529', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '49548eb44d62415582af820ab66decee', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:40:44.202162 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b964fb-bd55-402c-807e-5ab72e7f91e4 service nova] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:40:44.202435 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b964fb-bd55-402c-807e-5ab72e7f91e4 service nova] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:40:44.202701 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b964fb-bd55-402c-807e-5ab72e7f91e4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:40:44.204324 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.fernet_utils [None req-89b964fb-bd55-402c-807e-5ab72e7f91e4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88114) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:40:44.307407 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89b964fb-bd55-402c-807e-5ab72e7f91e4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:40:44.309105 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89b964fb-bd55-402c-807e-5ab72e7f91e4 service nova] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:40:44.320577 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=100 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:40:44.330676 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 689/1382] 10.222.0.13 () {60 vars in 1365 bytes} [Wed Aug 31 09:40:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 462 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:40:46.435668 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b825aec-fc2c-4b6d-9e7a-904cf5d2dce5 None None] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:40:46.436484 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b825aec-fc2c-4b6d-9e7a-904cf5d2dce5 None None] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:40:46.437083 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b825aec-fc2c-4b6d-9e7a-904cf5d2dce5 None None] PATH_INFO: `/` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:40:46.440023 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 694/1383] 10.222.0.13 () {54 vars in 927 bytes} [Wed Aug 31 09:40:46 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:40:46.446634 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b3e7dc-a4f2-4b4c-bfd1-f2201ac09269 None None] REQUEST_METHOD: `POST` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:40:46.447220 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b3e7dc-a4f2-4b4c-bfd1-f2201ac09269 None None] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:40:46.447220 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b3e7dc-a4f2-4b4c-bfd1-f2201ac09269 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:40:46.538044 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.auth.core [None req-97b3e7dc-a4f2-4b4c-bfd1-f2201ac09269 None None] MFA Rules not processed for user `fd836cf590a145df9acf19cf3ae2ac56`. Rule list: `[]` (Enabled: `True`). {{(pid=88114) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:40:46.539313 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.fernet_utils [None req-97b3e7dc-a4f2-4b4c-bfd1-f2201ac09269 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88114) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:40:46.653890 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 690/1384] 10.222.0.13 () {58 vars in 1024 bytes} [Wed Aug 31 09:40:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 208 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 09:40:46.959197 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b825aec-fc2c-4b6d-9e7a-904cf5d2dce5 None None] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:40:46.962052 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.fernet_utils [None req-0b825aec-fc2c-4b6d-9e7a-904cf5d2dce5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:40:47.062006 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=100 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:40:47.144697 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b825aec-fc2c-4b6d-9e7a-904cf5d2dce5 None None] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:40:47.299435 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e6b0b55-cfc6-41e4-a493-999ff89d3a9d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc076493239d4ccdbdc37c0369c49529', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '49548eb44d62415582af820ab66decee', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:40:47.300203 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6b0b55-cfc6-41e4-a493-999ff89d3a9d service nova] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:40:47.300536 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6b0b55-cfc6-41e4-a493-999ff89d3a9d service nova] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:40:47.300887 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6b0b55-cfc6-41e4-a493-999ff89d3a9d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:40:47.302658 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.fernet_utils [None req-0e6b0b55-cfc6-41e4-a493-999ff89d3a9d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:40:47.368325 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=100 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:40:47.421367 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e6b0b55-cfc6-41e4-a493-999ff89d3a9d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:40:47.423164 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e6b0b55-cfc6-41e4-a493-999ff89d3a9d service nova] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:40:47.447070 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 695/1385] 10.222.0.13 () {60 vars in 1365 bytes} [Wed Aug 31 09:40:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 490 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:40:49.489504 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7da540b-06c6-4108-9b2c-04434c3d4d0c None None] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:40:49.490303 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7da540b-06c6-4108-9b2c-04434c3d4d0c None None] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:40:49.490303 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7da540b-06c6-4108-9b2c-04434c3d4d0c None None] PATH_INFO: `/` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:40:49.491439 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 691/1386] 10.222.0.13 () {54 vars in 927 bytes} [Wed Aug 31 09:40:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:40:49.497146 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0089d0c3-af4f-4342-8ddb-8f8d4510cee7 None None] REQUEST_METHOD: `POST` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:40:49.497948 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0089d0c3-af4f-4342-8ddb-8f8d4510cee7 None None] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:40:49.497948 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0089d0c3-af4f-4342-8ddb-8f8d4510cee7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:40:49.589141 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.auth.core [None req-0089d0c3-af4f-4342-8ddb-8f8d4510cee7 None None] MFA Rules not processed for user `fd836cf590a145df9acf19cf3ae2ac56`. Rule list: `[]` (Enabled: `True`). {{(pid=88113) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:40:49.590616 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.fernet_utils [None req-0089d0c3-af4f-4342-8ddb-8f8d4510cee7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:40:49.705948 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 696/1387] 10.222.0.13 () {58 vars in 1024 bytes} [Wed Aug 31 09:40:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:40:50.092640 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7da540b-06c6-4108-9b2c-04434c3d4d0c None None] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:40:50.094995 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.fernet_utils [None req-f7da540b-06c6-4108-9b2c-04434c3d4d0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88114) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:40:50.186843 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=100 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:40:50.275008 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7da540b-06c6-4108-9b2c-04434c3d4d0c None None] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:40:50.445089 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beab6bf3-81bf-4fae-9c34-360e37a2cc1e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc076493239d4ccdbdc37c0369c49529', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '49548eb44d62415582af820ab66decee', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:40:50.445898 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beab6bf3-81bf-4fae-9c34-360e37a2cc1e service nova] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:40:50.446179 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beab6bf3-81bf-4fae-9c34-360e37a2cc1e service nova] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:40:50.446457 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beab6bf3-81bf-4fae-9c34-360e37a2cc1e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:40:50.448219 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.fernet_utils [None req-beab6bf3-81bf-4fae-9c34-360e37a2cc1e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88114) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:40:50.496866 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=100 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:40:50.559747 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beab6bf3-81bf-4fae-9c34-360e37a2cc1e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:40:50.561334 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beab6bf3-81bf-4fae-9c34-360e37a2cc1e service nova] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:40:50.584550 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 692/1388] 10.222.0.13 () {60 vars in 1365 bytes} [Wed Aug 31 09:40:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 492 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:40:52.867987 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fae2493-99e8-4cc9-a2db-d5d13d08bea0 None None] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:40:52.868565 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fae2493-99e8-4cc9-a2db-d5d13d08bea0 None None] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:40:52.868565 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fae2493-99e8-4cc9-a2db-d5d13d08bea0 None None] PATH_INFO: `/` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:40:52.870001 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 697/1389] 10.222.0.13 () {54 vars in 927 bytes} [Wed Aug 31 09:40:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:40:52.876274 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9330c80a-7eee-40ad-bcbe-f58b8cf6e592 None None] REQUEST_METHOD: `POST` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:40:52.876784 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9330c80a-7eee-40ad-bcbe-f58b8cf6e592 None None] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:40:52.876877 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9330c80a-7eee-40ad-bcbe-f58b8cf6e592 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:40:52.959826 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.auth.core [None req-9330c80a-7eee-40ad-bcbe-f58b8cf6e592 None None] MFA Rules not processed for user `fd836cf590a145df9acf19cf3ae2ac56`. Rule list: `[]` (Enabled: `True`). {{(pid=88114) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:40:52.961092 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.fernet_utils [None req-9330c80a-7eee-40ad-bcbe-f58b8cf6e592 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88114) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:40:53.071208 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 693/1390] 10.222.0.13 () {58 vars in 1024 bytes} [Wed Aug 31 09:40:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:40:53.370626 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fae2493-99e8-4cc9-a2db-d5d13d08bea0 None None] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:40:53.373533 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.fernet_utils [None req-0fae2493-99e8-4cc9-a2db-d5d13d08bea0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:40:53.417987 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=100 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:40:53.580086 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fae2493-99e8-4cc9-a2db-d5d13d08bea0 None None] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:40:53.707822 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=100 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:40:53.723944 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e30a36a6-435a-4ed8-8759-fd9bb3fbbfab service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc076493239d4ccdbdc37c0369c49529', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '49548eb44d62415582af820ab66decee', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:40:53.724604 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30a36a6-435a-4ed8-8759-fd9bb3fbbfab service nova] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:40:53.724891 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30a36a6-435a-4ed8-8759-fd9bb3fbbfab service nova] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:40:53.725190 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30a36a6-435a-4ed8-8759-fd9bb3fbbfab service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:40:53.726898 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.fernet_utils [None req-e30a36a6-435a-4ed8-8759-fd9bb3fbbfab service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:40:53.834518 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e30a36a6-435a-4ed8-8759-fd9bb3fbbfab service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:40:53.836212 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e30a36a6-435a-4ed8-8759-fd9bb3fbbfab service nova] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:40:53.857594 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 698/1391] 10.222.0.13 () {60 vars in 1365 bytes} [Wed Aug 31 09:40:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 488 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:40:56.310940 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f06b2cc-c278-4d7e-bc65-c6c109b8c04d None None] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:40:56.311517 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f06b2cc-c278-4d7e-bc65-c6c109b8c04d None None] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:40:56.311517 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f06b2cc-c278-4d7e-bc65-c6c109b8c04d None None] PATH_INFO: `/` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:40:56.313327 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 694/1392] 10.222.0.13 () {54 vars in 927 bytes} [Wed Aug 31 09:40:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:40:56.319254 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e2fbca-43f2-4cee-ba4b-0ce2f3a10fca None None] REQUEST_METHOD: `POST` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:40:56.319779 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e2fbca-43f2-4cee-ba4b-0ce2f3a10fca None None] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:40:56.319779 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e2fbca-43f2-4cee-ba4b-0ce2f3a10fca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:40:56.395492 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.auth.core [None req-e7e2fbca-43f2-4cee-ba4b-0ce2f3a10fca None None] MFA Rules not processed for user `fd836cf590a145df9acf19cf3ae2ac56`. Rule list: `[]` (Enabled: `True`). {{(pid=88113) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:40:56.396747 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.fernet_utils [None req-e7e2fbca-43f2-4cee-ba4b-0ce2f3a10fca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:40:56.498055 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=100 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:40:56.540410 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 699/1393] 10.222.0.13 () {58 vars in 1024 bytes} [Wed Aug 31 09:40:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 222 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:40:56.843144 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f06b2cc-c278-4d7e-bc65-c6c109b8c04d None None] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:40:56.843144 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.fernet_utils [None req-6f06b2cc-c278-4d7e-bc65-c6c109b8c04d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88114) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:40:56.858010 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=100 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:40:57.004967 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f06b2cc-c278-4d7e-bc65-c6c109b8c04d None None] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:40:57.096795 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=100 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:40:57.124029 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-942fd524-6bd5-4c7b-a022-35df8b4f8239 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc076493239d4ccdbdc37c0369c49529', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '49548eb44d62415582af820ab66decee', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:40:57.124643 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942fd524-6bd5-4c7b-a022-35df8b4f8239 service nova] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:40:57.124924 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942fd524-6bd5-4c7b-a022-35df8b4f8239 service nova] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:40:57.125275 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942fd524-6bd5-4c7b-a022-35df8b4f8239 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:40:57.126914 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.fernet_utils [None req-942fd524-6bd5-4c7b-a022-35df8b4f8239 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88114) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:40:57.231364 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-942fd524-6bd5-4c7b-a022-35df8b4f8239 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:40:57.233123 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-942fd524-6bd5-4c7b-a022-35df8b4f8239 service nova] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:40:57.254205 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 695/1394] 10.222.0.13 () {60 vars in 1365 bytes} [Wed Aug 31 09:40:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 413 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:40:57.941221 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c70ff6-7f8a-483e-aa88-05db7e72d7c6 None None] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:40:57.941847 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c70ff6-7f8a-483e-aa88-05db7e72d7c6 None None] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:40:57.941847 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c70ff6-7f8a-483e-aa88-05db7e72d7c6 None None] PATH_INFO: `/` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:40:57.943165 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 700/1395] 10.222.0.13 () {54 vars in 927 bytes} [Wed Aug 31 09:40:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:40:57.948689 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef16138-8092-4b8e-bf2c-3cf144e23b52 None None] REQUEST_METHOD: `POST` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:40:57.949199 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef16138-8092-4b8e-bf2c-3cf144e23b52 None None] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:40:57.949296 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef16138-8092-4b8e-bf2c-3cf144e23b52 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:40:58.029008 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.auth.core [None req-8ef16138-8092-4b8e-bf2c-3cf144e23b52 None None] MFA Rules not processed for user `fd836cf590a145df9acf19cf3ae2ac56`. Rule list: `[]` (Enabled: `True`). {{(pid=88114) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:40:58.030243 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.fernet_utils [None req-8ef16138-8092-4b8e-bf2c-3cf144e23b52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88114) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:40:58.091303 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=100 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:40:58.148550 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 696/1396] 10.222.0.13 () {58 vars in 1024 bytes} [Wed Aug 31 09:40:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 09:40:58.470036 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b609310b-6a89-425c-b4bd-40a5891eac7c None None] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:40:58.470322 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b609310b-6a89-425c-b4bd-40a5891eac7c None None] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:40:58.470592 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b609310b-6a89-425c-b4bd-40a5891eac7c None None] PATH_INFO: `/` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:40:58.471958 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 701/1397] 10.222.0.13 () {54 vars in 967 bytes} [Wed Aug 31 09:40:58 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:40:58.477371 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2dafce-a155-45ae-ac34-a1b58fc8295f None None] REQUEST_METHOD: `POST` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:40:58.477726 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2dafce-a155-45ae-ac34-a1b58fc8295f None None] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:40:58.478028 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2dafce-a155-45ae-ac34-a1b58fc8295f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:40:58.566676 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.auth.core [None req-4d2dafce-a155-45ae-ac34-a1b58fc8295f None None] MFA Rules not processed for user `be216d156b54452cbbf978c7734343aa`. Rule list: `[]` (Enabled: `True`). {{(pid=88114) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:40:58.567989 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.fernet_utils [None req-4d2dafce-a155-45ae-ac34-a1b58fc8295f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88114) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:40:58.654770 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 697/1398] 10.222.0.13 () {58 vars in 1064 bytes} [Wed Aug 31 09:40:58 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:40:58.664352 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b609310b-6a89-425c-b4bd-40a5891eac7c None None] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:40:58.665997 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.fernet_utils [None req-b609310b-6a89-425c-b4bd-40a5891eac7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:40:58.786894 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b609310b-6a89-425c-b4bd-40a5891eac7c None None] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:40:58.901103 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-086c346c-dffa-450f-8ba2-98ec21f1f894 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be216d156b54452cbbf978c7734343aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '49548eb44d62415582af820ab66decee', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:40:58.901857 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086c346c-dffa-450f-8ba2-98ec21f1f894 service glance] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:40:58.902106 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086c346c-dffa-450f-8ba2-98ec21f1f894 service glance] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:40:58.902375 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086c346c-dffa-450f-8ba2-98ec21f1f894 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:40:58.904061 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.fernet_utils [None req-086c346c-dffa-450f-8ba2-98ec21f1f894 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:40:58.926376 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=100 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:40:59.029328 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-086c346c-dffa-450f-8ba2-98ec21f1f894 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:40:59.030845 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-086c346c-dffa-450f-8ba2-98ec21f1f894 service glance] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:40:59.052550 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 702/1399] 10.222.0.13 () {60 vars in 1365 bytes} [Wed Aug 31 09:40:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 388 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:40:59.305560 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5828fa-edcb-4db9-80bd-b2af2d525d28 None None] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:40:59.306186 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5828fa-edcb-4db9-80bd-b2af2d525d28 None None] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:40:59.306388 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5828fa-edcb-4db9-80bd-b2af2d525d28 None None] PATH_INFO: `/` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:40:59.309061 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 698/1400] 10.222.0.13 () {54 vars in 927 bytes} [Wed Aug 31 09:40:59 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:40:59.317348 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f99bc5-1de8-4d77-b61f-ca9e49da9c1b None None] REQUEST_METHOD: `POST` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:40:59.317977 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f99bc5-1de8-4d77-b61f-ca9e49da9c1b None None] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:40:59.318433 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f99bc5-1de8-4d77-b61f-ca9e49da9c1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:40:59.450306 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.auth.core [None req-b5f99bc5-1de8-4d77-b61f-ca9e49da9c1b None None] MFA Rules not processed for user `fd836cf590a145df9acf19cf3ae2ac56`. Rule list: `[]` (Enabled: `True`). {{(pid=88113) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:40:59.452357 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.fernet_utils [None req-b5f99bc5-1de8-4d77-b61f-ca9e49da9c1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:40:59.546779 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=100 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:40:59.603535 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 703/1401] 10.222.0.13 () {58 vars in 1024 bytes} [Wed Aug 31 09:40:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 288 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:40:59.928292 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8c1df6-fdbe-4bd5-a34a-82cd7b6bd045 None None] REQUEST_METHOD: `POST` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:40:59.928610 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8c1df6-fdbe-4bd5-a34a-82cd7b6bd045 None None] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:40:59.928826 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8c1df6-fdbe-4bd5-a34a-82cd7b6bd045 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:40:59.934834 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f99bc5-1de8-4d77-b61f-ca9e49da9c1b None None] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:40:59.937010 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.fernet_utils [None req-b5f99bc5-1de8-4d77-b61f-ca9e49da9c1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:41:00.000717 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=100 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:41:00.011315 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.auth.core [None req-ba8c1df6-fdbe-4bd5-a34a-82cd7b6bd045 None None] MFA Rules not processed for user `768cb67c1ef24251aaa113c12d81a250`. Rule list: `[]` (Enabled: `True`). {{(pid=88114) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:41:00.012556 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.fernet_utils [None req-ba8c1df6-fdbe-4bd5-a34a-82cd7b6bd045 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88114) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:41:00.101968 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f99bc5-1de8-4d77-b61f-ca9e49da9c1b None None] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:41:00.151360 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 699/1402] 10.222.0.13 () {58 vars in 1009 bytes} [Wed Aug 31 09:40:59 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 224 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:41:00.211241 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d030bc-a260-401d-8603-432a934e2e0f None None] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:41:00.211694 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d030bc-a260-401d-8603-432a934e2e0f None None] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:41:00.212153 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d030bc-a260-401d-8603-432a934e2e0f None None] PATH_INFO: `/` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:41:00.214441 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=100 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:41:00.214641 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 700/1403] 10.222.0.13 () {54 vars in 943 bytes} [Wed Aug 31 09:41:00 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 31 09:41:00.223626 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2597895-5dbe-40bc-b542-95bff7669e23 None None] REQUEST_METHOD: `POST` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:41:00.224079 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2597895-5dbe-40bc-b542-95bff7669e23 None None] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:41:00.224716 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2597895-5dbe-40bc-b542-95bff7669e23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:41:00.263412 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e7e1c3c-27fc-49c7-b4cd-c7b98b3b8a71 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc076493239d4ccdbdc37c0369c49529', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '49548eb44d62415582af820ab66decee', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:41:00.264577 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7e1c3c-27fc-49c7-b4cd-c7b98b3b8a71 service nova] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:41:00.265203 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7e1c3c-27fc-49c7-b4cd-c7b98b3b8a71 service nova] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:41:00.266196 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7e1c3c-27fc-49c7-b4cd-c7b98b3b8a71 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:41:00.269062 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.fernet_utils [None req-5e7e1c3c-27fc-49c7-b4cd-c7b98b3b8a71 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:41:00.342354 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.auth.core [None req-b2597895-5dbe-40bc-b542-95bff7669e23 None None] MFA Rules not processed for user `202bb9db7a8a4977ba788a290ed9a3fa`. Rule list: `[]` (Enabled: `True`). {{(pid=88114) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:41:00.343726 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.fernet_utils [None req-b2597895-5dbe-40bc-b542-95bff7669e23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88114) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:41:00.405700 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e7e1c3c-27fc-49c7-b4cd-c7b98b3b8a71 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:41:00.408192 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e7e1c3c-27fc-49c7-b4cd-c7b98b3b8a71 service nova] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:41:00.432922 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 704/1404] 10.222.0.13 () {60 vars in 1365 bytes} [Wed Aug 31 09:40:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 499 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 09:41:00.440399 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=100 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:41:00.487151 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 701/1405] 10.222.0.13 () {58 vars in 1040 bytes} [Wed Aug 31 09:41:00 2022] POST /identity/v3/auth/tokens => generated 3304 bytes in 266 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 09:41:00.498006 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e7e1c3c-27fc-49c7-b4cd-c7b98b3b8a71 service nova] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:41:00.501063 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.fernet_utils [None req-5e7e1c3c-27fc-49c7-b4cd-c7b98b3b8a71 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:41:00.633154 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e7e1c3c-27fc-49c7-b4cd-c7b98b3b8a71 service nova] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:41:00.650510 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=100 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:41:00.803025 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac552e2-f7d9-4882-9e83-67e35fa4efb0 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '202bb9db7a8a4977ba788a290ed9a3fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '49548eb44d62415582af820ab66decee', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:41:00.803710 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac552e2-f7d9-4882-9e83-67e35fa4efb0 service swift] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:41:00.804092 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac552e2-f7d9-4882-9e83-67e35fa4efb0 service swift] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:41:00.804540 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac552e2-f7d9-4882-9e83-67e35fa4efb0 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:41:00.806505 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.fernet_utils [None req-fac552e2-f7d9-4882-9e83-67e35fa4efb0 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:41:00.936374 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fac552e2-f7d9-4882-9e83-67e35fa4efb0 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:41:00.938472 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fac552e2-f7d9-4882-9e83-67e35fa4efb0 service swift] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:41:00.947397 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 705/1406] 10.222.0.13 () {60 vars in 1384 bytes} [Wed Aug 31 09:41:00 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 451 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 31 09:41:02.790627 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b730275-2485-46e1-945d-18143ce7c722 None None] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:41:02.791434 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b730275-2485-46e1-945d-18143ce7c722 None None] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:41:02.791434 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b730275-2485-46e1-945d-18143ce7c722 None None] PATH_INFO: `/` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:41:02.792761 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 702/1407] 10.222.0.13 () {54 vars in 927 bytes} [Wed Aug 31 09:41:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:41:02.798756 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ac9d50-614f-4718-a9b7-2592973779f5 None None] REQUEST_METHOD: `POST` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:41:02.799514 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ac9d50-614f-4718-a9b7-2592973779f5 None None] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:41:02.799514 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ac9d50-614f-4718-a9b7-2592973779f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:41:02.879386 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=100 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:41:02.907725 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.auth.core [None req-a2ac9d50-614f-4718-a9b7-2592973779f5 None None] MFA Rules not processed for user `fd836cf590a145df9acf19cf3ae2ac56`. Rule list: `[]` (Enabled: `True`). {{(pid=88113) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:41:02.909648 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.fernet_utils [None req-a2ac9d50-614f-4718-a9b7-2592973779f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:41:03.112476 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 706/1408] 10.222.0.13 () {58 vars in 1024 bytes} [Wed Aug 31 09:41:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 315 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:41:03.533891 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b730275-2485-46e1-945d-18143ce7c722 None None] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:41:03.535509 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.fernet_utils [None req-4b730275-2485-46e1-945d-18143ce7c722 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88114) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:41:03.747896 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b730275-2485-46e1-945d-18143ce7c722 None None] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:41:03.812219 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=100 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:41:03.911686 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc37f24-ea10-43fd-ad2c-d245cf7c7cbf service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc076493239d4ccdbdc37c0369c49529', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '49548eb44d62415582af820ab66decee', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:41:03.912979 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc37f24-ea10-43fd-ad2c-d245cf7c7cbf service nova] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:41:03.913451 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc37f24-ea10-43fd-ad2c-d245cf7c7cbf service nova] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:41:03.913927 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc37f24-ea10-43fd-ad2c-d245cf7c7cbf service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:41:03.916626 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.fernet_utils [None req-abc37f24-ea10-43fd-ad2c-d245cf7c7cbf service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88114) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:41:04.038445 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abc37f24-ea10-43fd-ad2c-d245cf7c7cbf service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:41:04.039994 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abc37f24-ea10-43fd-ad2c-d245cf7c7cbf service nova] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:41:04.073652 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 703/1409] 10.222.0.13 () {60 vars in 1365 bytes} [Wed Aug 31 09:41:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 540 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:41:13.109193 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=46 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:41:14.059325 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=76 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:41:25.064527 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3fa8cde-3076-44e4-8efb-6cb61b96950b None None] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:41:25.065133 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3fa8cde-3076-44e4-8efb-6cb61b96950b None None] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:41:25.065133 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3fa8cde-3076-44e4-8efb-6cb61b96950b None None] PATH_INFO: `/` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:41:25.066492 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 707/1410] 10.222.0.13 () {54 vars in 918 bytes} [Wed Aug 31 09:41:25 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:41:25.072362 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13f199b-d27e-403d-85a8-f8bd28172c47 None None] REQUEST_METHOD: `POST` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:41:25.072896 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13f199b-d27e-403d-85a8-f8bd28172c47 None None] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:41:25.072896 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13f199b-d27e-403d-85a8-f8bd28172c47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:41:25.156377 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.auth.core [None req-f13f199b-d27e-403d-85a8-f8bd28172c47 None None] MFA Rules not processed for user `1ff3aff5ef444910be6dbb989d0e3194`. Rule list: `[]` (Enabled: `True`). {{(pid=88114) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:41:25.157616 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.fernet_utils [None req-f13f199b-d27e-403d-85a8-f8bd28172c47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88114) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:41:25.270822 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 704/1411] 10.222.0.13 () {58 vars in 1015 bytes} [Wed Aug 31 09:41:25 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:41:25.286755 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2968efc-3e2b-4d16-91c6-67ee74f63566 None None] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:41:25.287021 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2968efc-3e2b-4d16-91c6-67ee74f63566 None None] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:41:25.287291 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2968efc-3e2b-4d16-91c6-67ee74f63566 None None] PATH_INFO: `/` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:41:25.288660 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 708/1412] 10.222.0.13 () {54 vars in 918 bytes} [Wed Aug 31 09:41:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:41:25.293454 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20af2f52-784a-415d-80c0-f1cb9759a56e None None] REQUEST_METHOD: `POST` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:41:25.293749 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20af2f52-784a-415d-80c0-f1cb9759a56e None None] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:41:25.294024 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20af2f52-784a-415d-80c0-f1cb9759a56e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:41:25.370326 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=100 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:41:25.373526 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.auth.core [None req-20af2f52-784a-415d-80c0-f1cb9759a56e None None] MFA Rules not processed for user `dc076493239d4ccdbdc37c0369c49529`. Rule list: `[]` (Enabled: `True`). {{(pid=88114) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:41:25.376434 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.fernet_utils [None req-20af2f52-784a-415d-80c0-f1cb9759a56e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88114) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:41:25.495483 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 705/1413] 10.222.0.13 () {58 vars in 1015 bytes} [Wed Aug 31 09:41:25 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:41:27.982991 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d9b0dc-5159-480e-a66e-b00d55e15133 None None] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:41:27.984338 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d9b0dc-5159-480e-a66e-b00d55e15133 None None] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:41:27.984338 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d9b0dc-5159-480e-a66e-b00d55e15133 None None] PATH_INFO: `/` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:41:27.985872 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 709/1414] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 31 09:41:27 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 31 09:41:27.991474 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43638956-46d4-4d31-8443-da9a86b025ac None None] REQUEST_METHOD: `POST` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:41:27.992006 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43638956-46d4-4d31-8443-da9a86b025ac None None] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:41:27.992006 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43638956-46d4-4d31-8443-da9a86b025ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:41:28.074580 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.auth.core [None req-43638956-46d4-4d31-8443-da9a86b025ac None None] MFA Rules not processed for user `fd836cf590a145df9acf19cf3ae2ac56`. Rule list: `[]` (Enabled: `True`). {{(pid=88114) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:41:28.075816 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.fernet_utils [None req-43638956-46d4-4d31-8443-da9a86b025ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88114) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:41:28.137871 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=100 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:41:28.188197 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 706/1415] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 31 09:41:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 09:41:28.482926 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d9b0dc-5159-480e-a66e-b00d55e15133 None None] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:41:28.484888 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.fernet_utils [None req-e7d9b0dc-5159-480e-a66e-b00d55e15133 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:41:28.658189 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d9b0dc-5159-480e-a66e-b00d55e15133 None None] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:41:28.741779 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=100 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:41:28.788303 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc8531a0-ea9f-4be0-ac0a-c1b99cfc9e69 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc076493239d4ccdbdc37c0369c49529', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '49548eb44d62415582af820ab66decee', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:41:28.788918 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8531a0-ea9f-4be0-ac0a-c1b99cfc9e69 service nova] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:41:28.789198 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8531a0-ea9f-4be0-ac0a-c1b99cfc9e69 service nova] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:41:28.789503 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8531a0-ea9f-4be0-ac0a-c1b99cfc9e69 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:41:28.791140 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.fernet_utils [None req-dc8531a0-ea9f-4be0-ac0a-c1b99cfc9e69 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:41:28.891959 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc8531a0-ea9f-4be0-ac0a-c1b99cfc9e69 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:41:28.894370 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc8531a0-ea9f-4be0-ac0a-c1b99cfc9e69 service nova] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:41:28.918074 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 710/1416] 10.222.0.13 () {60 vars in 1365 bytes} [Wed Aug 31 09:41:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 436 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:41:29.880246 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48aa7326-dbb2-4e81-95e9-fe826b432658 None None] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:41:29.880865 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48aa7326-dbb2-4e81-95e9-fe826b432658 None None] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:41:29.880865 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48aa7326-dbb2-4e81-95e9-fe826b432658 None None] PATH_INFO: `/` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:41:29.882900 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 707/1417] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 31 09:41:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:41:29.889490 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2505369d-3302-4dad-afff-d3d806db5e49 None None] REQUEST_METHOD: `POST` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:41:29.891386 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2505369d-3302-4dad-afff-d3d806db5e49 None None] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:41:29.891386 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2505369d-3302-4dad-afff-d3d806db5e49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:41:29.998102 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.auth.core [None req-2505369d-3302-4dad-afff-d3d806db5e49 None None] MFA Rules not processed for user `fd836cf590a145df9acf19cf3ae2ac56`. Rule list: `[]` (Enabled: `True`). {{(pid=88113) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:41:29.999965 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.fernet_utils [None req-2505369d-3302-4dad-afff-d3d806db5e49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:41:30.044422 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=100 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:41:30.126882 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 711/1418] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 31 09:41:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 238 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:41:30.399553 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48aa7326-dbb2-4e81-95e9-fe826b432658 None None] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:41:30.401113 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.fernet_utils [None req-48aa7326-dbb2-4e81-95e9-fe826b432658 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88114) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:41:30.586878 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48aa7326-dbb2-4e81-95e9-fe826b432658 None None] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:41:30.631640 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=100 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:41:30.712140 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fce153b-ea95-4cd0-981e-c9c05319ab2d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc076493239d4ccdbdc37c0369c49529', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '49548eb44d62415582af820ab66decee', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:41:30.712759 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fce153b-ea95-4cd0-981e-c9c05319ab2d service nova] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:41:30.713040 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fce153b-ea95-4cd0-981e-c9c05319ab2d service nova] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:41:30.713314 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fce153b-ea95-4cd0-981e-c9c05319ab2d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:41:30.714977 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.fernet_utils [None req-1fce153b-ea95-4cd0-981e-c9c05319ab2d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88114) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:41:30.813467 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fce153b-ea95-4cd0-981e-c9c05319ab2d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:41:30.815073 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fce153b-ea95-4cd0-981e-c9c05319ab2d service nova] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:41:30.835816 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 708/1419] 10.222.0.13 () {60 vars in 1365 bytes} [Wed Aug 31 09:41:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 437 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:41:31.766972 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c100846a-c595-48b4-9444-07f711d8d678 None None] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:41:31.767534 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c100846a-c595-48b4-9444-07f711d8d678 None None] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:41:31.767534 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c100846a-c595-48b4-9444-07f711d8d678 None None] PATH_INFO: `/` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:41:31.768833 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 712/1420] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 31 09:41:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:41:31.774553 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1453e1-3a6c-4de8-ab98-78629cf25762 None None] REQUEST_METHOD: `POST` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:41:31.775132 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1453e1-3a6c-4de8-ab98-78629cf25762 None None] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:41:31.775132 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1453e1-3a6c-4de8-ab98-78629cf25762 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:41:31.836088 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=100 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:41:31.866059 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.auth.core [None req-ad1453e1-3a6c-4de8-ab98-78629cf25762 None None] MFA Rules not processed for user `fd836cf590a145df9acf19cf3ae2ac56`. Rule list: `[]` (Enabled: `True`). {{(pid=88114) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:41:31.867310 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.fernet_utils [None req-ad1453e1-3a6c-4de8-ab98-78629cf25762 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88114) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:41:31.982645 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 709/1421] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 31 09:41:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:41:32.143251 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c100846a-c595-48b4-9444-07f711d8d678 None None] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:41:32.144708 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.fernet_utils [None req-c100846a-c595-48b4-9444-07f711d8d678 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:41:32.346122 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c100846a-c595-48b4-9444-07f711d8d678 None None] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:41:32.365660 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=100 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:41:32.511063 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-942c68ec-2424-40da-ba93-7dd14c0a0366 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc076493239d4ccdbdc37c0369c49529', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '49548eb44d62415582af820ab66decee', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:41:32.512097 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942c68ec-2424-40da-ba93-7dd14c0a0366 service nova] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:41:32.512596 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942c68ec-2424-40da-ba93-7dd14c0a0366 service nova] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:41:32.513087 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942c68ec-2424-40da-ba93-7dd14c0a0366 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:41:32.516134 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.fernet_utils [None req-942c68ec-2424-40da-ba93-7dd14c0a0366 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:41:32.612714 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-942c68ec-2424-40da-ba93-7dd14c0a0366 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:41:32.614356 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-942c68ec-2424-40da-ba93-7dd14c0a0366 service nova] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:41:32.633663 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 713/1422] 10.222.0.13 () {60 vars in 1365 bytes} [Wed Aug 31 09:41:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 491 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:41:33.562453 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2309b58-f7f3-4fbe-ad04-b1cbf1674f9a None None] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:41:33.563001 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2309b58-f7f3-4fbe-ad04-b1cbf1674f9a None None] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:41:33.563172 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2309b58-f7f3-4fbe-ad04-b1cbf1674f9a None None] PATH_INFO: `/` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:41:33.564536 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 710/1423] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 31 09:41:33 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:41:33.569941 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5892d34b-e424-4e37-8187-ba789abeaff6 None None] REQUEST_METHOD: `POST` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:41:33.570441 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5892d34b-e424-4e37-8187-ba789abeaff6 None None] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:41:33.570553 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5892d34b-e424-4e37-8187-ba789abeaff6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:41:33.591454 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=100 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:41:33.656349 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.auth.core [None req-5892d34b-e424-4e37-8187-ba789abeaff6 None None] MFA Rules not processed for user `fd836cf590a145df9acf19cf3ae2ac56`. Rule list: `[]` (Enabled: `True`). {{(pid=88113) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:41:33.657678 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.fernet_utils [None req-5892d34b-e424-4e37-8187-ba789abeaff6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:41:33.767531 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 714/1424] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 31 09:41:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:41:33.924745 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2309b58-f7f3-4fbe-ad04-b1cbf1674f9a None None] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:41:33.926324 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.fernet_utils [None req-e2309b58-f7f3-4fbe-ad04-b1cbf1674f9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88114) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:41:34.049129 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=100 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:41:34.104215 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2309b58-f7f3-4fbe-ad04-b1cbf1674f9a None None] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:41:34.267129 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb98940-e5dd-4ff5-a290-3594a4993580 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc076493239d4ccdbdc37c0369c49529', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '49548eb44d62415582af820ab66decee', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:41:34.267749 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb98940-e5dd-4ff5-a290-3594a4993580 service nova] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:41:34.268026 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb98940-e5dd-4ff5-a290-3594a4993580 service nova] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:41:34.268294 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb98940-e5dd-4ff5-a290-3594a4993580 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:41:34.269955 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.fernet_utils [None req-bbb98940-e5dd-4ff5-a290-3594a4993580 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88114) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:41:34.370734 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=100 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:41:34.386941 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbb98940-e5dd-4ff5-a290-3594a4993580 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:41:34.388568 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbb98940-e5dd-4ff5-a290-3594a4993580 service nova] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:41:34.410907 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 711/1425] 10.222.0.13 () {60 vars in 1365 bytes} [Wed Aug 31 09:41:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 487 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:41:35.319440 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37525cb8-c3da-4416-839a-284545155ff2 None None] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:41:35.320016 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37525cb8-c3da-4416-839a-284545155ff2 None None] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:41:35.320016 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37525cb8-c3da-4416-839a-284545155ff2 None None] PATH_INFO: `/` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:41:35.321358 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 715/1426] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 31 09:41:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:41:35.329659 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d79597d-fcc7-45a3-a2b2-5dc6964ab25c None None] REQUEST_METHOD: `POST` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:41:35.330317 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d79597d-fcc7-45a3-a2b2-5dc6964ab25c None None] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:41:35.330954 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d79597d-fcc7-45a3-a2b2-5dc6964ab25c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:41:35.423461 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.auth.core [None req-1d79597d-fcc7-45a3-a2b2-5dc6964ab25c None None] MFA Rules not processed for user `fd836cf590a145df9acf19cf3ae2ac56`. Rule list: `[]` (Enabled: `True`). {{(pid=88114) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:41:35.424741 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.fernet_utils [None req-1d79597d-fcc7-45a3-a2b2-5dc6964ab25c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88114) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:41:35.536330 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 712/1427] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 31 09:41:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 208 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:41:35.695616 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37525cb8-c3da-4416-839a-284545155ff2 None None] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:41:35.697164 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.fernet_utils [None req-37525cb8-c3da-4416-839a-284545155ff2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:41:35.797907 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=100 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:41:35.906410 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37525cb8-c3da-4416-839a-284545155ff2 None None] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:41:36.028954 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94392cb1-447d-44ce-9bed-271821f7918f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc076493239d4ccdbdc37c0369c49529', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '49548eb44d62415582af820ab66decee', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:41:36.029587 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94392cb1-447d-44ce-9bed-271821f7918f service nova] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:41:36.029868 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94392cb1-447d-44ce-9bed-271821f7918f service nova] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:41:36.030139 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94392cb1-447d-44ce-9bed-271821f7918f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:41:36.032467 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.fernet_utils [None req-94392cb1-447d-44ce-9bed-271821f7918f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:41:36.084688 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=100 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:41:36.130516 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94392cb1-447d-44ce-9bed-271821f7918f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:41:36.132153 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94392cb1-447d-44ce-9bed-271821f7918f service nova] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:41:36.151748 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 716/1428] 10.222.0.13 () {60 vars in 1365 bytes} [Wed Aug 31 09:41:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 457 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:41:36.977779 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6cfd3a2-c6f4-4599-8189-5837f2827b19 None None] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:41:36.978250 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6cfd3a2-c6f4-4599-8189-5837f2827b19 None None] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:41:36.978406 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6cfd3a2-c6f4-4599-8189-5837f2827b19 None None] PATH_INFO: `/` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:41:36.979663 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 713/1429] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 31 09:41:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:41:36.986235 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe841df-c4a7-43d6-bc23-c136a77497cf None None] REQUEST_METHOD: `POST` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:41:36.986843 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe841df-c4a7-43d6-bc23-c136a77497cf None None] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:41:36.986983 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe841df-c4a7-43d6-bc23-c136a77497cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:41:37.073871 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.auth.core [None req-efe841df-c4a7-43d6-bc23-c136a77497cf None None] MFA Rules not processed for user `fd836cf590a145df9acf19cf3ae2ac56`. Rule list: `[]` (Enabled: `True`). {{(pid=88113) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:41:37.075123 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.fernet_utils [None req-efe841df-c4a7-43d6-bc23-c136a77497cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:41:37.184628 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 717/1430] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 31 09:41:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 09:41:37.338584 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6cfd3a2-c6f4-4599-8189-5837f2827b19 None None] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:41:37.340038 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.fernet_utils [None req-f6cfd3a2-c6f4-4599-8189-5837f2827b19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88114) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:41:37.401979 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=100 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:41:37.514264 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6cfd3a2-c6f4-4599-8189-5837f2827b19 None None] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:41:37.649549 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc9f3899-6d0d-4563-97df-0be04dcf5e60 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc076493239d4ccdbdc37c0369c49529', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '49548eb44d62415582af820ab66decee', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:41:37.650079 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9f3899-6d0d-4563-97df-0be04dcf5e60 service nova] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:41:37.650359 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9f3899-6d0d-4563-97df-0be04dcf5e60 service nova] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:41:37.650628 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9f3899-6d0d-4563-97df-0be04dcf5e60 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:41:37.652250 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.fernet_utils [None req-dc9f3899-6d0d-4563-97df-0be04dcf5e60 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88114) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:41:37.659740 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=100 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:41:37.750389 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc9f3899-6d0d-4563-97df-0be04dcf5e60 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:41:37.752028 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc9f3899-6d0d-4563-97df-0be04dcf5e60 service nova] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:41:37.771864 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 714/1431] 10.222.0.13 () {60 vars in 1365 bytes} [Wed Aug 31 09:41:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 434 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:41:38.610448 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27308139-d8ed-47dc-8600-d2a6e2001a04 None None] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:41:38.611091 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27308139-d8ed-47dc-8600-d2a6e2001a04 None None] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:41:38.611208 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27308139-d8ed-47dc-8600-d2a6e2001a04 None None] PATH_INFO: `/` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:41:38.613076 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 718/1432] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 31 09:41:38 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 31 09:41:38.619338 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047c9f40-9a22-4810-b595-c4448409ce7d None None] REQUEST_METHOD: `POST` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:41:38.619993 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047c9f40-9a22-4810-b595-c4448409ce7d None None] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:41:38.620231 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047c9f40-9a22-4810-b595-c4448409ce7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:41:38.714303 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.auth.core [None req-047c9f40-9a22-4810-b595-c4448409ce7d None None] MFA Rules not processed for user `fd836cf590a145df9acf19cf3ae2ac56`. Rule list: `[]` (Enabled: `True`). {{(pid=88114) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:41:38.715512 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.fernet_utils [None req-047c9f40-9a22-4810-b595-c4448409ce7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88114) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:41:38.812876 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=100 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:41:38.844177 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 715/1433] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 31 09:41:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 226 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:41:39.009583 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27308139-d8ed-47dc-8600-d2a6e2001a04 None None] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:41:39.011433 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.fernet_utils [None req-27308139-d8ed-47dc-8600-d2a6e2001a04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:41:39.031889 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=100 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:41:39.176257 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27308139-d8ed-47dc-8600-d2a6e2001a04 None None] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:41:39.271180 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=100 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:41:39.300322 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e05e32-c67b-4655-8d59-aaa4439a0e6a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc076493239d4ccdbdc37c0369c49529', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '49548eb44d62415582af820ab66decee', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:41:39.300938 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e05e32-c67b-4655-8d59-aaa4439a0e6a service nova] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:41:39.301218 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e05e32-c67b-4655-8d59-aaa4439a0e6a service nova] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:41:39.301535 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e05e32-c67b-4655-8d59-aaa4439a0e6a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:41:39.303165 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.fernet_utils [None req-43e05e32-c67b-4655-8d59-aaa4439a0e6a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:41:39.406359 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43e05e32-c67b-4655-8d59-aaa4439a0e6a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:41:39.407997 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43e05e32-c67b-4655-8d59-aaa4439a0e6a service nova] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:41:39.427242 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 719/1434] 10.222.0.13 () {60 vars in 1365 bytes} [Wed Aug 31 09:41:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 418 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:41:40.262454 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a014d1-19ab-4963-83ce-10d805e29c47 None None] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:41:40.263314 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a014d1-19ab-4963-83ce-10d805e29c47 None None] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:41:40.263314 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a014d1-19ab-4963-83ce-10d805e29c47 None None] PATH_INFO: `/` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:41:40.264241 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 716/1435] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 31 09:41:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 31 09:41:40.269847 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb84efa-6043-4281-bac4-2130ea101ec7 None None] REQUEST_METHOD: `POST` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:41:40.270339 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb84efa-6043-4281-bac4-2130ea101ec7 None None] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:41:40.270447 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb84efa-6043-4281-bac4-2130ea101ec7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:41:40.361999 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.auth.core [None req-ebb84efa-6043-4281-bac4-2130ea101ec7 None None] MFA Rules not processed for user `fd836cf590a145df9acf19cf3ae2ac56`. Rule list: `[]` (Enabled: `True`). {{(pid=88113) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:41:40.363210 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.fernet_utils [None req-ebb84efa-6043-4281-bac4-2130ea101ec7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:41:40.410918 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=100 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:41:40.477495 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 720/1436] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 31 09:41:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:41:40.639978 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a014d1-19ab-4963-83ce-10d805e29c47 None None] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:41:40.642264 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.fernet_utils [None req-66a014d1-19ab-4963-83ce-10d805e29c47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88114) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:41:40.824240 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a014d1-19ab-4963-83ce-10d805e29c47 None None] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:41:40.887866 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=100 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:41:40.958122 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d3b8a59-41e0-4218-a5a9-03515c2d34d8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc076493239d4ccdbdc37c0369c49529', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '49548eb44d62415582af820ab66decee', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:41:40.958723 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3b8a59-41e0-4218-a5a9-03515c2d34d8 service nova] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:41:40.958990 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3b8a59-41e0-4218-a5a9-03515c2d34d8 service nova] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:41:40.959257 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3b8a59-41e0-4218-a5a9-03515c2d34d8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:41:40.960852 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.fernet_utils [None req-4d3b8a59-41e0-4218-a5a9-03515c2d34d8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88114) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:41:41.064879 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d3b8a59-41e0-4218-a5a9-03515c2d34d8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:41:41.066551 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d3b8a59-41e0-4218-a5a9-03515c2d34d8 service nova] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:41:41.090225 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 717/1437] 10.222.0.13 () {60 vars in 1365 bytes} [Wed Aug 31 09:41:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 451 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:41:41.970655 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e723d54d-f62b-40cf-95ba-22ca528ea249 None None] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:41:41.971228 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e723d54d-f62b-40cf-95ba-22ca528ea249 None None] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:41:41.971621 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e723d54d-f62b-40cf-95ba-22ca528ea249 None None] PATH_INFO: `/` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:41:41.973794 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 721/1438] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 31 09:41:41 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:41:41.980240 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2accf99-1383-491c-ab8d-43cee42e0952 None None] REQUEST_METHOD: `POST` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:41:41.980721 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2accf99-1383-491c-ab8d-43cee42e0952 None None] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:41:41.980838 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2accf99-1383-491c-ab8d-43cee42e0952 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:41:42.066548 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.auth.core [None req-b2accf99-1383-491c-ab8d-43cee42e0952 None None] MFA Rules not processed for user `fd836cf590a145df9acf19cf3ae2ac56`. Rule list: `[]` (Enabled: `True`). {{(pid=88114) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:41:42.067868 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.fernet_utils [None req-b2accf99-1383-491c-ab8d-43cee42e0952 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88114) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:41:42.070450 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=100 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:41:42.178859 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 718/1439] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 31 09:41:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 09:41:42.380001 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e723d54d-f62b-40cf-95ba-22ca528ea249 None None] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:41:42.381502 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.fernet_utils [None req-e723d54d-f62b-40cf-95ba-22ca528ea249 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:41:42.541130 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e723d54d-f62b-40cf-95ba-22ca528ea249 None None] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:41:42.570097 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=100 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:41:42.671681 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65538f76-8764-4cf5-9bcb-821634fb4e6e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc076493239d4ccdbdc37c0369c49529', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '49548eb44d62415582af820ab66decee', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:41:42.672234 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65538f76-8764-4cf5-9bcb-821634fb4e6e service nova] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:41:42.672513 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65538f76-8764-4cf5-9bcb-821634fb4e6e service nova] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:41:42.672784 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65538f76-8764-4cf5-9bcb-821634fb4e6e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:41:42.674450 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.fernet_utils [None req-65538f76-8764-4cf5-9bcb-821634fb4e6e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:41:42.773864 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65538f76-8764-4cf5-9bcb-821634fb4e6e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:41:42.775505 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65538f76-8764-4cf5-9bcb-821634fb4e6e service nova] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:41:42.795085 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 722/1440] 10.222.0.13 () {60 vars in 1365 bytes} [Wed Aug 31 09:41:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 416 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:41:44.726171 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e696450a-52f5-40db-a0eb-fcf63536a468 None None] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:41:44.726767 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e696450a-52f5-40db-a0eb-fcf63536a468 None None] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:41:44.726767 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e696450a-52f5-40db-a0eb-fcf63536a468 None None] PATH_INFO: `/` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:41:44.728416 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 719/1441] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 31 09:41:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:41:44.734763 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d0e3e9-bb6b-4ad8-9f8f-53fe6d0eb1b6 None None] REQUEST_METHOD: `POST` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:41:44.735350 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d0e3e9-bb6b-4ad8-9f8f-53fe6d0eb1b6 None None] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:41:44.735350 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d0e3e9-bb6b-4ad8-9f8f-53fe6d0eb1b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:41:44.782008 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=100 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:41:44.814995 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.auth.core [None req-d2d0e3e9-bb6b-4ad8-9f8f-53fe6d0eb1b6 None None] MFA Rules not processed for user `fd836cf590a145df9acf19cf3ae2ac56`. Rule list: `[]` (Enabled: `True`). {{(pid=88113) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:41:44.816221 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.fernet_utils [None req-d2d0e3e9-bb6b-4ad8-9f8f-53fe6d0eb1b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:41:44.929428 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 723/1442] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 31 09:41:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:41:45.237984 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e696450a-52f5-40db-a0eb-fcf63536a468 None None] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:41:45.239630 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.fernet_utils [None req-e696450a-52f5-40db-a0eb-fcf63536a468 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88114) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:41:45.361714 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e696450a-52f5-40db-a0eb-fcf63536a468 None None] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:41:45.398790 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=100 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:41:45.474695 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c553fe4f-fd29-44d5-8c9b-7f85d446ddfd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9bdaacd597442c2bcde2c555f2cc140', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '49548eb44d62415582af820ab66decee', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:41:45.475706 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c553fe4f-fd29-44d5-8c9b-7f85d446ddfd service neutron] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:41:45.476204 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c553fe4f-fd29-44d5-8c9b-7f85d446ddfd service neutron] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:41:45.476687 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c553fe4f-fd29-44d5-8c9b-7f85d446ddfd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:41:45.479791 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.fernet_utils [None req-c553fe4f-fd29-44d5-8c9b-7f85d446ddfd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88114) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:41:45.613156 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c553fe4f-fd29-44d5-8c9b-7f85d446ddfd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:41:45.614659 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c553fe4f-fd29-44d5-8c9b-7f85d446ddfd service neutron] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:41:45.633990 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 720/1443] 10.222.0.13 () {60 vars in 1365 bytes} [Wed Aug 31 09:41:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 397 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 09:41:47.824285 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a422a63-3471-48b1-b53d-f4222222925f None None] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:41:47.825040 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a422a63-3471-48b1-b53d-f4222222925f None None] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:41:47.825040 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a422a63-3471-48b1-b53d-f4222222925f None None] PATH_INFO: `/` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:41:47.826140 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 724/1444] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 31 09:41:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:41:47.833231 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229191e5-dce7-41df-ae69-75de0f753082 None None] REQUEST_METHOD: `POST` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:41:47.833945 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229191e5-dce7-41df-ae69-75de0f753082 None None] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:41:47.834058 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229191e5-dce7-41df-ae69-75de0f753082 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:41:47.910301 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.auth.core [None req-229191e5-dce7-41df-ae69-75de0f753082 None None] MFA Rules not processed for user `fd836cf590a145df9acf19cf3ae2ac56`. Rule list: `[]` (Enabled: `True`). {{(pid=88114) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:41:47.911501 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.fernet_utils [None req-229191e5-dce7-41df-ae69-75de0f753082 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88114) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:41:47.925359 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=100 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:41:48.017927 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 721/1445] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 31 09:41:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:41:48.302135 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a422a63-3471-48b1-b53d-f4222222925f None None] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:41:48.303742 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.fernet_utils [None req-1a422a63-3471-48b1-b53d-f4222222925f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:41:48.474533 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a422a63-3471-48b1-b53d-f4222222925f None None] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:41:48.500816 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=100 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:41:48.580166 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-717ecded-f048-496b-bd51-433ead5d5208 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9bdaacd597442c2bcde2c555f2cc140', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '49548eb44d62415582af820ab66decee', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:41:48.580675 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717ecded-f048-496b-bd51-433ead5d5208 service neutron] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:41:48.580955 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717ecded-f048-496b-bd51-433ead5d5208 service neutron] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:41:48.581219 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717ecded-f048-496b-bd51-433ead5d5208 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:41:48.582824 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.fernet_utils [None req-717ecded-f048-496b-bd51-433ead5d5208 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:41:48.698947 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-717ecded-f048-496b-bd51-433ead5d5208 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:41:48.700424 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-717ecded-f048-496b-bd51-433ead5d5208 service neutron] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:41:48.721487 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 725/1446] 10.222.0.13 () {60 vars in 1365 bytes} [Wed Aug 31 09:41:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 420 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:41:51.025208 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27f53ab-165d-4e64-808b-0ef69f33dbb3 None None] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:41:51.025998 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27f53ab-165d-4e64-808b-0ef69f33dbb3 None None] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:41:51.025998 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27f53ab-165d-4e64-808b-0ef69f33dbb3 None None] PATH_INFO: `/` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:41:51.027437 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 722/1447] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 31 09:41:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:41:51.038302 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aadf28cf-5943-4aba-bc6a-4ea85682080c None None] REQUEST_METHOD: `POST` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:41:51.038855 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aadf28cf-5943-4aba-bc6a-4ea85682080c None None] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:41:51.039077 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aadf28cf-5943-4aba-bc6a-4ea85682080c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:41:51.095855 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=100 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:41:51.116273 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.auth.core [None req-aadf28cf-5943-4aba-bc6a-4ea85682080c None None] MFA Rules not processed for user `fd836cf590a145df9acf19cf3ae2ac56`. Rule list: `[]` (Enabled: `True`). {{(pid=88113) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:41:51.117543 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.fernet_utils [None req-aadf28cf-5943-4aba-bc6a-4ea85682080c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:41:51.222097 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 726/1448] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 31 09:41:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:41:51.512067 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a27f53ab-165d-4e64-808b-0ef69f33dbb3 None None] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:41:51.513673 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.fernet_utils [None req-a27f53ab-165d-4e64-808b-0ef69f33dbb3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88114) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:41:51.627015 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a27f53ab-165d-4e64-808b-0ef69f33dbb3 None None] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:41:51.668964 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=100 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:41:51.707391 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea9d3da-58a7-46d7-82a0-1f3c7d25859f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9bdaacd597442c2bcde2c555f2cc140', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '49548eb44d62415582af820ab66decee', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:41:51.707995 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea9d3da-58a7-46d7-82a0-1f3c7d25859f service neutron] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:41:51.708268 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea9d3da-58a7-46d7-82a0-1f3c7d25859f service neutron] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:41:51.708539 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea9d3da-58a7-46d7-82a0-1f3c7d25859f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:41:51.710177 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.fernet_utils [None req-2ea9d3da-58a7-46d7-82a0-1f3c7d25859f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88114) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:41:51.825464 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ea9d3da-58a7-46d7-82a0-1f3c7d25859f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:41:51.826904 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ea9d3da-58a7-46d7-82a0-1f3c7d25859f service neutron] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:41:51.846854 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 723/1449] 10.222.0.13 () {60 vars in 1365 bytes} [Wed Aug 31 09:41:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 335 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:41:53.953724 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab5db6f-a7c9-4f00-8c96-2b8d11cfe046 None None] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:41:53.954425 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab5db6f-a7c9-4f00-8c96-2b8d11cfe046 None None] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:41:53.954425 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab5db6f-a7c9-4f00-8c96-2b8d11cfe046 None None] PATH_INFO: `/` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:41:53.955637 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 727/1450] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 31 09:41:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:41:53.962278 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad2bcd1-22e9-44f0-be93-7bd51bf57946 None None] REQUEST_METHOD: `POST` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:41:53.962786 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad2bcd1-22e9-44f0-be93-7bd51bf57946 None None] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:41:53.963334 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad2bcd1-22e9-44f0-be93-7bd51bf57946 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:41:54.036524 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.auth.core [None req-fad2bcd1-22e9-44f0-be93-7bd51bf57946 None None] MFA Rules not processed for user `fd836cf590a145df9acf19cf3ae2ac56`. Rule list: `[]` (Enabled: `True`). {{(pid=88114) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:41:54.037777 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.fernet_utils [None req-fad2bcd1-22e9-44f0-be93-7bd51bf57946 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88114) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:41:54.059432 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=100 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:41:54.146959 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 724/1451] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 31 09:41:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:41:54.435104 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab5db6f-a7c9-4f00-8c96-2b8d11cfe046 None None] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:41:54.437223 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.fernet_utils [None req-4ab5db6f-a7c9-4f00-8c96-2b8d11cfe046 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:41:54.547859 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab5db6f-a7c9-4f00-8c96-2b8d11cfe046 None None] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:41:54.573589 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=100 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:41:54.630135 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f7b3958-89a4-4131-849e-83de433d98e4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9bdaacd597442c2bcde2c555f2cc140', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '49548eb44d62415582af820ab66decee', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:41:54.630741 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7b3958-89a4-4131-849e-83de433d98e4 service neutron] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:41:54.631030 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7b3958-89a4-4131-849e-83de433d98e4 service neutron] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:41:54.631280 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7b3958-89a4-4131-849e-83de433d98e4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:41:54.632876 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.fernet_utils [None req-2f7b3958-89a4-4131-849e-83de433d98e4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:41:54.770669 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f7b3958-89a4-4131-849e-83de433d98e4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:41:54.772160 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f7b3958-89a4-4131-849e-83de433d98e4 service neutron] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:41:54.791905 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 728/1452] 10.222.0.13 () {60 vars in 1365 bytes} [Wed Aug 31 09:41:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 357 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:41:56.871905 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24407c96-56e8-4e0c-bb00-333293362669 None None] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:41:56.872441 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24407c96-56e8-4e0c-bb00-333293362669 None None] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:41:56.872441 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24407c96-56e8-4e0c-bb00-333293362669 None None] PATH_INFO: `/` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:41:56.873716 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 725/1453] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 31 09:41:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:41:56.879131 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e16118-cf37-43c5-ac70-e3d39b47c0d5 None None] REQUEST_METHOD: `POST` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:41:56.879549 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e16118-cf37-43c5-ac70-e3d39b47c0d5 None None] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:41:56.879672 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e16118-cf37-43c5-ac70-e3d39b47c0d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:41:56.946791 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=100 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:41:56.949710 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.auth.core [None req-f7e16118-cf37-43c5-ac70-e3d39b47c0d5 None None] MFA Rules not processed for user `fd836cf590a145df9acf19cf3ae2ac56`. Rule list: `[]` (Enabled: `True`). {{(pid=88113) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:41:56.952197 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.fernet_utils [None req-f7e16118-cf37-43c5-ac70-e3d39b47c0d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:41:57.051759 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 729/1454] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 31 09:41:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:41:57.342731 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24407c96-56e8-4e0c-bb00-333293362669 None None] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:41:57.344298 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.fernet_utils [None req-24407c96-56e8-4e0c-bb00-333293362669 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88114) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:41:57.457201 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24407c96-56e8-4e0c-bb00-333293362669 None None] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:41:57.509649 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=100 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:41:57.539288 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce549720-3010-4054-b107-599c7aeed3a2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9bdaacd597442c2bcde2c555f2cc140', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '49548eb44d62415582af820ab66decee', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:41:57.539894 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce549720-3010-4054-b107-599c7aeed3a2 service neutron] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:41:57.540172 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce549720-3010-4054-b107-599c7aeed3a2 service neutron] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:41:57.540431 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce549720-3010-4054-b107-599c7aeed3a2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:41:57.542058 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.fernet_utils [None req-ce549720-3010-4054-b107-599c7aeed3a2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88114) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:41:57.660693 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce549720-3010-4054-b107-599c7aeed3a2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:41:57.662068 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce549720-3010-4054-b107-599c7aeed3a2 service neutron] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:41:57.681296 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 726/1455] 10.222.0.13 () {60 vars in 1365 bytes} [Wed Aug 31 09:41:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 339 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:41:59.727681 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e211a2-1d82-408e-8cbf-8e8733f4d528 None None] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:41:59.728251 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e211a2-1d82-408e-8cbf-8e8733f4d528 None None] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:41:59.728251 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e211a2-1d82-408e-8cbf-8e8733f4d528 None None] PATH_INFO: `/` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:41:59.729655 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 730/1456] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 31 09:41:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:41:59.735235 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae6a04c-c54e-4198-9aa0-cdc28851fefd None None] REQUEST_METHOD: `POST` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:41:59.735711 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae6a04c-c54e-4198-9aa0-cdc28851fefd None None] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:41:59.735809 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae6a04c-c54e-4198-9aa0-cdc28851fefd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:41:59.806707 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.auth.core [None req-dae6a04c-c54e-4198-9aa0-cdc28851fefd None None] MFA Rules not processed for user `fd836cf590a145df9acf19cf3ae2ac56`. Rule list: `[]` (Enabled: `True`). {{(pid=88114) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:41:59.807962 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.fernet_utils [None req-dae6a04c-c54e-4198-9aa0-cdc28851fefd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88114) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:41:59.838768 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=100 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:41:59.913725 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 727/1457] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 31 09:41:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:42:00.202634 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e211a2-1d82-408e-8cbf-8e8733f4d528 None None] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:42:00.204172 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.fernet_utils [None req-51e211a2-1d82-408e-8cbf-8e8733f4d528 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:42:00.312221 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e211a2-1d82-408e-8cbf-8e8733f4d528 None None] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:42:00.342674 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=100 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:42:00.392153 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a6234ee-7f6b-42c7-a5d9-8bbd605e72f6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9bdaacd597442c2bcde2c555f2cc140', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '49548eb44d62415582af820ab66decee', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:42:00.392784 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6234ee-7f6b-42c7-a5d9-8bbd605e72f6 service neutron] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:42:00.393062 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6234ee-7f6b-42c7-a5d9-8bbd605e72f6 service neutron] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:42:00.393331 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6234ee-7f6b-42c7-a5d9-8bbd605e72f6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:42:00.394973 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.fernet_utils [None req-1a6234ee-7f6b-42c7-a5d9-8bbd605e72f6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:42:00.507432 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a6234ee-7f6b-42c7-a5d9-8bbd605e72f6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:42:00.508803 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a6234ee-7f6b-42c7-a5d9-8bbd605e72f6 service neutron] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:42:00.528288 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 731/1458] 10.222.0.13 () {60 vars in 1365 bytes} [Wed Aug 31 09:42:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:42:02.529355 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8a5865-98b3-44ec-a63a-6d4459809311 None None] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:42:02.530447 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8a5865-98b3-44ec-a63a-6d4459809311 None None] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:42:02.531249 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8a5865-98b3-44ec-a63a-6d4459809311 None None] PATH_INFO: `/` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:42:02.532627 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 728/1459] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 31 09:42:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:42:02.538459 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6dbbaf-2743-4174-8ecd-a5241992a9eb None None] REQUEST_METHOD: `POST` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:42:02.538995 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6dbbaf-2743-4174-8ecd-a5241992a9eb None None] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:42:02.538995 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6dbbaf-2743-4174-8ecd-a5241992a9eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:42:02.609310 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.auth.core [None req-1c6dbbaf-2743-4174-8ecd-a5241992a9eb None None] MFA Rules not processed for user `fd836cf590a145df9acf19cf3ae2ac56`. Rule list: `[]` (Enabled: `True`). {{(pid=88113) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:42:02.610551 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.fernet_utils [None req-1c6dbbaf-2743-4174-8ecd-a5241992a9eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:42:02.619111 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=100 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:42:02.712340 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 732/1460] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 31 09:42:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:42:02.996797 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb8a5865-98b3-44ec-a63a-6d4459809311 None None] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:42:02.998494 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.fernet_utils [None req-bb8a5865-98b3-44ec-a63a-6d4459809311 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88114) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:42:03.104587 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb8a5865-98b3-44ec-a63a-6d4459809311 None None] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:42:03.161550 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=100 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:42:03.182803 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff21b46-eb7f-43bb-a8f2-5b8c27ddb321 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9bdaacd597442c2bcde2c555f2cc140', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '49548eb44d62415582af820ab66decee', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:42:03.183430 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff21b46-eb7f-43bb-a8f2-5b8c27ddb321 service neutron] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:42:03.183707 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff21b46-eb7f-43bb-a8f2-5b8c27ddb321 service neutron] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:42:03.183973 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff21b46-eb7f-43bb-a8f2-5b8c27ddb321 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:42:03.186658 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.fernet_utils [None req-eff21b46-eb7f-43bb-a8f2-5b8c27ddb321 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88114) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:42:03.436924 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eff21b46-eb7f-43bb-a8f2-5b8c27ddb321 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:42:03.438572 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eff21b46-eb7f-43bb-a8f2-5b8c27ddb321 service neutron] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:42:03.460412 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 729/1461] 10.222.0.13 () {60 vars in 1365 bytes} [Wed Aug 31 09:42:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 464 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:42:05.458708 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3d278c-89f1-476a-a65e-90735e11cf65 None None] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:42:05.459324 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3d278c-89f1-476a-a65e-90735e11cf65 None None] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:42:05.459324 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3d278c-89f1-476a-a65e-90735e11cf65 None None] PATH_INFO: `/` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:42:05.460617 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 733/1462] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 31 09:42:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:42:05.466197 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e162cf1-5b86-47a3-9445-4a8e26a2cd54 None None] REQUEST_METHOD: `POST` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:42:05.466693 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e162cf1-5b86-47a3-9445-4a8e26a2cd54 None None] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:42:05.466859 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e162cf1-5b86-47a3-9445-4a8e26a2cd54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:42:05.537872 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.auth.core [None req-5e162cf1-5b86-47a3-9445-4a8e26a2cd54 None None] MFA Rules not processed for user `fd836cf590a145df9acf19cf3ae2ac56`. Rule list: `[]` (Enabled: `True`). {{(pid=88114) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:42:05.539083 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.fernet_utils [None req-5e162cf1-5b86-47a3-9445-4a8e26a2cd54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88114) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:42:05.580677 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=100 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:42:05.641887 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 730/1463] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 31 09:42:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:42:05.924013 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a3d278c-89f1-476a-a65e-90735e11cf65 None None] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:42:05.925608 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.fernet_utils [None req-1a3d278c-89f1-476a-a65e-90735e11cf65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:42:06.037167 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a3d278c-89f1-476a-a65e-90735e11cf65 None None] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:42:06.076075 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=100 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:42:06.113811 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cc3d53c-8f1d-4072-b898-d8e63a1a4ec4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9bdaacd597442c2bcde2c555f2cc140', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '49548eb44d62415582af820ab66decee', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:42:06.114416 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc3d53c-8f1d-4072-b898-d8e63a1a4ec4 service neutron] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:42:06.114698 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc3d53c-8f1d-4072-b898-d8e63a1a4ec4 service neutron] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:42:06.114960 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc3d53c-8f1d-4072-b898-d8e63a1a4ec4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:42:06.116552 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.fernet_utils [None req-7cc3d53c-8f1d-4072-b898-d8e63a1a4ec4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:42:06.232224 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cc3d53c-8f1d-4072-b898-d8e63a1a4ec4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:42:06.234245 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cc3d53c-8f1d-4072-b898-d8e63a1a4ec4 service neutron] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:42:06.253574 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 734/1464] 10.222.0.13 () {60 vars in 1365 bytes} [Wed Aug 31 09:42:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 330 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:42:08.262891 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc8856a-0327-42aa-a663-17c8b0bed883 None None] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:42:08.263420 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc8856a-0327-42aa-a663-17c8b0bed883 None None] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:42:08.263420 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc8856a-0327-42aa-a663-17c8b0bed883 None None] PATH_INFO: `/` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:42:08.264720 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 731/1465] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 31 09:42:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:42:08.270083 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb389f79-4c3d-4180-9121-d1fd4b612f75 None None] REQUEST_METHOD: `POST` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:42:08.270544 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb389f79-4c3d-4180-9121-d1fd4b612f75 None None] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:42:08.270649 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb389f79-4c3d-4180-9121-d1fd4b612f75 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:42:08.338503 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.auth.core [None req-bb389f79-4c3d-4180-9121-d1fd4b612f75 None None] MFA Rules not processed for user `fd836cf590a145df9acf19cf3ae2ac56`. Rule list: `[]` (Enabled: `True`). {{(pid=88113) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:42:08.339707 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.fernet_utils [None req-bb389f79-4c3d-4180-9121-d1fd4b612f75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:42:08.358477 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=100 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:42:08.438987 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 735/1466] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 31 09:42:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:42:08.722563 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc8856a-0327-42aa-a663-17c8b0bed883 None None] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:42:08.724080 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.fernet_utils [None req-bcc8856a-0327-42aa-a663-17c8b0bed883 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88114) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:42:08.830967 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc8856a-0327-42aa-a663-17c8b0bed883 None None] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:42:08.904686 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=100 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:42:08.913864 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab1b487-82e7-4ffa-9c9e-566f2362a483 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9bdaacd597442c2bcde2c555f2cc140', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '49548eb44d62415582af820ab66decee', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:42:08.914511 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab1b487-82e7-4ffa-9c9e-566f2362a483 service neutron] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:42:08.914991 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab1b487-82e7-4ffa-9c9e-566f2362a483 service neutron] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:42:08.915485 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab1b487-82e7-4ffa-9c9e-566f2362a483 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:42:08.917319 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.fernet_utils [None req-0ab1b487-82e7-4ffa-9c9e-566f2362a483 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88114) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:42:09.047499 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ab1b487-82e7-4ffa-9c9e-566f2362a483 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:42:09.048923 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ab1b487-82e7-4ffa-9c9e-566f2362a483 service neutron] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:42:09.068297 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 732/1467] 10.222.0.13 () {60 vars in 1365 bytes} [Wed Aug 31 09:42:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 346 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:42:11.028265 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354cac03-cc6e-4a57-ae70-cf9ee618769e None None] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:42:11.028849 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354cac03-cc6e-4a57-ae70-cf9ee618769e None None] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:42:11.028849 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354cac03-cc6e-4a57-ae70-cf9ee618769e None None] PATH_INFO: `/` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:42:11.030400 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 736/1468] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 31 09:42:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:42:11.036338 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a887d188-3d39-4481-84f2-049f44e5c3fc None None] REQUEST_METHOD: `POST` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:42:11.036797 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a887d188-3d39-4481-84f2-049f44e5c3fc None None] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:42:11.036890 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a887d188-3d39-4481-84f2-049f44e5c3fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:42:11.106957 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.auth.core [None req-a887d188-3d39-4481-84f2-049f44e5c3fc None None] MFA Rules not processed for user `fd836cf590a145df9acf19cf3ae2ac56`. Rule list: `[]` (Enabled: `True`). {{(pid=88114) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:42:11.108416 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.fernet_utils [None req-a887d188-3d39-4481-84f2-049f44e5c3fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88114) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:42:11.161553 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=100 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:42:11.213124 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 733/1469] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 31 09:42:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:42:11.497430 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-354cac03-cc6e-4a57-ae70-cf9ee618769e None None] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:42:11.499166 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.fernet_utils [None req-354cac03-cc6e-4a57-ae70-cf9ee618769e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:42:11.607245 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-354cac03-cc6e-4a57-ae70-cf9ee618769e None None] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:42:11.655820 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=100 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:42:11.682176 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1259d561-50a9-43a7-83c3-e29c8dc54fe1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9bdaacd597442c2bcde2c555f2cc140', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '49548eb44d62415582af820ab66decee', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:42:11.682784 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1259d561-50a9-43a7-83c3-e29c8dc54fe1 service neutron] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:42:11.683058 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1259d561-50a9-43a7-83c3-e29c8dc54fe1 service neutron] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:42:11.683325 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1259d561-50a9-43a7-83c3-e29c8dc54fe1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:42:11.684939 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.fernet_utils [None req-1259d561-50a9-43a7-83c3-e29c8dc54fe1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:42:11.796865 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1259d561-50a9-43a7-83c3-e29c8dc54fe1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:42:11.798260 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1259d561-50a9-43a7-83c3-e29c8dc54fe1 service neutron] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:42:11.817676 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 737/1470] 10.222.0.13 () {60 vars in 1365 bytes} [Wed Aug 31 09:42:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:42:21.211353 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=22 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:42:21.815416 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=66 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:42:32.007423 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309756b7-6feb-4833-bda3-8d26b36d594f None None] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:42:32.007949 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309756b7-6feb-4833-bda3-8d26b36d594f None None] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:42:32.008055 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309756b7-6feb-4833-bda3-8d26b36d594f None None] PATH_INFO: `/` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:42:32.009617 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 734/1471] 10.222.0.13 () {54 vars in 927 bytes} [Wed Aug 31 09:42:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:42:32.015409 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c6aacf-e76b-4a7d-800d-4ab5c8176c85 None None] REQUEST_METHOD: `POST` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:42:32.015807 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c6aacf-e76b-4a7d-800d-4ab5c8176c85 None None] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:42:32.015959 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c6aacf-e76b-4a7d-800d-4ab5c8176c85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:42:32.088209 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.auth.core [None req-58c6aacf-e76b-4a7d-800d-4ab5c8176c85 None None] MFA Rules not processed for user `fd836cf590a145df9acf19cf3ae2ac56`. Rule list: `[]` (Enabled: `True`). {{(pid=88113) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:42:32.089460 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.fernet_utils [None req-58c6aacf-e76b-4a7d-800d-4ab5c8176c85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:42:32.190066 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 738/1472] 10.222.0.13 () {58 vars in 1024 bytes} [Wed Aug 31 09:42:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:42:32.490325 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2888f2-049e-4e7d-9a81-b040f27606c1 None None] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:42:32.490620 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2888f2-049e-4e7d-9a81-b040f27606c1 None None] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:42:32.490775 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2888f2-049e-4e7d-9a81-b040f27606c1 None None] PATH_INFO: `/` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:42:32.492091 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 735/1473] 10.222.0.13 () {54 vars in 967 bytes} [Wed Aug 31 09:42:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:42:32.496818 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b243858-0541-43d9-841a-7720b2f62a30 None None] REQUEST_METHOD: `POST` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:42:32.497148 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b243858-0541-43d9-841a-7720b2f62a30 None None] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:42:32.497467 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b243858-0541-43d9-841a-7720b2f62a30 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:42:32.577940 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.auth.core [None req-8b243858-0541-43d9-841a-7720b2f62a30 None None] MFA Rules not processed for user `be216d156b54452cbbf978c7734343aa`. Rule list: `[]` (Enabled: `True`). {{(pid=88113) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:42:32.579165 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.fernet_utils [None req-8b243858-0541-43d9-841a-7720b2f62a30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:42:32.600174 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=100 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:42:32.659293 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 739/1474] 10.222.0.13 () {58 vars in 1064 bytes} [Wed Aug 31 09:42:32 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:42:32.667006 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d2888f2-049e-4e7d-9a81-b040f27606c1 None None] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:42:32.668506 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.fernet_utils [None req-9d2888f2-049e-4e7d-9a81-b040f27606c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88114) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:42:32.789652 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d2888f2-049e-4e7d-9a81-b040f27606c1 None None] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:42:32.869906 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac830562-d1cb-493e-b719-363573c44778 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be216d156b54452cbbf978c7734343aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '49548eb44d62415582af820ab66decee', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:42:32.870944 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac830562-d1cb-493e-b719-363573c44778 service glance] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:42:32.871437 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac830562-d1cb-493e-b719-363573c44778 service glance] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:42:32.871921 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac830562-d1cb-493e-b719-363573c44778 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:42:32.875062 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.fernet_utils [None req-ac830562-d1cb-493e-b719-363573c44778 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88114) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:42:32.937681 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=100 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:42:32.994527 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac830562-d1cb-493e-b719-363573c44778 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:42:32.996021 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac830562-d1cb-493e-b719-363573c44778 service glance] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:42:33.015964 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 736/1475] 10.222.0.13 () {60 vars in 1365 bytes} [Wed Aug 31 09:42:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 349 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:42:35.065912 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba161a98-b422-456f-9c73-8842807b50c2 None None] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:42:35.066487 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba161a98-b422-456f-9c73-8842807b50c2 None None] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:42:35.066487 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba161a98-b422-456f-9c73-8842807b50c2 None None] PATH_INFO: `/` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:42:35.067820 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 740/1476] 10.222.0.13 () {54 vars in 927 bytes} [Wed Aug 31 09:42:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:42:35.073055 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e640ba-5af0-477e-90e6-0d48437bdebd None None] REQUEST_METHOD: `POST` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:42:35.073494 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e640ba-5af0-477e-90e6-0d48437bdebd None None] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:42:35.073605 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e640ba-5af0-477e-90e6-0d48437bdebd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:42:35.146527 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.auth.core [None req-f8e640ba-5af0-477e-90e6-0d48437bdebd None None] MFA Rules not processed for user `fd836cf590a145df9acf19cf3ae2ac56`. Rule list: `[]` (Enabled: `True`). {{(pid=88114) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:42:35.147752 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.fernet_utils [None req-f8e640ba-5af0-477e-90e6-0d48437bdebd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88114) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:42:35.248467 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 737/1477] 10.222.0.13 () {58 vars in 1024 bytes} [Wed Aug 31 09:42:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:42:35.527238 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba161a98-b422-456f-9c73-8842807b50c2 None None] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:42:35.529333 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.fernet_utils [None req-ba161a98-b422-456f-9c73-8842807b50c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:42:35.663956 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba161a98-b422-456f-9c73-8842807b50c2 None None] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:42:35.715936 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=100 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:42:35.761805 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a0be9e-94ad-43d6-a97c-4c64c818e1fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:42:35.762421 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a0be9e-94ad-43d6-a97c-4c64c818e1fd admin admin] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:42:35.763362 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a0be9e-94ad-43d6-a97c-4c64c818e1fd admin admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:42:35.764113 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a0be9e-94ad-43d6-a97c-4c64c818e1fd admin admin] PATH_INFO: `/v3/projects` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:42:35.764263 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95a0be9e-94ad-43d6-a97c-4c64c818e1fd admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:42:35.766126 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: WARNING py.warnings [None req-95a0be9e-94ad-43d6-a97c-4c64c818e1fd admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 31 09:42:35.766126 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: warnings.warn(msg) Aug 31 09:42:35.766126 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: Aug 31 09:42:35.767271 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95a0be9e-94ad-43d6-a97c-4c64c818e1fd admin admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:42:35.787218 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 741/1478] 10.222.0.13 () {56 vars in 1108 bytes} [Wed Aug 31 09:42:35 2022] GET /identity/v3/projects => generated 2553 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 09:42:37.754607 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5b21ab-79a0-4dcf-9198-1b0469d198ee None None] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:42:37.755206 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5b21ab-79a0-4dcf-9198-1b0469d198ee None None] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:42:37.755206 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5b21ab-79a0-4dcf-9198-1b0469d198ee None None] PATH_INFO: `/` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:42:37.756757 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 738/1479] 10.222.0.13 () {54 vars in 927 bytes} [Wed Aug 31 09:42:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:42:37.763229 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7790fd-013f-48ed-8fc7-8ab89dc2af51 None None] REQUEST_METHOD: `POST` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:42:37.763801 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7790fd-013f-48ed-8fc7-8ab89dc2af51 None None] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:42:37.763801 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7790fd-013f-48ed-8fc7-8ab89dc2af51 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:42:37.951041 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.auth.core [None req-0a7790fd-013f-48ed-8fc7-8ab89dc2af51 None None] MFA Rules not processed for user `fd836cf590a145df9acf19cf3ae2ac56`. Rule list: `[]` (Enabled: `True`). {{(pid=88113) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:42:37.952259 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.fernet_utils [None req-0a7790fd-013f-48ed-8fc7-8ab89dc2af51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:42:38.048472 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 742/1480] 10.222.0.13 () {58 vars in 1024 bytes} [Wed Aug 31 09:42:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 287 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:42:38.345845 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c5b21ab-79a0-4dcf-9198-1b0469d198ee None None] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:42:38.347440 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.fernet_utils [None req-8c5b21ab-79a0-4dcf-9198-1b0469d198ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88114) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:42:38.358111 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=100 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:42:38.502285 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c5b21ab-79a0-4dcf-9198-1b0469d198ee None None] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:42:38.586971 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=100 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:42:38.621086 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-745e4973-ca22-4ea8-a4c6-e4f9a269e0c6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc076493239d4ccdbdc37c0369c49529', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '49548eb44d62415582af820ab66decee', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:42:38.621793 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745e4973-ca22-4ea8-a4c6-e4f9a269e0c6 service nova] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:42:38.622630 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745e4973-ca22-4ea8-a4c6-e4f9a269e0c6 service nova] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:42:38.623588 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745e4973-ca22-4ea8-a4c6-e4f9a269e0c6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:42:38.625188 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.fernet_utils [None req-745e4973-ca22-4ea8-a4c6-e4f9a269e0c6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88114) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:42:38.716725 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-745e4973-ca22-4ea8-a4c6-e4f9a269e0c6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:42:38.718339 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-745e4973-ca22-4ea8-a4c6-e4f9a269e0c6 service nova] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:42:38.736297 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 739/1481] 10.222.0.13 () {60 vars in 1365 bytes} [Wed Aug 31 09:42:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 391 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:42:40.917202 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0b9df1-2427-4baa-b8e9-1e0e37ae1a08 None None] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:42:40.917781 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0b9df1-2427-4baa-b8e9-1e0e37ae1a08 None None] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:42:40.917781 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0b9df1-2427-4baa-b8e9-1e0e37ae1a08 None None] PATH_INFO: `/` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:42:40.920214 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 743/1482] 10.222.0.13 () {54 vars in 927 bytes} [Wed Aug 31 09:42:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:42:40.925602 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41b7fe7-d159-48f6-8243-52c3a8da5b1b None None] REQUEST_METHOD: `POST` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:42:40.926026 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41b7fe7-d159-48f6-8243-52c3a8da5b1b None None] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:42:40.926150 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41b7fe7-d159-48f6-8243-52c3a8da5b1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:42:40.995628 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.auth.core [None req-f41b7fe7-d159-48f6-8243-52c3a8da5b1b None None] MFA Rules not processed for user `fd836cf590a145df9acf19cf3ae2ac56`. Rule list: `[]` (Enabled: `True`). {{(pid=88114) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:42:40.996899 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.fernet_utils [None req-f41b7fe7-d159-48f6-8243-52c3a8da5b1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88114) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:42:41.043242 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=100 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:42:41.096699 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 740/1483] 10.222.0.13 () {58 vars in 1024 bytes} [Wed Aug 31 09:42:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:42:41.388661 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f0b9df1-2427-4baa-b8e9-1e0e37ae1a08 None None] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:42:41.390359 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.fernet_utils [None req-4f0b9df1-2427-4baa-b8e9-1e0e37ae1a08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:42:41.417582 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=100 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:42:41.502567 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f0b9df1-2427-4baa-b8e9-1e0e37ae1a08 None None] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:42:41.579424 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2adf668f-4a4b-471d-9420-c28ea5cce778 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9bdaacd597442c2bcde2c555f2cc140', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '49548eb44d62415582af820ab66decee', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:42:41.580008 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2adf668f-4a4b-471d-9420-c28ea5cce778 service neutron] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:42:41.580286 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2adf668f-4a4b-471d-9420-c28ea5cce778 service neutron] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:42:41.580546 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2adf668f-4a4b-471d-9420-c28ea5cce778 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:42:41.582143 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.fernet_utils [None req-2adf668f-4a4b-471d-9420-c28ea5cce778 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:42:41.656508 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=100 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:42:41.694238 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2adf668f-4a4b-471d-9420-c28ea5cce778 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:42:41.695949 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2adf668f-4a4b-471d-9420-c28ea5cce778 service neutron] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:42:41.714847 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 744/1484] 10.222.0.13 () {60 vars in 1365 bytes} [Wed Aug 31 09:42:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:42:43.733070 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b4c109-2039-4825-91a6-f119f05bdc0c None None] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:42:43.733673 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b4c109-2039-4825-91a6-f119f05bdc0c None None] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:42:43.733673 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b4c109-2039-4825-91a6-f119f05bdc0c None None] PATH_INFO: `/` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:42:43.735072 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 741/1485] 10.222.0.13 () {54 vars in 927 bytes} [Wed Aug 31 09:42:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:42:43.740814 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d61563-3902-49ce-bc02-a20f0cc699f9 None None] REQUEST_METHOD: `POST` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:42:43.741269 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d61563-3902-49ce-bc02-a20f0cc699f9 None None] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:42:43.741411 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d61563-3902-49ce-bc02-a20f0cc699f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:42:43.818153 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.auth.core [None req-d4d61563-3902-49ce-bc02-a20f0cc699f9 None None] MFA Rules not processed for user `fd836cf590a145df9acf19cf3ae2ac56`. Rule list: `[]` (Enabled: `True`). {{(pid=88113) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:42:43.819278 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.fernet_utils [None req-d4d61563-3902-49ce-bc02-a20f0cc699f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:42:43.917023 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 745/1486] 10.222.0.13 () {58 vars in 1024 bytes} [Wed Aug 31 09:42:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:42:44.212819 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b4c109-2039-4825-91a6-f119f05bdc0c None None] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:42:44.214424 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.fernet_utils [None req-a0b4c109-2039-4825-91a6-f119f05bdc0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88114) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:42:44.323325 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b4c109-2039-4825-91a6-f119f05bdc0c None None] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:42:44.396469 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=100 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:42:44.398292 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c8944dd-cd8f-437c-b6d9-d93f4cb58d77 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9bdaacd597442c2bcde2c555f2cc140', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '49548eb44d62415582af820ab66decee', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:42:44.398909 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c8944dd-cd8f-437c-b6d9-d93f4cb58d77 service neutron] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:42:44.399376 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c8944dd-cd8f-437c-b6d9-d93f4cb58d77 service neutron] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:42:44.399861 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c8944dd-cd8f-437c-b6d9-d93f4cb58d77 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:42:44.401739 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.fernet_utils [None req-9c8944dd-cd8f-437c-b6d9-d93f4cb58d77 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88114) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:42:44.516153 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c8944dd-cd8f-437c-b6d9-d93f4cb58d77 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:42:44.517532 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c8944dd-cd8f-437c-b6d9-d93f4cb58d77 service neutron] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:42:44.536226 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 742/1487] 10.222.0.13 () {60 vars in 1365 bytes} [Wed Aug 31 09:42:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:42:46.635573 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1db6565-b2b5-4f09-b2dd-37db5d2884ca None None] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:42:46.636184 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1db6565-b2b5-4f09-b2dd-37db5d2884ca None None] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:42:46.636321 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1db6565-b2b5-4f09-b2dd-37db5d2884ca None None] PATH_INFO: `/` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:42:46.638327 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 746/1488] 10.222.0.13 () {54 vars in 927 bytes} [Wed Aug 31 09:42:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:42:46.643904 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46cd0cd-9c92-4f8c-be1f-475cd1e7551e None None] REQUEST_METHOD: `POST` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:42:46.644380 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46cd0cd-9c92-4f8c-be1f-475cd1e7551e None None] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:42:46.644486 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46cd0cd-9c92-4f8c-be1f-475cd1e7551e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:42:46.735770 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.auth.core [None req-a46cd0cd-9c92-4f8c-be1f-475cd1e7551e None None] MFA Rules not processed for user `fd836cf590a145df9acf19cf3ae2ac56`. Rule list: `[]` (Enabled: `True`). {{(pid=88114) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:42:46.736977 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.fernet_utils [None req-a46cd0cd-9c92-4f8c-be1f-475cd1e7551e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88114) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:42:46.792327 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=100 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:42:46.842465 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 743/1489] 10.222.0.13 () {58 vars in 1024 bytes} [Wed Aug 31 09:42:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:42:47.148093 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1db6565-b2b5-4f09-b2dd-37db5d2884ca None None] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:42:47.149657 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.fernet_utils [None req-b1db6565-b2b5-4f09-b2dd-37db5d2884ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:42:47.187828 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=100 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:42:47.261707 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1db6565-b2b5-4f09-b2dd-37db5d2884ca None None] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:42:47.342945 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40956b50-2fb3-4f2c-b861-d53c0c7732a8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9bdaacd597442c2bcde2c555f2cc140', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '49548eb44d62415582af820ab66decee', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:42:47.343558 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40956b50-2fb3-4f2c-b861-d53c0c7732a8 service neutron] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:42:47.343835 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40956b50-2fb3-4f2c-b861-d53c0c7732a8 service neutron] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:42:47.344106 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40956b50-2fb3-4f2c-b861-d53c0c7732a8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:42:47.345759 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.fernet_utils [None req-40956b50-2fb3-4f2c-b861-d53c0c7732a8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:42:47.432002 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=100 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:42:47.467536 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40956b50-2fb3-4f2c-b861-d53c0c7732a8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:42:47.469718 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40956b50-2fb3-4f2c-b861-d53c0c7732a8 service neutron] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:42:47.496891 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 747/1490] 10.222.0.13 () {60 vars in 1365 bytes} [Wed Aug 31 09:42:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 349 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:42:49.680664 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93d2b67-21b9-413f-8a18-5530b749ff8d None None] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:42:49.681216 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93d2b67-21b9-413f-8a18-5530b749ff8d None None] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:42:49.681341 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93d2b67-21b9-413f-8a18-5530b749ff8d None None] PATH_INFO: `/` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:42:49.682621 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 744/1491] 10.222.0.13 () {54 vars in 927 bytes} [Wed Aug 31 09:42:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:42:49.688761 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4376847d-70d4-47c6-b8ff-119cc7d7e92d None None] REQUEST_METHOD: `POST` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:42:49.689234 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4376847d-70d4-47c6-b8ff-119cc7d7e92d None None] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:42:49.689346 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4376847d-70d4-47c6-b8ff-119cc7d7e92d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:42:49.757778 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.auth.core [None req-4376847d-70d4-47c6-b8ff-119cc7d7e92d None None] MFA Rules not processed for user `fd836cf590a145df9acf19cf3ae2ac56`. Rule list: `[]` (Enabled: `True`). {{(pid=88113) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:42:49.759243 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.fernet_utils [None req-4376847d-70d4-47c6-b8ff-119cc7d7e92d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:42:49.856915 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 748/1492] 10.222.0.13 () {58 vars in 1024 bytes} [Wed Aug 31 09:42:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:42:50.148888 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d93d2b67-21b9-413f-8a18-5530b749ff8d None None] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:42:50.150435 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.fernet_utils [None req-d93d2b67-21b9-413f-8a18-5530b749ff8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88114) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:42:50.256453 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d93d2b67-21b9-413f-8a18-5530b749ff8d None None] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:42:50.332139 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63735743-3f42-4ddb-962c-11a1ae8e8eff service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9bdaacd597442c2bcde2c555f2cc140', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '49548eb44d62415582af820ab66decee', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:42:50.332748 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63735743-3f42-4ddb-962c-11a1ae8e8eff service neutron] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:42:50.333028 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63735743-3f42-4ddb-962c-11a1ae8e8eff service neutron] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:42:50.333306 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63735743-3f42-4ddb-962c-11a1ae8e8eff service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:42:50.334937 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.fernet_utils [None req-63735743-3f42-4ddb-962c-11a1ae8e8eff service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88114) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:42:50.345121 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=100 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:42:50.447744 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63735743-3f42-4ddb-962c-11a1ae8e8eff service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:42:50.449183 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63735743-3f42-4ddb-962c-11a1ae8e8eff service neutron] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:42:50.469180 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 745/1493] 10.222.0.13 () {60 vars in 1365 bytes} [Wed Aug 31 09:42:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 09:42:59.855005 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=82 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:43:00.465990 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=50 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:44:04.807037 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e244308f-4ea2-4f6b-b84b-fb9ae0606af0 None None] REQUEST_METHOD: `POST` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:44:04.807657 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e244308f-4ea2-4f6b-b84b-fb9ae0606af0 None None] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:44:04.807657 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e244308f-4ea2-4f6b-b84b-fb9ae0606af0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:44:04.885784 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.auth.core [None req-e244308f-4ea2-4f6b-b84b-fb9ae0606af0 None None] MFA Rules not processed for user `fd836cf590a145df9acf19cf3ae2ac56`. Rule list: `[]` (Enabled: `True`). {{(pid=88113) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:44:04.887034 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.fernet_utils [None req-e244308f-4ea2-4f6b-b84b-fb9ae0606af0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:44:04.988861 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 749/1494] 10.222.0.13 () {58 vars in 958 bytes} [Wed Aug 31 09:44:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:44:04.997917 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b3352b-afef-4e90-a043-3a25fdea69e2 None None] REQUEST_METHOD: `POST` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:44:04.998423 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b3352b-afef-4e90-a043-3a25fdea69e2 None None] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:44:04.998540 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b3352b-afef-4e90-a043-3a25fdea69e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:44:05.082676 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.auth.core [None req-f5b3352b-afef-4e90-a043-3a25fdea69e2 None None] MFA Rules not processed for user `fd836cf590a145df9acf19cf3ae2ac56`. Rule list: `[]` (Enabled: `True`). {{(pid=88114) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:44:05.083803 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.fernet_utils [None req-f5b3352b-afef-4e90-a043-3a25fdea69e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88114) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:44:05.183308 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 746/1495] 10.222.0.13 () {58 vars in 958 bytes} [Wed Aug 31 09:44:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:44:05.188054 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e244308f-4ea2-4f6b-b84b-fb9ae0606af0 None None] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:44:05.189249 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.fernet_utils [None req-e244308f-4ea2-4f6b-b84b-fb9ae0606af0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:44:05.201272 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e244308f-4ea2-4f6b-b84b-fb9ae0606af0 None None] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:44:05.265017 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=100 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:44:05.298968 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca5d5821-67fa-4537-839b-80d5680329f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:44:05.299557 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5d5821-67fa-4537-839b-80d5680329f4 admin admin] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:44:05.299833 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5d5821-67fa-4537-839b-80d5680329f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:44:05.300100 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5d5821-67fa-4537-839b-80d5680329f4 admin admin] PATH_INFO: `/v3/domains` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:44:05.300736 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca5d5821-67fa-4537-839b-80d5680329f4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88113) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:44:05.301032 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca5d5821-67fa-4537-839b-80d5680329f4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:44:05.302075 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: WARNING py.warnings [None req-ca5d5821-67fa-4537-839b-80d5680329f4 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 31 09:44:05.302075 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: warnings.warn(msg) Aug 31 09:44:05.302075 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: Aug 31 09:44:05.302951 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca5d5821-67fa-4537-839b-80d5680329f4 admin admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:44:05.315055 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 750/1496] 10.222.0.13 () {58 vars in 1153 bytes} [Wed Aug 31 09:44:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 127 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:44:05.323987 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b3352b-afef-4e90-a043-3a25fdea69e2 None None] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:44:05.333264 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b3352b-afef-4e90-a043-3a25fdea69e2 None None] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:44:05.396583 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=100 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:44:05.431149 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a210d8-7b15-48f9-bbd5-12f199e388bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:44:05.432554 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a210d8-7b15-48f9-bbd5-12f199e388bf admin admin] REQUEST_METHOD: `POST` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:44:05.432968 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a210d8-7b15-48f9-bbd5-12f199e388bf admin admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:44:05.433433 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a210d8-7b15-48f9-bbd5-12f199e388bf admin admin] PATH_INFO: `/v3/projects` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:44:05.434074 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07a210d8-7b15-48f9-bbd5-12f199e388bf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:44:05.435826 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: WARNING py.warnings [None req-07a210d8-7b15-48f9-bbd5-12f199e388bf admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 31 09:44:05.435826 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: warnings.warn(msg) Aug 31 09:44:05.435826 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: Aug 31 09:44:05.437006 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07a210d8-7b15-48f9-bbd5-12f199e388bf admin admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:44:05.494091 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 747/1497] 10.222.0.13 () {60 vars in 1153 bytes} [Wed Aug 31 09:44:05 2022] POST /identity/v3/projects => generated 377 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 09:44:05.498461 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca5d5821-67fa-4537-839b-80d5680329f4 admin admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:44:05.500013 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.fernet_utils [None req-ca5d5821-67fa-4537-839b-80d5680329f4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:44:05.534591 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca5d5821-67fa-4537-839b-80d5680329f4 admin admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:44:05.630192 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bfb3fea-50cf-4807-ba5c-4528bb4fe014 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:44:05.630708 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bfb3fea-50cf-4807-ba5c-4528bb4fe014 admin admin] REQUEST_METHOD: `POST` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:44:05.630988 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bfb3fea-50cf-4807-ba5c-4528bb4fe014 admin admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:44:05.631255 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bfb3fea-50cf-4807-ba5c-4528bb4fe014 admin admin] PATH_INFO: `/v3/users` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:44:05.631879 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bfb3fea-50cf-4807-ba5c-4528bb4fe014 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:44:05.633153 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: WARNING py.warnings [None req-1bfb3fea-50cf-4807-ba5c-4528bb4fe014 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 31 09:44:05.633153 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: warnings.warn(msg) Aug 31 09:44:05.633153 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: Aug 31 09:44:05.634386 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bfb3fea-50cf-4807-ba5c-4528bb4fe014 admin admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:44:05.676478 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 751/1498] 10.222.0.13 () {60 vars in 1144 bytes} [Wed Aug 31 09:44:05 2022] POST /identity/v3/users => generated 289 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 09:44:05.680716 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a210d8-7b15-48f9-bbd5-12f199e388bf admin admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:44:05.682856 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.fernet_utils [None req-07a210d8-7b15-48f9-bbd5-12f199e388bf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88114) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:44:05.748517 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a210d8-7b15-48f9-bbd5-12f199e388bf admin admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:44:05.833826 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:44:05.845286 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4599fc39-4fc5-4e0d-bd49-4ea348149e58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:44:05.845957 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4599fc39-4fc5-4e0d-bd49-4ea348149e58 admin admin] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:44:05.846431 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4599fc39-4fc5-4e0d-bd49-4ea348149e58 admin admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:44:05.846917 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4599fc39-4fc5-4e0d-bd49-4ea348149e58 admin admin] PATH_INFO: `/v3/roles` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:44:05.847835 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4599fc39-4fc5-4e0d-bd49-4ea348149e58 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:44:05.848863 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: WARNING py.warnings [None req-4599fc39-4fc5-4e0d-bd49-4ea348149e58 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 31 09:44:05.848863 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: warnings.warn(msg) Aug 31 09:44:05.848863 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: Aug 31 09:44:05.849790 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4599fc39-4fc5-4e0d-bd49-4ea348149e58 admin admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:44:05.862638 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 748/1499] 10.222.0.13 () {58 vars in 1122 bytes} [Wed Aug 31 09:44:05 2022] GET /identity/v3/roles => generated 1420 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 09:44:05.866846 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bfb3fea-50cf-4807-ba5c-4528bb4fe014 admin admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:44:05.868374 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.fernet_utils [None req-1bfb3fea-50cf-4807-ba5c-4528bb4fe014 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:44:05.880379 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bfb3fea-50cf-4807-ba5c-4528bb4fe014 admin admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:44:05.905799 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:44:05.983735 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61dfbe6f-ed94-4438-947e-ab0535682fae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:44:05.984355 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61dfbe6f-ed94-4438-947e-ab0535682fae admin admin] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:44:05.984631 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61dfbe6f-ed94-4438-947e-ab0535682fae admin admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:44:05.984895 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61dfbe6f-ed94-4438-947e-ab0535682fae admin admin] PATH_INFO: `/v3/roles` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:44:05.985564 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61dfbe6f-ed94-4438-947e-ab0535682fae admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:44:05.986577 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: WARNING py.warnings [None req-61dfbe6f-ed94-4438-947e-ab0535682fae admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 31 09:44:05.986577 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: warnings.warn(msg) Aug 31 09:44:05.986577 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: Aug 31 09:44:05.987456 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61dfbe6f-ed94-4438-947e-ab0535682fae admin admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:44:06.000883 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 752/1500] 10.222.0.13 () {58 vars in 1122 bytes} [Wed Aug 31 09:44:05 2022] GET /identity/v3/roles => generated 1420 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 09:44:06.005184 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4599fc39-4fc5-4e0d-bd49-4ea348149e58 admin admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:44:06.006739 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.fernet_utils [None req-4599fc39-4fc5-4e0d-bd49-4ea348149e58 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88114) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:44:06.040825 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4599fc39-4fc5-4e0d-bd49-4ea348149e58 admin admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:44:06.138801 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06fcc1a5-a444-4635-a08e-32a86a124219 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:44:06.139372 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06fcc1a5-a444-4635-a08e-32a86a124219 admin admin] REQUEST_METHOD: `PUT` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:44:06.139641 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06fcc1a5-a444-4635-a08e-32a86a124219 admin admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:44:06.139902 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06fcc1a5-a444-4635-a08e-32a86a124219 admin admin] PATH_INFO: `/v3/projects/954fbb77916e4641ba59cd19f5924b63/users/8d66d0950add4450bfe7c9833ce6c2e3/roles/6c99f5d9f1f243eaa134d8540f21095d` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:44:06.161396 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06fcc1a5-a444-4635-a08e-32a86a124219 admin admin] RBAC: Authorizing `identity:create_grant(project_id=954fbb77916e4641ba59cd19f5924b63, user_id=8d66d0950add4450bfe7c9833ce6c2e3, role_id=6c99f5d9f1f243eaa134d8540f21095d)` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:44:06.162424 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: WARNING py.warnings [None req-06fcc1a5-a444-4635-a08e-32a86a124219 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 31 09:44:06.162424 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: warnings.warn(msg) Aug 31 09:44:06.162424 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: Aug 31 09:44:06.164345 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06fcc1a5-a444-4635-a08e-32a86a124219 admin admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:44:06.190606 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 749/1501] 10.222.0.13 () {60 vars in 1483 bytes} [Wed Aug 31 09:44:06 2022] PUT /identity/v3/projects/954fbb77916e4641ba59cd19f5924b63/users/8d66d0950add4450bfe7c9833ce6c2e3/roles/6c99f5d9f1f243eaa134d8540f21095d => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:44:06.202708 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fecd9081-00b0-4a63-9ffa-c65e09569a19 None None] REQUEST_METHOD: `POST` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:44:06.202991 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fecd9081-00b0-4a63-9ffa-c65e09569a19 None None] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:44:06.203259 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fecd9081-00b0-4a63-9ffa-c65e09569a19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:44:06.260761 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.auth.core [None req-fecd9081-00b0-4a63-9ffa-c65e09569a19 None None] MFA Rules not processed for user `8d66d0950add4450bfe7c9833ce6c2e3`. Rule list: `[]` (Enabled: `True`). {{(pid=88113) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:44:06.261991 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.fernet_utils [None req-fecd9081-00b0-4a63-9ffa-c65e09569a19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:44:06.338979 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 753/1502] 10.222.0.13 () {58 vars in 958 bytes} [Wed Aug 31 09:44:06 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:44:06.352293 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c3fc50-a5bf-4b4d-99e7-e85eab36b806 None None] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:44:06.352582 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c3fc50-a5bf-4b4d-99e7-e85eab36b806 None None] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:44:06.352863 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c3fc50-a5bf-4b4d-99e7-e85eab36b806 None None] PATH_INFO: `/` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:44:06.355250 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 750/1503] 10.222.0.13 () {54 vars in 964 bytes} [Wed Aug 31 09:44:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:44:06.360096 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040ec0b0-883a-4a46-b32a-1e0be2ef2856 None None] REQUEST_METHOD: `POST` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:44:06.360382 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040ec0b0-883a-4a46-b32a-1e0be2ef2856 None None] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:44:06.360653 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040ec0b0-883a-4a46-b32a-1e0be2ef2856 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:44:06.392733 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=100 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:44:06.438957 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.auth.core [None req-040ec0b0-883a-4a46-b32a-1e0be2ef2856 None None] MFA Rules not processed for user `0d52b69c8e94441dacfcff944326f169`. Rule list: `[]` (Enabled: `True`). {{(pid=88113) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:44:06.440165 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.fernet_utils [None req-040ec0b0-883a-4a46-b32a-1e0be2ef2856 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:44:06.518388 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 754/1504] 10.222.0.13 () {58 vars in 1061 bytes} [Wed Aug 31 09:44:06 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:44:06.526234 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c3fc50-a5bf-4b4d-99e7-e85eab36b806 None None] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:44:06.527770 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.fernet_utils [None req-a3c3fc50-a5bf-4b4d-99e7-e85eab36b806 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88114) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:44:06.584346 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:44:06.642134 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c3fc50-a5bf-4b4d-99e7-e85eab36b806 None None] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:44:06.718173 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05bbbd90-a07f-44d3-8b7d-62e931e73c5a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d52b69c8e94441dacfcff944326f169', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '49548eb44d62415582af820ab66decee', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:44:06.718781 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05bbbd90-a07f-44d3-8b7d-62e931e73c5a service cinder] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:44:06.719058 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05bbbd90-a07f-44d3-8b7d-62e931e73c5a service cinder] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:44:06.719327 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05bbbd90-a07f-44d3-8b7d-62e931e73c5a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:44:06.721687 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.fernet_utils [None req-05bbbd90-a07f-44d3-8b7d-62e931e73c5a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88114) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:44:06.825888 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05bbbd90-a07f-44d3-8b7d-62e931e73c5a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:44:06.827329 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05bbbd90-a07f-44d3-8b7d-62e931e73c5a service cinder] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:44:06.830817 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=100 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:44:06.846155 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 751/1505] 10.222.0.13 () {60 vars in 1365 bytes} [Wed Aug 31 09:44:06 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:44:06.892945 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce027f8a-b252-45a8-9933-bcd633af7e48 None None] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:44:06.893405 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce027f8a-b252-45a8-9933-bcd633af7e48 None None] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:44:06.893555 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce027f8a-b252-45a8-9933-bcd633af7e48 None None] PATH_INFO: `/` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:44:06.894790 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 755/1506] 10.222.0.13 () {52 vars in 830 bytes} [Wed Aug 31 09:44:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:44:06.898469 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05bbbd90-a07f-44d3-8b7d-62e931e73c5a service cinder] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:44:06.899680 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.fernet_utils [None req-05bbbd90-a07f-44d3-8b7d-62e931e73c5a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88114) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:44:07.001146 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05bbbd90-a07f-44d3-8b7d-62e931e73c5a service cinder] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:44:07.097196 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-890e51c0-599e-4e0a-81d1-cfa1f3d6a044 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:44:07.097964 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890e51c0-599e-4e0a-81d1-cfa1f3d6a044 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:44:07.098138 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890e51c0-599e-4e0a-81d1-cfa1f3d6a044 admin admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:44:07.098403 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890e51c0-599e-4e0a-81d1-cfa1f3d6a044 admin admin] PATH_INFO: `/v3/users/8d66d0950add4450bfe7c9833ce6c2e3` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:44:07.110728 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-890e51c0-599e-4e0a-81d1-cfa1f3d6a044 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8d66d0950add4450bfe7c9833ce6c2e3)` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:44:07.111752 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: WARNING py.warnings [None req-890e51c0-599e-4e0a-81d1-cfa1f3d6a044 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 31 09:44:07.111752 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: warnings.warn(msg) Aug 31 09:44:07.111752 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: Aug 31 09:44:07.112953 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-890e51c0-599e-4e0a-81d1-cfa1f3d6a044 admin admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:44:07.123797 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=100 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:44:07.187099 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.notifications [None req-890e51c0-599e-4e0a-81d1-cfa1f3d6a044 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8d66d0950add4450bfe7c9833ce6c2e3'} {{(pid=88114) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:44:07.202284 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.notifications [None req-890e51c0-599e-4e0a-81d1-cfa1f3d6a044 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8d66d0950add4450bfe7c9833ce6c2e3'} {{(pid=88114) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:44:07.212093 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.notifications [None req-890e51c0-599e-4e0a-81d1-cfa1f3d6a044 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8d66d0950add4450bfe7c9833ce6c2e3'} {{(pid=88114) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:44:07.212657 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.notifications [None req-890e51c0-599e-4e0a-81d1-cfa1f3d6a044 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8d66d0950add4450bfe7c9833ce6c2e3'} {{(pid=88114) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:44:07.240622 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.notifications [None req-890e51c0-599e-4e0a-81d1-cfa1f3d6a044 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8d66d0950add4450bfe7c9833ce6c2e3'} {{(pid=88114) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:44:07.247109 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 752/1507] 10.222.0.13 () {58 vars in 1224 bytes} [Wed Aug 31 09:44:06 2022] DELETE /identity/v3/users/8d66d0950add4450bfe7c9833ce6c2e3 => generated 0 bytes in 349 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:44:07.251445 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5949ef2-6438-4b02-b310-eee964a13c89 None None] REQUEST_METHOD: `POST` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:44:07.251732 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5949ef2-6438-4b02-b310-eee964a13c89 None None] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:44:07.252009 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5949ef2-6438-4b02-b310-eee964a13c89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:44:07.310628 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.auth.core [None req-a5949ef2-6438-4b02-b310-eee964a13c89 None None] MFA Rules not processed for user `fd836cf590a145df9acf19cf3ae2ac56`. Rule list: `[]` (Enabled: `True`). {{(pid=88113) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:44:07.311834 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.fernet_utils [None req-a5949ef2-6438-4b02-b310-eee964a13c89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:44:07.392619 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=100 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:44:07.407057 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 756/1508] 10.222.0.13 () {58 vars in 958 bytes} [Wed Aug 31 09:44:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:44:07.419019 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-890e51c0-599e-4e0a-81d1-cfa1f3d6a044 admin admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:44:07.420530 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.fernet_utils [None req-890e51c0-599e-4e0a-81d1-cfa1f3d6a044 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88114) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:44:07.527306 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-890e51c0-599e-4e0a-81d1-cfa1f3d6a044 admin admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:44:07.583473 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:44:07.605683 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d3499b-bee8-464e-a3f8-3e8142ea2b87 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9bdaacd597442c2bcde2c555f2cc140', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '49548eb44d62415582af820ab66decee', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:44:07.606305 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d3499b-bee8-464e-a3f8-3e8142ea2b87 service neutron] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:44:07.606581 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d3499b-bee8-464e-a3f8-3e8142ea2b87 service neutron] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:44:07.606848 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d3499b-bee8-464e-a3f8-3e8142ea2b87 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:44:07.608457 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.fernet_utils [None req-a9d3499b-bee8-464e-a3f8-3e8142ea2b87 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88114) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:44:07.723574 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9d3499b-bee8-464e-a3f8-3e8142ea2b87 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:44:07.725002 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9d3499b-bee8-464e-a3f8-3e8142ea2b87 service neutron] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:44:07.743803 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 753/1509] 10.222.0.13 () {60 vars in 1365 bytes} [Wed Aug 31 09:44:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:44:08.209892 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5949ef2-6438-4b02-b310-eee964a13c89 None None] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:44:08.211227 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.fernet_utils [None req-a5949ef2-6438-4b02-b310-eee964a13c89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:44:08.222909 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5949ef2-6438-4b02-b310-eee964a13c89 None None] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:44:08.330462 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c328e120-6e2f-4087-824e-decb71538317 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:44:08.331100 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c328e120-6e2f-4087-824e-decb71538317 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:44:08.331375 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c328e120-6e2f-4087-824e-decb71538317 admin admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:44:08.331644 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c328e120-6e2f-4087-824e-decb71538317 admin admin] PATH_INFO: `/v3/projects/954fbb77916e4641ba59cd19f5924b63` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:44:08.341809 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c328e120-6e2f-4087-824e-decb71538317 admin admin] RBAC: Authorizing `identity:delete_project(project_id=954fbb77916e4641ba59cd19f5924b63)` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:44:08.342842 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: WARNING py.warnings [None req-c328e120-6e2f-4087-824e-decb71538317 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 31 09:44:08.342842 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: warnings.warn(msg) Aug 31 09:44:08.342842 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: Aug 31 09:44:08.344042 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c328e120-6e2f-4087-824e-decb71538317 admin admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:44:08.404568 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.notifications [None req-c328e120-6e2f-4087-824e-decb71538317 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '954fbb77916e4641ba59cd19f5924b63', 'request_id': 'req-c328e120-6e2f-4087-824e-decb71538317'} {{(pid=88113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:44:08.410603 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.notifications [None req-c328e120-6e2f-4087-824e-decb71538317 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '954fbb77916e4641ba59cd19f5924b63', 'request_id': 'req-c328e120-6e2f-4087-824e-decb71538317'} {{(pid=88113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:44:08.421538 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.notifications [None req-c328e120-6e2f-4087-824e-decb71538317 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '954fbb77916e4641ba59cd19f5924b63', 'request_id': 'req-c328e120-6e2f-4087-824e-decb71538317'} {{(pid=88113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:44:08.452589 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.notifications [None req-c328e120-6e2f-4087-824e-decb71538317 admin admin] The token cache is being invalidate because project 954fbb77916e4641ba59cd19f5924b63 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88113) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 09:44:08.452851 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.notifications [None req-c328e120-6e2f-4087-824e-decb71538317 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:44:08.453404 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.notifications [None req-c328e120-6e2f-4087-824e-decb71538317 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:44:08.454725 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 757/1510] 10.222.0.13 () {58 vars in 1233 bytes} [Wed Aug 31 09:44:08 2022] DELETE /identity/v3/projects/954fbb77916e4641ba59cd19f5924b63 => generated 0 bytes in 245 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:44:17.741484 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=62 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:44:18.450581 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=84,keystone:DELETE=7,keystone:INSERT=1 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:45:45.811366 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ab7a98-a75d-4a8d-a7e1-552422bb4d92 None None] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:45:45.811988 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ab7a98-a75d-4a8d-a7e1-552422bb4d92 None None] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:45:45.811988 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ab7a98-a75d-4a8d-a7e1-552422bb4d92 None None] PATH_INFO: `/` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:45:45.813246 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 754/1511] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 31 09:45:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:45:45.818338 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7759b4b-9579-4a76-b4df-c273df658e87 None None] REQUEST_METHOD: `POST` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:45:45.818840 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7759b4b-9579-4a76-b4df-c273df658e87 None None] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:45:45.818954 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7759b4b-9579-4a76-b4df-c273df658e87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:45:45.908234 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.auth.core [None req-e7759b4b-9579-4a76-b4df-c273df658e87 None None] MFA Rules not processed for user `fd836cf590a145df9acf19cf3ae2ac56`. Rule list: `[]` (Enabled: `True`). {{(pid=88113) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:45:45.909404 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.fernet_utils [None req-e7759b4b-9579-4a76-b4df-c273df658e87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:45:46.018680 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 758/1512] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 31 09:45:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:45:46.175815 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ab7a98-a75d-4a8d-a7e1-552422bb4d92 None None] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:45:46.178022 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.fernet_utils [None req-f4ab7a98-a75d-4a8d-a7e1-552422bb4d92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88114) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:45:46.296872 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ab7a98-a75d-4a8d-a7e1-552422bb4d92 None None] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:45:46.302325 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a7289d-cfb9-4d87-9499-f47e11fea44c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc076493239d4ccdbdc37c0369c49529', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '49548eb44d62415582af820ab66decee', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:45:46.302927 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a7289d-cfb9-4d87-9499-f47e11fea44c service nova] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:45:46.303207 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a7289d-cfb9-4d87-9499-f47e11fea44c service nova] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:45:46.303479 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a7289d-cfb9-4d87-9499-f47e11fea44c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:45:46.312105 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02a7289d-cfb9-4d87-9499-f47e11fea44c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:45:46.313709 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02a7289d-cfb9-4d87-9499-f47e11fea44c service nova] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:45:46.323110 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 755/1513] 10.222.0.13 () {60 vars in 1365 bytes} [Wed Aug 31 09:45:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 148 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:45:47.151452 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e4ad05-8c7e-498f-9233-73647a9bc4ee None None] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:45:47.152023 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e4ad05-8c7e-498f-9233-73647a9bc4ee None None] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:45:47.152023 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e4ad05-8c7e-498f-9233-73647a9bc4ee None None] PATH_INFO: `/` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:45:47.153359 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 759/1514] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 31 09:45:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:45:47.158233 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8435735-d0e4-4911-b2df-e23fd85c6762 None None] REQUEST_METHOD: `POST` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:45:47.158637 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8435735-d0e4-4911-b2df-e23fd85c6762 None None] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:45:47.158764 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8435735-d0e4-4911-b2df-e23fd85c6762 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:45:47.178569 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.auth.core [None req-e8435735-d0e4-4911-b2df-e23fd85c6762 None None] MFA Rules not processed for user `fd836cf590a145df9acf19cf3ae2ac56`. Rule list: `[]` (Enabled: `True`). {{(pid=88114) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:45:47.179761 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.fernet_utils [None req-e8435735-d0e4-4911-b2df-e23fd85c6762 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88114) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:45:47.187678 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 756/1515] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 31 09:45:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:45:47.345863 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e4ad05-8c7e-498f-9233-73647a9bc4ee None None] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:45:47.360861 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e4ad05-8c7e-498f-9233-73647a9bc4ee None None] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:45:47.366313 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b1a5106-e2ba-464c-83c1-0d042ca32569 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc076493239d4ccdbdc37c0369c49529', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '49548eb44d62415582af820ab66decee', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:45:47.366922 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b1a5106-e2ba-464c-83c1-0d042ca32569 service nova] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:45:47.367200 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b1a5106-e2ba-464c-83c1-0d042ca32569 service nova] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:45:47.367489 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b1a5106-e2ba-464c-83c1-0d042ca32569 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:45:47.376419 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b1a5106-e2ba-464c-83c1-0d042ca32569 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:45:47.378067 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b1a5106-e2ba-464c-83c1-0d042ca32569 service nova] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:45:47.388121 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 760/1516] 10.222.0.13 () {60 vars in 1365 bytes} [Wed Aug 31 09:45:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:45:47.689081 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dff4719-fd8d-44da-afc3-9eee146cda80 None None] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:45:47.689352 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dff4719-fd8d-44da-afc3-9eee146cda80 None None] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:45:47.689656 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dff4719-fd8d-44da-afc3-9eee146cda80 None None] PATH_INFO: `/v3` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:45:47.690923 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 757/1517] 10.222.0.64 () {54 vars in 925 bytes} [Wed Aug 31 09:45:47 2022] GET /identity/v3 => generated 255 bytes in 3 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:45:47.701954 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648177e9-2133-465f-a11f-2d6b5d90eb3f None None] REQUEST_METHOD: `POST` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:45:47.702272 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648177e9-2133-465f-a11f-2d6b5d90eb3f None None] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:45:47.702542 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648177e9-2133-465f-a11f-2d6b5d90eb3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:45:47.768004 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.auth.core [None req-648177e9-2133-465f-a11f-2d6b5d90eb3f None None] MFA Rules not processed for user `1ff3aff5ef444910be6dbb989d0e3194`. Rule list: `[]` (Enabled: `True`). {{(pid=88113) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:45:47.769221 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.fernet_utils [None req-648177e9-2133-465f-a11f-2d6b5d90eb3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:45:47.787179 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=100 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:45:47.848645 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 761/1518] 10.222.0.64 () {58 vars in 1015 bytes} [Wed Aug 31 09:45:47 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:45:57.173634 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=56 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:45:57.250249 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed51df0b-0a31-4cde-819b-87cbcf8ad1b3 None None] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:45:57.250493 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed51df0b-0a31-4cde-819b-87cbcf8ad1b3 None None] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:45:57.250712 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed51df0b-0a31-4cde-819b-87cbcf8ad1b3 None None] PATH_INFO: `/` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:45:57.252213 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 758/1519] 10.222.0.13 () {54 vars in 961 bytes} [Wed Aug 31 09:45:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:45:57.257789 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3ac2a5-ab76-4b0a-a9d4-ff955b41d710 None None] REQUEST_METHOD: `POST` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:45:57.258266 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3ac2a5-ab76-4b0a-a9d4-ff955b41d710 None None] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:45:57.258375 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3ac2a5-ab76-4b0a-a9d4-ff955b41d710 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:45:57.277602 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.auth.core [None req-0e3ac2a5-ab76-4b0a-a9d4-ff955b41d710 None None] MFA Rules not processed for user `1ff3aff5ef444910be6dbb989d0e3194`. Rule list: `[]` (Enabled: `True`). {{(pid=88113) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:45:57.278790 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.fernet_utils [None req-0e3ac2a5-ab76-4b0a-a9d4-ff955b41d710 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:45:57.287544 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 762/1520] 10.222.0.13 () {58 vars in 1058 bytes} [Wed Aug 31 09:45:57 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:45:57.295965 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed51df0b-0a31-4cde-819b-87cbcf8ad1b3 None None] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:45:57.310016 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed51df0b-0a31-4cde-819b-87cbcf8ad1b3 None None] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:45:57.315423 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ce278f-30ce-4bc8-8114-5ad828b833eb service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ff3aff5ef444910be6dbb989d0e3194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '49548eb44d62415582af820ab66decee', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:45:57.316010 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ce278f-30ce-4bc8-8114-5ad828b833eb service placement] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:45:57.316283 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ce278f-30ce-4bc8-8114-5ad828b833eb service placement] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:45:57.316550 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ce278f-30ce-4bc8-8114-5ad828b833eb service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:45:57.324658 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81ce278f-30ce-4bc8-8114-5ad828b833eb service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:45:57.326262 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81ce278f-30ce-4bc8-8114-5ad828b833eb service placement] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:45:57.335672 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 759/1521] 10.222.0.13 () {60 vars in 1365 bytes} [Wed Aug 31 09:45:57 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:45:58.623137 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51dc81b-d070-485e-a4ee-9f6d1738d31d None None] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:45:58.623743 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51dc81b-d070-485e-a4ee-9f6d1738d31d None None] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:45:58.623743 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51dc81b-d070-485e-a4ee-9f6d1738d31d None None] PATH_INFO: `/` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:45:58.625033 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 763/1522] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 31 09:45:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:45:58.629971 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af0f83e-684b-4584-92db-06a7fb08e9df None None] REQUEST_METHOD: `POST` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:45:58.630659 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af0f83e-684b-4584-92db-06a7fb08e9df None None] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:45:58.630798 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af0f83e-684b-4584-92db-06a7fb08e9df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:45:58.652418 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.auth.core [None req-5af0f83e-684b-4584-92db-06a7fb08e9df None None] MFA Rules not processed for user `fd836cf590a145df9acf19cf3ae2ac56`. Rule list: `[]` (Enabled: `True`). {{(pid=88114) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:45:58.653643 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.fernet_utils [None req-5af0f83e-684b-4584-92db-06a7fb08e9df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88114) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:45:58.661507 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 760/1523] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 31 09:45:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:45:58.818072 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c51dc81b-d070-485e-a4ee-9f6d1738d31d None None] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:45:58.832863 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c51dc81b-d070-485e-a4ee-9f6d1738d31d None None] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:45:58.839370 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eeca268-dcb8-4be8-b289-0cbc4a173e44 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc076493239d4ccdbdc37c0369c49529', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '49548eb44d62415582af820ab66decee', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:45:58.839980 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eeca268-dcb8-4be8-b289-0cbc4a173e44 service nova] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:45:58.840258 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eeca268-dcb8-4be8-b289-0cbc4a173e44 service nova] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:45:58.840530 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eeca268-dcb8-4be8-b289-0cbc4a173e44 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:45:58.849061 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eeca268-dcb8-4be8-b289-0cbc4a173e44 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:45:58.850728 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eeca268-dcb8-4be8-b289-0cbc4a173e44 service nova] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:45:58.861196 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 764/1524] 10.222.0.13 () {60 vars in 1365 bytes} [Wed Aug 31 09:45:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:45:59.692854 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe722f7-3d7e-4789-aca0-ff5a82b7229a None None] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:45:59.693447 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe722f7-3d7e-4789-aca0-ff5a82b7229a None None] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:45:59.693447 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe722f7-3d7e-4789-aca0-ff5a82b7229a None None] PATH_INFO: `/` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:45:59.694786 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 761/1525] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 31 09:45:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:45:59.700356 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914289ca-83b9-4829-88e4-5b1011732bc6 None None] REQUEST_METHOD: `POST` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:45:59.700816 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914289ca-83b9-4829-88e4-5b1011732bc6 None None] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:45:59.700923 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914289ca-83b9-4829-88e4-5b1011732bc6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:45:59.720243 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.auth.core [None req-914289ca-83b9-4829-88e4-5b1011732bc6 None None] MFA Rules not processed for user `fd836cf590a145df9acf19cf3ae2ac56`. Rule list: `[]` (Enabled: `True`). {{(pid=88113) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:45:59.721474 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.fernet_utils [None req-914289ca-83b9-4829-88e4-5b1011732bc6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:45:59.729368 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 765/1526] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 31 09:45:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:45:59.883127 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe722f7-3d7e-4789-aca0-ff5a82b7229a None None] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:45:59.897984 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe722f7-3d7e-4789-aca0-ff5a82b7229a None None] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:45:59.904113 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ed6ac6-4ae4-4917-9be4-6d456e42cc56 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc076493239d4ccdbdc37c0369c49529', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '49548eb44d62415582af820ab66decee', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:45:59.904706 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ed6ac6-4ae4-4917-9be4-6d456e42cc56 service nova] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:45:59.904985 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ed6ac6-4ae4-4917-9be4-6d456e42cc56 service nova] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:45:59.905256 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ed6ac6-4ae4-4917-9be4-6d456e42cc56 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:45:59.913334 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97ed6ac6-4ae4-4917-9be4-6d456e42cc56 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:45:59.914968 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97ed6ac6-4ae4-4917-9be4-6d456e42cc56 service nova] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:45:59.925182 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 762/1527] 10.222.0.13 () {60 vars in 1365 bytes} [Wed Aug 31 09:45:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:46:04.266025 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ece0a6-0ec2-40c4-b084-59ff7082037b None None] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:46:04.266564 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ece0a6-0ec2-40c4-b084-59ff7082037b None None] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:46:04.266564 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ece0a6-0ec2-40c4-b084-59ff7082037b None None] PATH_INFO: `/` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:46:04.267857 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 766/1528] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 31 09:46:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:46:04.273043 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b45e8d8-b73c-4157-808d-4a431caf0a66 None None] REQUEST_METHOD: `POST` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:46:04.273472 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b45e8d8-b73c-4157-808d-4a431caf0a66 None None] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:46:04.273627 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b45e8d8-b73c-4157-808d-4a431caf0a66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:46:04.293015 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.auth.core [None req-6b45e8d8-b73c-4157-808d-4a431caf0a66 None None] MFA Rules not processed for user `fd836cf590a145df9acf19cf3ae2ac56`. Rule list: `[]` (Enabled: `True`). {{(pid=88114) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:46:04.294239 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.fernet_utils [None req-6b45e8d8-b73c-4157-808d-4a431caf0a66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88114) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:46:04.301922 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 763/1529] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 31 09:46:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:46:04.587898 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ece0a6-0ec2-40c4-b084-59ff7082037b None None] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:46:04.589712 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.fernet_utils [None req-54ece0a6-0ec2-40c4-b084-59ff7082037b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:46:04.667362 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ece0a6-0ec2-40c4-b084-59ff7082037b None None] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:46:04.670907 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc3cfc0f-6fb9-4032-b34b-71e05863b216 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9bdaacd597442c2bcde2c555f2cc140', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '49548eb44d62415582af820ab66decee', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:46:04.671507 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3cfc0f-6fb9-4032-b34b-71e05863b216 service neutron] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:46:04.671781 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3cfc0f-6fb9-4032-b34b-71e05863b216 service neutron] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:46:04.672048 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3cfc0f-6fb9-4032-b34b-71e05863b216 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:46:04.680828 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc3cfc0f-6fb9-4032-b34b-71e05863b216 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:46:04.682326 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc3cfc0f-6fb9-4032-b34b-71e05863b216 service neutron] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:46:04.691496 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 767/1530] 10.222.0.13 () {60 vars in 1365 bytes} [Wed Aug 31 09:46:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:46:06.615502 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d4ca23-30ed-4cc9-8c78-3daf317e5086 None None] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:46:06.616083 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d4ca23-30ed-4cc9-8c78-3daf317e5086 None None] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:46:06.616083 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d4ca23-30ed-4cc9-8c78-3daf317e5086 None None] PATH_INFO: `/` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:46:06.617509 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 764/1531] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 31 09:46:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:46:06.624422 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36276dc0-2586-43b1-90dd-67159a619e47 None None] REQUEST_METHOD: `POST` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:46:06.624870 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36276dc0-2586-43b1-90dd-67159a619e47 None None] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:46:06.624970 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36276dc0-2586-43b1-90dd-67159a619e47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:46:06.645575 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.auth.core [None req-36276dc0-2586-43b1-90dd-67159a619e47 None None] MFA Rules not processed for user `fd836cf590a145df9acf19cf3ae2ac56`. Rule list: `[]` (Enabled: `True`). {{(pid=88113) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:46:06.646772 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.fernet_utils [None req-36276dc0-2586-43b1-90dd-67159a619e47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:46:06.654709 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 768/1532] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 31 09:46:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:46:06.946924 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d4ca23-30ed-4cc9-8c78-3daf317e5086 None None] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:46:06.948676 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.fernet_utils [None req-e7d4ca23-30ed-4cc9-8c78-3daf317e5086 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88114) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:46:06.961633 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d4ca23-30ed-4cc9-8c78-3daf317e5086 None None] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:46:06.965784 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f3d463d-9f27-4e57-aa13-df5efd7acbd7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9bdaacd597442c2bcde2c555f2cc140', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '49548eb44d62415582af820ab66decee', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:46:06.966373 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3d463d-9f27-4e57-aa13-df5efd7acbd7 service neutron] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:46:06.966650 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3d463d-9f27-4e57-aa13-df5efd7acbd7 service neutron] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:46:06.966919 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3d463d-9f27-4e57-aa13-df5efd7acbd7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:46:06.976117 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f3d463d-9f27-4e57-aa13-df5efd7acbd7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:46:06.977581 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f3d463d-9f27-4e57-aa13-df5efd7acbd7 service neutron] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:46:06.987964 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 765/1533] 10.222.0.13 () {60 vars in 1365 bytes} [Wed Aug 31 09:46:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:46:10.720828 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f124c4c1-fcb3-4acd-a561-f8977e19b6d3 None None] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:46:10.721515 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f124c4c1-fcb3-4acd-a561-f8977e19b6d3 None None] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:46:10.721515 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f124c4c1-fcb3-4acd-a561-f8977e19b6d3 None None] PATH_INFO: `/` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:46:10.722756 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 769/1534] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 31 09:46:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:46:10.728608 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd5b9fb-526b-4721-a17c-9a83aec2aed7 None None] REQUEST_METHOD: `POST` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:46:10.729066 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd5b9fb-526b-4721-a17c-9a83aec2aed7 None None] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:46:10.729258 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd5b9fb-526b-4721-a17c-9a83aec2aed7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:46:10.749572 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.auth.core [None req-8fd5b9fb-526b-4721-a17c-9a83aec2aed7 None None] MFA Rules not processed for user `fd836cf590a145df9acf19cf3ae2ac56`. Rule list: `[]` (Enabled: `True`). {{(pid=88114) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:46:10.750797 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.fernet_utils [None req-8fd5b9fb-526b-4721-a17c-9a83aec2aed7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88114) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:46:10.758895 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 766/1535] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 31 09:46:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:46:11.311681 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f124c4c1-fcb3-4acd-a561-f8977e19b6d3 None None] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:46:11.313401 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.fernet_utils [None req-f124c4c1-fcb3-4acd-a561-f8977e19b6d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:46:11.379919 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=100 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:46:11.393947 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f124c4c1-fcb3-4acd-a561-f8977e19b6d3 None None] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:46:11.397671 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b4ecf28-8c68-4814-80b6-62174c4b4dcf service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d52b69c8e94441dacfcff944326f169', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '49548eb44d62415582af820ab66decee', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:46:11.398278 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4ecf28-8c68-4814-80b6-62174c4b4dcf service cinder] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:46:11.398556 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4ecf28-8c68-4814-80b6-62174c4b4dcf service cinder] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:46:11.398827 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4ecf28-8c68-4814-80b6-62174c4b4dcf service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:46:11.408642 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b4ecf28-8c68-4814-80b6-62174c4b4dcf service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:46:11.410123 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b4ecf28-8c68-4814-80b6-62174c4b4dcf service cinder] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:46:11.420441 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 770/1536] 10.222.0.13 () {60 vars in 1365 bytes} [Wed Aug 31 09:46:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:46:13.388438 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b92162f-1a79-401a-bd7d-e96fb1d82ec5 None None] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:46:13.388975 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b92162f-1a79-401a-bd7d-e96fb1d82ec5 None None] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:46:13.389366 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b92162f-1a79-401a-bd7d-e96fb1d82ec5 None None] PATH_INFO: `/` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:46:13.391324 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 767/1537] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 31 09:46:13 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:46:13.397025 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9632ecd0-8dd8-4d60-b20f-6aba3eae5626 None None] REQUEST_METHOD: `POST` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:46:13.397461 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9632ecd0-8dd8-4d60-b20f-6aba3eae5626 None None] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:46:13.397609 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9632ecd0-8dd8-4d60-b20f-6aba3eae5626 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:46:13.417612 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.auth.core [None req-9632ecd0-8dd8-4d60-b20f-6aba3eae5626 None None] MFA Rules not processed for user `fd836cf590a145df9acf19cf3ae2ac56`. Rule list: `[]` (Enabled: `True`). {{(pid=88113) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:46:13.418834 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.fernet_utils [None req-9632ecd0-8dd8-4d60-b20f-6aba3eae5626 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:46:13.427091 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 771/1538] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 31 09:46:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:46:13.985623 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b92162f-1a79-401a-bd7d-e96fb1d82ec5 None None] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:46:13.987301 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.fernet_utils [None req-1b92162f-1a79-401a-bd7d-e96fb1d82ec5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88114) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:46:14.002246 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b92162f-1a79-401a-bd7d-e96fb1d82ec5 None None] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:46:14.006074 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93705e6c-2820-4c6a-a707-7f49c57e746a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d52b69c8e94441dacfcff944326f169', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '49548eb44d62415582af820ab66decee', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:46:14.006669 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93705e6c-2820-4c6a-a707-7f49c57e746a service cinder] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:46:14.006943 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93705e6c-2820-4c6a-a707-7f49c57e746a service cinder] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:46:14.007213 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93705e6c-2820-4c6a-a707-7f49c57e746a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:46:14.016579 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93705e6c-2820-4c6a-a707-7f49c57e746a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:46:14.018051 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93705e6c-2820-4c6a-a707-7f49c57e746a service cinder] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:46:14.027536 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 768/1539] 10.222.0.13 () {60 vars in 1365 bytes} [Wed Aug 31 09:46:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:46:23.412341 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=11 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:46:24.023259 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=33 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:47:43.592828 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2fbbf0-7d79-40e1-b229-7b79762571e5 None None] REQUEST_METHOD: `POST` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:47:43.593545 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2fbbf0-7d79-40e1-b229-7b79762571e5 None None] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:47:43.593545 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2fbbf0-7d79-40e1-b229-7b79762571e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:47:43.618333 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.auth.core [None req-ef2fbbf0-7d79-40e1-b229-7b79762571e5 None None] MFA Rules not processed for user `fd836cf590a145df9acf19cf3ae2ac56`. Rule list: `[]` (Enabled: `True`). {{(pid=88113) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:47:43.619633 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.fernet_utils [None req-ef2fbbf0-7d79-40e1-b229-7b79762571e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:47:43.636835 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 772/1540] 10.222.0.13 () {58 vars in 958 bytes} [Wed Aug 31 09:47:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:47:43.646110 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7319da68-5944-45dd-8b4e-1138fcfd6617 None None] REQUEST_METHOD: `POST` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:47:43.647286 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7319da68-5944-45dd-8b4e-1138fcfd6617 None None] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:47:43.647424 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7319da68-5944-45dd-8b4e-1138fcfd6617 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:47:43.682717 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.auth.core [None req-7319da68-5944-45dd-8b4e-1138fcfd6617 None None] MFA Rules not processed for user `fd836cf590a145df9acf19cf3ae2ac56`. Rule list: `[]` (Enabled: `True`). {{(pid=88114) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:47:43.683989 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.fernet_utils [None req-7319da68-5944-45dd-8b4e-1138fcfd6617 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88114) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:47:43.697613 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 769/1541] 10.222.0.13 () {58 vars in 958 bytes} [Wed Aug 31 09:47:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:47:43.702172 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef2fbbf0-7d79-40e1-b229-7b79762571e5 None None] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:47:43.715906 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef2fbbf0-7d79-40e1-b229-7b79762571e5 None None] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:47:43.724212 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94da3572-664b-40d6-ad1b-34650a3b2212 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:47:43.726188 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94da3572-664b-40d6-ad1b-34650a3b2212 admin admin] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:47:43.726893 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94da3572-664b-40d6-ad1b-34650a3b2212 admin admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:47:43.727401 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94da3572-664b-40d6-ad1b-34650a3b2212 admin admin] PATH_INFO: `/v3/domains` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:47:43.728373 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94da3572-664b-40d6-ad1b-34650a3b2212 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88113) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:47:43.728922 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94da3572-664b-40d6-ad1b-34650a3b2212 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:47:43.731172 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94da3572-664b-40d6-ad1b-34650a3b2212 admin admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:47:43.753561 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 773/1542] 10.222.0.13 () {58 vars in 1153 bytes} [Wed Aug 31 09:47:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:47:43.762521 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7319da68-5944-45dd-8b4e-1138fcfd6617 None None] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:47:43.773470 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7319da68-5944-45dd-8b4e-1138fcfd6617 None None] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:47:43.786212 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-840a311c-4efe-4e60-918d-17c7cb85c0ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:47:43.786884 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840a311c-4efe-4e60-918d-17c7cb85c0ae admin admin] REQUEST_METHOD: `POST` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:47:43.787167 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840a311c-4efe-4e60-918d-17c7cb85c0ae admin admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:47:43.787427 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840a311c-4efe-4e60-918d-17c7cb85c0ae admin admin] PATH_INFO: `/v3/projects` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:47:43.788248 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-840a311c-4efe-4e60-918d-17c7cb85c0ae admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:47:43.790417 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-840a311c-4efe-4e60-918d-17c7cb85c0ae admin admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:47:43.850148 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 770/1543] 10.222.0.13 () {60 vars in 1153 bytes} [Wed Aug 31 09:47:43 2022] POST /identity/v3/projects => generated 375 bytes in 88 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 09:47:43.854408 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94da3572-664b-40d6-ad1b-34650a3b2212 admin admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:47:43.869674 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94da3572-664b-40d6-ad1b-34650a3b2212 admin admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:47:43.925653 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e11d01-ef76-420a-a2d3-e8d72f802869 None None] REQUEST_METHOD: `POST` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:47:43.926022 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e11d01-ef76-420a-a2d3-e8d72f802869 None None] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:47:43.926207 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e11d01-ef76-420a-a2d3-e8d72f802869 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:47:43.957355 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.auth.core [None req-c6e11d01-ef76-420a-a2d3-e8d72f802869 None None] MFA Rules not processed for user `fd836cf590a145df9acf19cf3ae2ac56`. Rule list: `[]` (Enabled: `True`). {{(pid=88114) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:47:43.961281 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.fernet_utils [None req-c6e11d01-ef76-420a-a2d3-e8d72f802869 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88114) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:47:43.984196 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d95736ec-dc98-47ab-a8e0-200640a65cbb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:47:43.984830 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95736ec-dc98-47ab-a8e0-200640a65cbb admin admin] REQUEST_METHOD: `POST` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:47:43.985111 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95736ec-dc98-47ab-a8e0-200640a65cbb admin admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:47:43.985404 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95736ec-dc98-47ab-a8e0-200640a65cbb admin admin] PATH_INFO: `/v3/users` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:47:43.986078 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d95736ec-dc98-47ab-a8e0-200640a65cbb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:47:43.993486 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d95736ec-dc98-47ab-a8e0-200640a65cbb admin admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:47:44.043848 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 774/1544] 10.222.0.13 () {60 vars in 1144 bytes} [Wed Aug 31 09:47:43 2022] POST /identity/v3/users => generated 288 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 09:47:44.045652 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af71db63-78f0-49da-9ae7-21d158a36fd7 None None] REQUEST_METHOD: `POST` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:47:44.045944 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af71db63-78f0-49da-9ae7-21d158a36fd7 None None] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:47:44.046219 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af71db63-78f0-49da-9ae7-21d158a36fd7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:47:44.095621 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.auth.core [None req-af71db63-78f0-49da-9ae7-21d158a36fd7 None None] MFA Rules not processed for user `fd836cf590a145df9acf19cf3ae2ac56`. Rule list: `[]` (Enabled: `True`). {{(pid=88113) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:47:44.096855 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.fernet_utils [None req-af71db63-78f0-49da-9ae7-21d158a36fd7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:47:44.108061 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 771/1545] 10.222.0.13 () {58 vars in 958 bytes} [Wed Aug 31 09:47:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 09:47:44.108883 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e11d01-ef76-420a-a2d3-e8d72f802869 None None] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:47:44.118963 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 775/1546] 10.222.0.13 () {58 vars in 958 bytes} [Wed Aug 31 09:47:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 75 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:47:44.121473 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33642e87-81b0-4613-89db-17fd0d4e1a1c None None] REQUEST_METHOD: `POST` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:47:44.121772 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33642e87-81b0-4613-89db-17fd0d4e1a1c None None] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:47:44.122054 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33642e87-81b0-4613-89db-17fd0d4e1a1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:47:44.134394 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e11d01-ef76-420a-a2d3-e8d72f802869 None None] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:47:44.152323 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2478ef34-c76f-4bb7-b8e8-3129315530e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:47:44.152956 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2478ef34-c76f-4bb7-b8e8-3129315530e1 admin admin] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:47:44.153437 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2478ef34-c76f-4bb7-b8e8-3129315530e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:47:44.153717 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2478ef34-c76f-4bb7-b8e8-3129315530e1 admin admin] PATH_INFO: `/v3/roles` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:47:44.158467 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.auth.core [None req-33642e87-81b0-4613-89db-17fd0d4e1a1c None None] MFA Rules not processed for user `fd836cf590a145df9acf19cf3ae2ac56`. Rule list: `[]` (Enabled: `True`). {{(pid=88113) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:47:44.160891 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2478ef34-c76f-4bb7-b8e8-3129315530e1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:47:44.164460 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2478ef34-c76f-4bb7-b8e8-3129315530e1 admin admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:47:44.168083 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.fernet_utils [None req-33642e87-81b0-4613-89db-17fd0d4e1a1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:47:44.178156 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 772/1547] 10.222.0.13 () {58 vars in 1122 bytes} [Wed Aug 31 09:47:44 2022] GET /identity/v3/roles => generated 1420 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 09:47:44.179523 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835ba1cc-a3cb-43d8-b9c3-bcaca95bed8f None None] REQUEST_METHOD: `POST` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:47:44.179809 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835ba1cc-a3cb-43d8-b9c3-bcaca95bed8f None None] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:47:44.180148 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835ba1cc-a3cb-43d8-b9c3-bcaca95bed8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:47:44.183697 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 776/1548] 10.222.0.13 () {58 vars in 958 bytes} [Wed Aug 31 09:47:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 64 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:47:44.184430 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffee9643-b8fd-480f-ace5-a87c2a154914 None None] REQUEST_METHOD: `POST` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:47:44.184703 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffee9643-b8fd-480f-ace5-a87c2a154914 None None] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:47:44.184994 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffee9643-b8fd-480f-ace5-a87c2a154914 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:47:44.204361 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.auth.core [None req-835ba1cc-a3cb-43d8-b9c3-bcaca95bed8f None None] MFA Rules not processed for user `fd836cf590a145df9acf19cf3ae2ac56`. Rule list: `[]` (Enabled: `True`). {{(pid=88114) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:47:44.205938 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.fernet_utils [None req-835ba1cc-a3cb-43d8-b9c3-bcaca95bed8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88114) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:47:44.212517 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.auth.core [None req-ffee9643-b8fd-480f-ace5-a87c2a154914 None None] MFA Rules not processed for user `fd836cf590a145df9acf19cf3ae2ac56`. Rule list: `[]` (Enabled: `True`). {{(pid=88113) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:47:44.213733 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.fernet_utils [None req-ffee9643-b8fd-480f-ace5-a87c2a154914 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:47:44.217590 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 773/1549] 10.222.0.13 () {58 vars in 958 bytes} [Wed Aug 31 09:47:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:47:44.219123 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5756eafa-ba5f-4eb7-b6af-697d7b8250f5 None None] REQUEST_METHOD: `POST` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:47:44.219591 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5756eafa-ba5f-4eb7-b6af-697d7b8250f5 None None] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:47:44.220037 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5756eafa-ba5f-4eb7-b6af-697d7b8250f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:47:44.227136 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 777/1550] 10.222.0.13 () {58 vars in 958 bytes} [Wed Aug 31 09:47:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:47:44.228566 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812f1012-a5b3-4f2e-b410-bec3fead5243 None None] REQUEST_METHOD: `POST` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:47:44.228850 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812f1012-a5b3-4f2e-b410-bec3fead5243 None None] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:47:44.229122 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812f1012-a5b3-4f2e-b410-bec3fead5243 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:47:44.246635 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.auth.core [None req-5756eafa-ba5f-4eb7-b6af-697d7b8250f5 None None] MFA Rules not processed for user `fd836cf590a145df9acf19cf3ae2ac56`. Rule list: `[]` (Enabled: `True`). {{(pid=88114) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:47:44.248308 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.fernet_utils [None req-5756eafa-ba5f-4eb7-b6af-697d7b8250f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88114) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:47:44.252795 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.auth.core [None req-812f1012-a5b3-4f2e-b410-bec3fead5243 None None] MFA Rules not processed for user `fd836cf590a145df9acf19cf3ae2ac56`. Rule list: `[]` (Enabled: `True`). {{(pid=88113) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:47:44.254280 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.fernet_utils [None req-812f1012-a5b3-4f2e-b410-bec3fead5243 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:47:44.259063 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 774/1551] 10.222.0.13 () {58 vars in 958 bytes} [Wed Aug 31 09:47:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:47:44.260198 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5756eafa-ba5f-4eb7-b6af-697d7b8250f5 None None] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:47:44.269840 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 778/1552] 10.222.0.13 () {58 vars in 958 bytes} [Wed Aug 31 09:47:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:47:44.271243 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de77650f-90af-408a-a163-0464964cf094 None None] REQUEST_METHOD: `POST` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:47:44.271595 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de77650f-90af-408a-a163-0464964cf094 None None] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:47:44.272263 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de77650f-90af-408a-a163-0464964cf094 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:47:44.276991 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5756eafa-ba5f-4eb7-b6af-697d7b8250f5 None None] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:47:44.286587 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-967759c7-10a5-45ec-b854-c4d38ba9813b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:47:44.287470 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967759c7-10a5-45ec-b854-c4d38ba9813b admin admin] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:47:44.287746 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967759c7-10a5-45ec-b854-c4d38ba9813b admin admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:47:44.288035 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967759c7-10a5-45ec-b854-c4d38ba9813b admin admin] PATH_INFO: `/v3/roles` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:47:44.288858 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-967759c7-10a5-45ec-b854-c4d38ba9813b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:47:44.290717 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-967759c7-10a5-45ec-b854-c4d38ba9813b admin admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:47:44.303092 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 775/1553] 10.222.0.13 () {58 vars in 1122 bytes} [Wed Aug 31 09:47:44 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 09:47:44.304376 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.auth.core [None req-de77650f-90af-408a-a163-0464964cf094 None None] MFA Rules not processed for user `fd836cf590a145df9acf19cf3ae2ac56`. Rule list: `[]` (Enabled: `True`). {{(pid=88113) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:47:44.304501 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-967759c7-10a5-45ec-b854-c4d38ba9813b admin admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:47:44.305207 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.fernet_utils [None req-de77650f-90af-408a-a163-0464964cf094 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:47:44.315205 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 779/1554] 10.222.0.13 () {58 vars in 958 bytes} [Wed Aug 31 09:47:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:47:44.316441 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de77650f-90af-408a-a163-0464964cf094 None None] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:47:44.317522 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-967759c7-10a5-45ec-b854-c4d38ba9813b admin admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:47:44.325129 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ddfe6c-312e-4a64-a0be-d7f7b27fe99b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:47:44.325771 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ddfe6c-312e-4a64-a0be-d7f7b27fe99b admin admin] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:47:44.326045 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ddfe6c-312e-4a64-a0be-d7f7b27fe99b admin admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:47:44.326312 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ddfe6c-312e-4a64-a0be-d7f7b27fe99b admin admin] PATH_INFO: `/v3/domains` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:47:44.326958 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8ddfe6c-312e-4a64-a0be-d7f7b27fe99b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88114) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:47:44.327253 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8ddfe6c-312e-4a64-a0be-d7f7b27fe99b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:47:44.328259 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: WARNING py.warnings [None req-a8ddfe6c-312e-4a64-a0be-d7f7b27fe99b admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 31 09:47:44.328259 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: warnings.warn(msg) Aug 31 09:47:44.328259 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: Aug 31 09:47:44.328853 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de77650f-90af-408a-a163-0464964cf094 None None] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:47:44.329173 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8ddfe6c-312e-4a64-a0be-d7f7b27fe99b admin admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:47:44.333657 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc00f17-38a9-45a0-b469-3ec7d8a57213 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:47:44.334308 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc00f17-38a9-45a0-b469-3ec7d8a57213 admin admin] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:47:44.334589 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc00f17-38a9-45a0-b469-3ec7d8a57213 admin admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:47:44.334854 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc00f17-38a9-45a0-b469-3ec7d8a57213 admin admin] PATH_INFO: `/v3/domains` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:47:44.335499 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dc00f17-38a9-45a0-b469-3ec7d8a57213 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88113) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:47:44.335800 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dc00f17-38a9-45a0-b469-3ec7d8a57213 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:47:44.337434 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dc00f17-38a9-45a0-b469-3ec7d8a57213 admin admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:47:44.348260 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 776/1555] 10.222.0.13 () {58 vars in 1153 bytes} [Wed Aug 31 09:47:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:47:44.348260 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2528f135-b877-4d71-84d2-def0812fc896 None None] REQUEST_METHOD: `POST` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:47:44.348260 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2528f135-b877-4d71-84d2-def0812fc896 None None] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:47:44.348260 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2528f135-b877-4d71-84d2-def0812fc896 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:47:44.355016 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 780/1556] 10.222.0.13 () {58 vars in 1153 bytes} [Wed Aug 31 09:47:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:47:44.356173 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c878115-1ffe-4fb2-bf7a-b9d7a5900d6f None None] REQUEST_METHOD: `POST` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:47:44.356537 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c878115-1ffe-4fb2-bf7a-b9d7a5900d6f None None] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:47:44.357223 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c878115-1ffe-4fb2-bf7a-b9d7a5900d6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:47:44.363260 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.auth.core [None req-2528f135-b877-4d71-84d2-def0812fc896 None None] MFA Rules not processed for user `fd836cf590a145df9acf19cf3ae2ac56`. Rule list: `[]` (Enabled: `True`). {{(pid=88114) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:47:44.364308 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.fernet_utils [None req-2528f135-b877-4d71-84d2-def0812fc896 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88114) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:47:44.373489 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 777/1557] 10.222.0.13 () {58 vars in 958 bytes} [Wed Aug 31 09:47:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:47:44.374436 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006648be-3e53-490e-a27f-2f652a7696df None None] REQUEST_METHOD: `POST` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:47:44.374716 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006648be-3e53-490e-a27f-2f652a7696df None None] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:47:44.375090 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006648be-3e53-490e-a27f-2f652a7696df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:47:44.379713 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.auth.core [None req-6c878115-1ffe-4fb2-bf7a-b9d7a5900d6f None None] MFA Rules not processed for user `fd836cf590a145df9acf19cf3ae2ac56`. Rule list: `[]` (Enabled: `True`). {{(pid=88113) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:47:44.382123 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.fernet_utils [None req-6c878115-1ffe-4fb2-bf7a-b9d7a5900d6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:47:44.390142 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 781/1558] 10.222.0.13 () {58 vars in 958 bytes} [Wed Aug 31 09:47:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:47:44.390932 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c878115-1ffe-4fb2-bf7a-b9d7a5900d6f None None] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:47:44.396459 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.auth.core [None req-006648be-3e53-490e-a27f-2f652a7696df None None] MFA Rules not processed for user `fd836cf590a145df9acf19cf3ae2ac56`. Rule list: `[]` (Enabled: `True`). {{(pid=88114) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:47:44.397716 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.fernet_utils [None req-006648be-3e53-490e-a27f-2f652a7696df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88114) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:47:44.401552 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c878115-1ffe-4fb2-bf7a-b9d7a5900d6f None None] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:47:44.405645 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-878d260c-c4b3-4cc6-909e-219a65f84959 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:47:44.406238 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 778/1559] 10.222.0.13 () {58 vars in 958 bytes} [Wed Aug 31 09:47:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:47:44.406371 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878d260c-c4b3-4cc6-909e-219a65f84959 admin admin] REQUEST_METHOD: `PUT` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:47:44.406574 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878d260c-c4b3-4cc6-909e-219a65f84959 admin admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:47:44.406847 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878d260c-c4b3-4cc6-909e-219a65f84959 admin admin] PATH_INFO: `/v3/projects/1073c2fa1d8a4810a4846b7ccd823a19/users/bf41ca204722491abce3c27f2ff87126/roles/6c99f5d9f1f243eaa134d8540f21095d` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:47:44.407026 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-006648be-3e53-490e-a27f-2f652a7696df None None] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:47:44.418211 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-006648be-3e53-490e-a27f-2f652a7696df None None] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:47:44.423234 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-974ea9ed-078b-4fab-8606-120941666d33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:47:44.423846 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974ea9ed-078b-4fab-8606-120941666d33 admin admin] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:47:44.424125 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974ea9ed-078b-4fab-8606-120941666d33 admin admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:47:44.424507 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974ea9ed-078b-4fab-8606-120941666d33 admin admin] PATH_INFO: `/v3/domains` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:47:44.425162 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-974ea9ed-078b-4fab-8606-120941666d33 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88114) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:47:44.425496 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-974ea9ed-078b-4fab-8606-120941666d33 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:47:44.426517 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-878d260c-c4b3-4cc6-909e-219a65f84959 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1073c2fa1d8a4810a4846b7ccd823a19, user_id=bf41ca204722491abce3c27f2ff87126, role_id=6c99f5d9f1f243eaa134d8540f21095d)` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:47:44.427112 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-974ea9ed-078b-4fab-8606-120941666d33 admin admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:47:44.428171 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: WARNING py.warnings [None req-878d260c-c4b3-4cc6-909e-219a65f84959 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 31 09:47:44.428171 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: warnings.warn(msg) Aug 31 09:47:44.428171 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: Aug 31 09:47:44.430252 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-878d260c-c4b3-4cc6-909e-219a65f84959 admin admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:47:44.440093 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 779/1560] 10.222.0.13 () {58 vars in 1153 bytes} [Wed Aug 31 09:47:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:47:44.441514 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a8dd3a-c75e-448e-bbcd-0c9412804ea9 None None] REQUEST_METHOD: `POST` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:47:44.441806 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a8dd3a-c75e-448e-bbcd-0c9412804ea9 None None] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:47:44.442083 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a8dd3a-c75e-448e-bbcd-0c9412804ea9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:47:44.445798 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 782/1561] 10.222.0.13 () {60 vars in 1483 bytes} [Wed Aug 31 09:47:44 2022] PUT /identity/v3/projects/1073c2fa1d8a4810a4846b7ccd823a19/users/bf41ca204722491abce3c27f2ff87126/roles/6c99f5d9f1f243eaa134d8540f21095d => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:47:44.446399 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-878d260c-c4b3-4cc6-909e-219a65f84959 admin admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:47:44.468663 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.auth.core [None req-a4a8dd3a-c75e-448e-bbcd-0c9412804ea9 None None] MFA Rules not processed for user `fd836cf590a145df9acf19cf3ae2ac56`. Rule list: `[]` (Enabled: `True`). {{(pid=88114) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:47:44.469937 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.fernet_utils [None req-a4a8dd3a-c75e-448e-bbcd-0c9412804ea9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88114) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:47:44.509038 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:47:44.519158 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:47:44.522038 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-878d260c-c4b3-4cc6-909e-219a65f84959 admin admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:47:44.527457 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c5e711a-cbef-487a-91a8-7c4e1915fff6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:47:44.528069 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5e711a-cbef-487a-91a8-7c4e1915fff6 admin admin] REQUEST_METHOD: `POST` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:47:44.528347 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5e711a-cbef-487a-91a8-7c4e1915fff6 admin admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:47:44.528617 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5e711a-cbef-487a-91a8-7c4e1915fff6 admin admin] PATH_INFO: `/v3/projects` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:47:44.529252 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c5e711a-cbef-487a-91a8-7c4e1915fff6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:47:44.530314 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: WARNING py.warnings [None req-6c5e711a-cbef-487a-91a8-7c4e1915fff6 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 31 09:47:44.530314 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: warnings.warn(msg) Aug 31 09:47:44.530314 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: Aug 31 09:47:44.531736 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c5e711a-cbef-487a-91a8-7c4e1915fff6 admin admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:47:44.541453 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 780/1562] 10.222.0.13 () {58 vars in 958 bytes} [Wed Aug 31 09:47:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:47:44.541453 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a8dd3a-c75e-448e-bbcd-0c9412804ea9 None None] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:47:44.549029 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a8dd3a-c75e-448e-bbcd-0c9412804ea9 None None] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:47:44.555555 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0da86db-5df4-4363-a4c9-20e2d6aa6419 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:47:44.556172 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0da86db-5df4-4363-a4c9-20e2d6aa6419 admin admin] REQUEST_METHOD: `POST` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:47:44.556447 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0da86db-5df4-4363-a4c9-20e2d6aa6419 admin admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:47:44.556708 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0da86db-5df4-4363-a4c9-20e2d6aa6419 admin admin] PATH_INFO: `/v3/projects` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:47:44.557338 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0da86db-5df4-4363-a4c9-20e2d6aa6419 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:47:44.559297 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0da86db-5df4-4363-a4c9-20e2d6aa6419 admin admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:47:44.560870 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 783/1563] 10.222.0.13 () {60 vars in 1153 bytes} [Wed Aug 31 09:47:44 2022] POST /identity/v3/projects => generated 371 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 09:47:44.561763 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c5e711a-cbef-487a-91a8-7c4e1915fff6 admin admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:47:44.591628 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 781/1564] 10.222.0.13 () {60 vars in 1153 bytes} [Wed Aug 31 09:47:44 2022] POST /identity/v3/projects => generated 391 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 09:47:44.591628 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0da86db-5df4-4363-a4c9-20e2d6aa6419 admin admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:47:44.633128 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c5e711a-cbef-487a-91a8-7c4e1915fff6 admin admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:47:44.654749 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0da86db-5df4-4363-a4c9-20e2d6aa6419 admin admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:47:44.660044 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b43cbf-a71a-4699-bd41-55f967a0fb07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:47:44.661053 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b43cbf-a71a-4699-bd41-55f967a0fb07 admin admin] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:47:44.661522 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b43cbf-a71a-4699-bd41-55f967a0fb07 admin admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:47:44.661792 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b43cbf-a71a-4699-bd41-55f967a0fb07 admin admin] PATH_INFO: `/v3/domains` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:47:44.663556 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3b43cbf-a71a-4699-bd41-55f967a0fb07 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88114) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:47:44.663868 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3b43cbf-a71a-4699-bd41-55f967a0fb07 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:47:44.665503 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3b43cbf-a71a-4699-bd41-55f967a0fb07 admin admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:47:44.678185 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 782/1565] 10.222.0.13 () {58 vars in 1153 bytes} [Wed Aug 31 09:47:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:47:44.686959 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29bc260-23d6-4abb-aa9f-27e1c4a9cd1d None None] REQUEST_METHOD: `POST` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:47:44.687251 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29bc260-23d6-4abb-aa9f-27e1c4a9cd1d None None] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:47:44.687526 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29bc260-23d6-4abb-aa9f-27e1c4a9cd1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:47:44.693228 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ffa4f71-c607-4e69-bfc1-581508d936e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:47:44.694028 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ffa4f71-c607-4e69-bfc1-581508d936e6 admin admin] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:47:44.694324 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ffa4f71-c607-4e69-bfc1-581508d936e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:47:44.694592 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ffa4f71-c607-4e69-bfc1-581508d936e6 admin admin] PATH_INFO: `/v3/domains` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:47:44.695315 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ffa4f71-c607-4e69-bfc1-581508d936e6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88113) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:47:44.695807 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ffa4f71-c607-4e69-bfc1-581508d936e6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:47:44.697453 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ffa4f71-c607-4e69-bfc1-581508d936e6 admin admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:47:44.707432 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.auth.core [None req-c29bc260-23d6-4abb-aa9f-27e1c4a9cd1d None None] MFA Rules not processed for user `fd836cf590a145df9acf19cf3ae2ac56`. Rule list: `[]` (Enabled: `True`). {{(pid=88114) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:47:44.708642 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.fernet_utils [None req-c29bc260-23d6-4abb-aa9f-27e1c4a9cd1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88114) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:47:44.709219 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 784/1566] 10.222.0.13 () {58 vars in 1153 bytes} [Wed Aug 31 09:47:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:47:44.710233 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ffa4f71-c607-4e69-bfc1-581508d936e6 admin admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:47:44.717166 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 783/1567] 10.222.0.13 () {58 vars in 958 bytes} [Wed Aug 31 09:47:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:47:44.718432 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff07da9e-c622-42cf-9416-b9b8543db8e4 None None] REQUEST_METHOD: `POST` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:47:44.718594 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff07da9e-c622-42cf-9416-b9b8543db8e4 None None] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:47:44.718866 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff07da9e-c622-42cf-9416-b9b8543db8e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:47:44.720988 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ffa4f71-c607-4e69-bfc1-581508d936e6 admin admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:47:44.726458 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad75f562-1c3b-4d88-9647-a7eec6f8495b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:47:44.727193 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad75f562-1c3b-4d88-9647-a7eec6f8495b admin admin] REQUEST_METHOD: `POST` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:47:44.727474 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad75f562-1c3b-4d88-9647-a7eec6f8495b admin admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:47:44.727741 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad75f562-1c3b-4d88-9647-a7eec6f8495b admin admin] PATH_INFO: `/v3/projects` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:47:44.728369 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad75f562-1c3b-4d88-9647-a7eec6f8495b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:47:44.737730 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad75f562-1c3b-4d88-9647-a7eec6f8495b admin admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:47:44.751164 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.auth.core [None req-ff07da9e-c622-42cf-9416-b9b8543db8e4 None None] MFA Rules not processed for user `bf41ca204722491abce3c27f2ff87126`. Rule list: `[]` (Enabled: `True`). {{(pid=88114) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:47:44.752609 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.fernet_utils [None req-ff07da9e-c622-42cf-9416-b9b8543db8e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88114) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:47:44.768818 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 785/1568] 10.222.0.13 () {60 vars in 1153 bytes} [Wed Aug 31 09:47:44 2022] POST /identity/v3/projects => generated 377 bytes in 59 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 09:47:44.770103 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e9f254-360a-4c7f-b5f0-332a281b8df5 None None] REQUEST_METHOD: `POST` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:47:44.770390 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e9f254-360a-4c7f-b5f0-332a281b8df5 None None] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:47:44.770655 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e9f254-360a-4c7f-b5f0-332a281b8df5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:47:44.791469 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.auth.core [None req-02e9f254-360a-4c7f-b5f0-332a281b8df5 None None] MFA Rules not processed for user `fd836cf590a145df9acf19cf3ae2ac56`. Rule list: `[]` (Enabled: `True`). {{(pid=88113) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:47:44.793011 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.fernet_utils [None req-02e9f254-360a-4c7f-b5f0-332a281b8df5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:47:44.822130 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 784/1569] 10.222.0.13 () {58 vars in 958 bytes} [Wed Aug 31 09:47:44 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:47:44.822951 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff07da9e-c622-42cf-9416-b9b8543db8e4 None None] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:47:44.834572 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff07da9e-c622-42cf-9416-b9b8543db8e4 None None] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:47:44.859437 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:47:44.873616 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 786/1570] 10.222.0.13 () {58 vars in 958 bytes} [Wed Aug 31 09:47:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:47:44.878315 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e9f254-360a-4c7f-b5f0-332a281b8df5 None None] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:47:44.887156 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:47:44.887889 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e9f254-360a-4c7f-b5f0-332a281b8df5 None None] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:47:44.893043 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed3f8216-80ea-41f2-8b78-d5fe7d618052 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:47:44.893567 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3f8216-80ea-41f2-8b78-d5fe7d618052 admin admin] REQUEST_METHOD: `POST` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:47:44.893855 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3f8216-80ea-41f2-8b78-d5fe7d618052 admin admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:47:44.894125 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3f8216-80ea-41f2-8b78-d5fe7d618052 admin admin] PATH_INFO: `/v3/users` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:47:44.894769 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed3f8216-80ea-41f2-8b78-d5fe7d618052 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:47:44.897280 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed3f8216-80ea-41f2-8b78-d5fe7d618052 admin admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:47:44.904886 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6108a0b-2bce-47d6-acf4-4284ed8888ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:47:44.905541 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6108a0b-2bce-47d6-acf4-4284ed8888ad admin admin] REQUEST_METHOD: `POST` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:47:44.905820 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6108a0b-2bce-47d6-acf4-4284ed8888ad admin admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:47:44.906093 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6108a0b-2bce-47d6-acf4-4284ed8888ad admin admin] PATH_INFO: `/v3/users` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:47:44.906728 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6108a0b-2bce-47d6-acf4-4284ed8888ad admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:47:44.907738 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: WARNING py.warnings [None req-c6108a0b-2bce-47d6-acf4-4284ed8888ad admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 31 09:47:44.907738 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: warnings.warn(msg) Aug 31 09:47:44.907738 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: Aug 31 09:47:44.909211 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6108a0b-2bce-47d6-acf4-4284ed8888ad admin admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:47:44.919769 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 787/1571] 10.222.0.13 () {60 vars in 1144 bytes} [Wed Aug 31 09:47:44 2022] POST /identity/v3/users => generated 296 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 09:47:44.920823 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed3f8216-80ea-41f2-8b78-d5fe7d618052 admin admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:47:44.934653 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed3f8216-80ea-41f2-8b78-d5fe7d618052 admin admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:47:44.937262 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 785/1572] 10.222.0.13 () {60 vars in 1144 bytes} [Wed Aug 31 09:47:44 2022] POST /identity/v3/users => generated 286 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 09:47:44.939523 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5299a5c4-362a-4d41-894f-4b212b5a9054 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:47:44.940158 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5299a5c4-362a-4d41-894f-4b212b5a9054 admin admin] REQUEST_METHOD: `POST` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:47:44.940533 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5299a5c4-362a-4d41-894f-4b212b5a9054 admin admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:47:44.940732 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5299a5c4-362a-4d41-894f-4b212b5a9054 admin admin] PATH_INFO: `/v3/projects` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:47:44.941091 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6108a0b-2bce-47d6-acf4-4284ed8888ad admin admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:47:44.941360 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5299a5c4-362a-4d41-894f-4b212b5a9054 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:47:44.943377 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5299a5c4-362a-4d41-894f-4b212b5a9054 admin admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:47:44.951852 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6108a0b-2bce-47d6-acf4-4284ed8888ad admin admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:47:44.957802 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02cb709c-b30f-414c-a540-c6e062bbf8d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:47:44.958415 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02cb709c-b30f-414c-a540-c6e062bbf8d2 admin admin] REQUEST_METHOD: `POST` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:47:44.959265 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02cb709c-b30f-414c-a540-c6e062bbf8d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:47:44.959678 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02cb709c-b30f-414c-a540-c6e062bbf8d2 admin admin] PATH_INFO: `/v3/projects` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:47:44.960372 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02cb709c-b30f-414c-a540-c6e062bbf8d2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:47:44.962505 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02cb709c-b30f-414c-a540-c6e062bbf8d2 admin admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:47:44.972722 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 788/1573] 10.222.0.13 () {60 vars in 1153 bytes} [Wed Aug 31 09:47:44 2022] POST /identity/v3/projects => generated 383 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 09:47:44.973619 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5299a5c4-362a-4d41-894f-4b212b5a9054 admin admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:47:44.999120 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 786/1574] 10.222.0.13 () {60 vars in 1153 bytes} [Wed Aug 31 09:47:44 2022] POST /identity/v3/projects => generated 379 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 09:47:44.999120 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02cb709c-b30f-414c-a540-c6e062bbf8d2 admin admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:47:45.040658 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5299a5c4-362a-4d41-894f-4b212b5a9054 admin admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:47:45.062022 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02cb709c-b30f-414c-a540-c6e062bbf8d2 admin admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:47:45.066778 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26188e29-8b79-4de9-8832-a5f0f77df897 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:47:45.067389 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26188e29-8b79-4de9-8832-a5f0f77df897 admin admin] REQUEST_METHOD: `POST` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:47:45.067664 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26188e29-8b79-4de9-8832-a5f0f77df897 admin admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:47:45.067931 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26188e29-8b79-4de9-8832-a5f0f77df897 admin admin] PATH_INFO: `/v3/users` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:47:45.068601 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26188e29-8b79-4de9-8832-a5f0f77df897 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:47:45.070630 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26188e29-8b79-4de9-8832-a5f0f77df897 admin admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:47:45.093726 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 787/1575] 10.222.0.13 () {60 vars in 1144 bytes} [Wed Aug 31 09:47:44 2022] POST /identity/v3/users => generated 289 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 09:47:45.095033 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68f414e-c3e4-4ff2-a92e-0d97ee528d6c None None] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:47:45.095315 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68f414e-c3e4-4ff2-a92e-0d97ee528d6c None None] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:47:45.095584 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68f414e-c3e4-4ff2-a92e-0d97ee528d6c None None] PATH_INFO: `/` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:47:45.097171 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 788/1576] 10.222.0.13 () {54 vars in 962 bytes} [Wed Aug 31 09:47:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:47:45.097920 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c68f414e-c3e4-4ff2-a92e-0d97ee528d6c None None] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:47:45.103521 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8764769-8de4-4584-a475-e53a8fdb3619 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:47:45.104141 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8764769-8de4-4584-a475-e53a8fdb3619 admin admin] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:47:45.104421 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8764769-8de4-4584-a475-e53a8fdb3619 admin admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:47:45.104690 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8764769-8de4-4584-a475-e53a8fdb3619 admin admin] PATH_INFO: `/v3/domains` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:47:45.105338 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8764769-8de4-4584-a475-e53a8fdb3619 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88113) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:47:45.105679 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8764769-8de4-4584-a475-e53a8fdb3619 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:47:45.107724 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8764769-8de4-4584-a475-e53a8fdb3619 admin admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:47:45.112306 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c68f414e-c3e4-4ff2-a92e-0d97ee528d6c None None] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:47:45.117733 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b66e68c6-a08b-4904-9b76-b37f91e90d8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:47:45.118332 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66e68c6-a08b-4904-9b76-b37f91e90d8d admin admin] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:47:45.118609 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66e68c6-a08b-4904-9b76-b37f91e90d8d admin admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:47:45.118887 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66e68c6-a08b-4904-9b76-b37f91e90d8d admin admin] PATH_INFO: `/v3/domains` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:47:45.119554 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b66e68c6-a08b-4904-9b76-b37f91e90d8d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88114) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:47:45.119851 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b66e68c6-a08b-4904-9b76-b37f91e90d8d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:47:45.120340 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 789/1577] 10.222.0.13 () {58 vars in 1153 bytes} [Wed Aug 31 09:47:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:47:45.121190 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8764769-8de4-4584-a475-e53a8fdb3619 admin admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:47:45.121523 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b66e68c6-a08b-4904-9b76-b37f91e90d8d admin admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:47:45.135292 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 789/1578] 10.222.0.13 () {58 vars in 1153 bytes} [Wed Aug 31 09:47:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:47:45.136295 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b66e68c6-a08b-4904-9b76-b37f91e90d8d admin admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:47:45.138435 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8764769-8de4-4584-a475-e53a8fdb3619 admin admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:47:45.143294 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c347d907-4eea-458c-bd9e-d90755cf73e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:47:45.143895 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c347d907-4eea-458c-bd9e-d90755cf73e8 admin admin] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:47:45.144171 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c347d907-4eea-458c-bd9e-d90755cf73e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:47:45.144436 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c347d907-4eea-458c-bd9e-d90755cf73e8 admin admin] PATH_INFO: `/v3/roles` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:47:45.145072 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c347d907-4eea-458c-bd9e-d90755cf73e8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:47:45.146807 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c347d907-4eea-458c-bd9e-d90755cf73e8 admin admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:47:45.147964 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b66e68c6-a08b-4904-9b76-b37f91e90d8d admin admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:47:45.153238 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e060c40-cd03-4155-8e31-395bc156ed39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:47:45.153862 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e060c40-cd03-4155-8e31-395bc156ed39 admin admin] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:47:45.154140 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e060c40-cd03-4155-8e31-395bc156ed39 admin admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:47:45.154410 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e060c40-cd03-4155-8e31-395bc156ed39 admin admin] PATH_INFO: `/v3/roles` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:47:45.155048 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e060c40-cd03-4155-8e31-395bc156ed39 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:47:45.156637 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e060c40-cd03-4155-8e31-395bc156ed39 admin admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:47:45.157808 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 790/1579] 10.222.0.13 () {58 vars in 1122 bytes} [Wed Aug 31 09:47:45 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 09:47:45.158591 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c347d907-4eea-458c-bd9e-d90755cf73e8 admin admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:47:45.168954 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c347d907-4eea-458c-bd9e-d90755cf73e8 admin admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:47:45.173850 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 790/1580] 10.222.0.13 () {58 vars in 1122 bytes} [Wed Aug 31 09:47:45 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 09:47:45.174010 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e060c40-cd03-4155-8e31-395bc156ed39 admin admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:47:45.178679 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d064fb6-bbae-4363-9801-8cfbee805cb8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:47:45.179573 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d064fb6-bbae-4363-9801-8cfbee805cb8 admin admin] REQUEST_METHOD: `POST` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:47:45.180099 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d064fb6-bbae-4363-9801-8cfbee805cb8 admin admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:47:45.180576 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d064fb6-bbae-4363-9801-8cfbee805cb8 admin admin] PATH_INFO: `/v3/users` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:47:45.181530 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d064fb6-bbae-4363-9801-8cfbee805cb8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:47:45.183550 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d064fb6-bbae-4363-9801-8cfbee805cb8 admin admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:47:45.185975 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e060c40-cd03-4155-8e31-395bc156ed39 admin admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:47:45.190865 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3745afed-bd19-4bac-bbcf-7f85581686a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:47:45.191466 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3745afed-bd19-4bac-bbcf-7f85581686a0 admin admin] REQUEST_METHOD: `POST` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:47:45.191742 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3745afed-bd19-4bac-bbcf-7f85581686a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:47:45.192010 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3745afed-bd19-4bac-bbcf-7f85581686a0 admin admin] PATH_INFO: `/v3/users` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:47:45.192633 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3745afed-bd19-4bac-bbcf-7f85581686a0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:47:45.195232 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3745afed-bd19-4bac-bbcf-7f85581686a0 admin admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:47:45.210142 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 791/1581] 10.222.0.13 () {60 vars in 1144 bytes} [Wed Aug 31 09:47:45 2022] POST /identity/v3/users => generated 292 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 09:47:45.211075 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d064fb6-bbae-4363-9801-8cfbee805cb8 admin admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:47:45.221788 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d064fb6-bbae-4363-9801-8cfbee805cb8 admin admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:47:45.226155 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 791/1582] 10.222.0.13 () {60 vars in 1144 bytes} [Wed Aug 31 09:47:45 2022] POST /identity/v3/users => generated 290 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 09:47:45.226155 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a264518d-0bdc-4a2f-8bf1-0809f39b93ad None None] REQUEST_METHOD: `POST` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:47:45.226155 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a264518d-0bdc-4a2f-8bf1-0809f39b93ad None None] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:47:45.226155 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a264518d-0bdc-4a2f-8bf1-0809f39b93ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:47:45.229684 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17841894-1f80-47c9-ab59-3048f5bdb8af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:47:45.230355 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17841894-1f80-47c9-ab59-3048f5bdb8af admin admin] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:47:45.230637 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17841894-1f80-47c9-ab59-3048f5bdb8af admin admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:47:45.230908 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17841894-1f80-47c9-ab59-3048f5bdb8af admin admin] PATH_INFO: `/v3/roles` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:47:45.231578 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17841894-1f80-47c9-ab59-3048f5bdb8af admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:47:45.233199 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17841894-1f80-47c9-ab59-3048f5bdb8af admin admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:47:45.245159 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 792/1583] 10.222.0.13 () {58 vars in 1122 bytes} [Wed Aug 31 09:47:45 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 09:47:45.246101 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17841894-1f80-47c9-ab59-3048f5bdb8af admin admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:47:45.256188 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17841894-1f80-47c9-ab59-3048f5bdb8af admin admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:47:45.261127 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df77e422-de4e-4d20-8f09-c7452aaf4da0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:47:45.261742 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df77e422-de4e-4d20-8f09-c7452aaf4da0 admin admin] REQUEST_METHOD: `POST` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:47:45.262068 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df77e422-de4e-4d20-8f09-c7452aaf4da0 admin admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:47:45.262340 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df77e422-de4e-4d20-8f09-c7452aaf4da0 admin admin] PATH_INFO: `/v3/projects` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:47:45.262976 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df77e422-de4e-4d20-8f09-c7452aaf4da0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:47:45.264952 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df77e422-de4e-4d20-8f09-c7452aaf4da0 admin admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:47:45.267417 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.auth.core [None req-a264518d-0bdc-4a2f-8bf1-0809f39b93ad None None] MFA Rules not processed for user `dc076493239d4ccdbdc37c0369c49529`. Rule list: `[]` (Enabled: `True`). {{(pid=88114) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:47:45.268613 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.fernet_utils [None req-a264518d-0bdc-4a2f-8bf1-0809f39b93ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88114) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:47:45.286669 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:47:45.296256 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 793/1584] 10.222.0.13 () {60 vars in 1153 bytes} [Wed Aug 31 09:47:45 2022] POST /identity/v3/projects => generated 363 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 09:47:45.297224 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df77e422-de4e-4d20-8f09-c7452aaf4da0 admin admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:47:45.317529 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=90,keystone:INSERT=10 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:47:45.348268 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 792/1585] 10.222.0.13 () {58 vars in 1059 bytes} [Wed Aug 31 09:47:45 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:47:45.349111 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a264518d-0bdc-4a2f-8bf1-0809f39b93ad None None] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:47:45.359791 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a264518d-0bdc-4a2f-8bf1-0809f39b93ad None None] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:47:45.362438 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df77e422-de4e-4d20-8f09-c7452aaf4da0 admin admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:47:45.364316 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e14db8e8-7967-4ca1-a990-4fb4005fb71a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:47:45.365613 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14db8e8-7967-4ca1-a990-4fb4005fb71a admin admin] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:47:45.365965 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14db8e8-7967-4ca1-a990-4fb4005fb71a admin admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:47:45.366233 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14db8e8-7967-4ca1-a990-4fb4005fb71a admin admin] PATH_INFO: `/v3/roles` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:47:45.366878 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e14db8e8-7967-4ca1-a990-4fb4005fb71a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:47:45.367936 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48879fa1-b6d1-4290-97e2-52dc7b0d5815 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:47:45.368455 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e14db8e8-7967-4ca1-a990-4fb4005fb71a admin admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:47:45.368565 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48879fa1-b6d1-4290-97e2-52dc7b0d5815 admin admin] REQUEST_METHOD: `POST` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:47:45.368809 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48879fa1-b6d1-4290-97e2-52dc7b0d5815 admin admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:47:45.369083 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48879fa1-b6d1-4290-97e2-52dc7b0d5815 admin admin] PATH_INFO: `/v3/projects` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:47:45.369756 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48879fa1-b6d1-4290-97e2-52dc7b0d5815 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:47:45.371677 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48879fa1-b6d1-4290-97e2-52dc7b0d5815 admin admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:47:45.379071 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 793/1586] 10.222.0.13 () {58 vars in 1122 bytes} [Wed Aug 31 09:47:45 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 09:47:45.379959 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e14db8e8-7967-4ca1-a990-4fb4005fb71a admin admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:47:45.392876 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e14db8e8-7967-4ca1-a990-4fb4005fb71a admin admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:47:45.403684 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d2c731-c2c1-4a7c-97c5-f685db13ae67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:47:45.404377 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d2c731-c2c1-4a7c-97c5-f685db13ae67 admin admin] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:47:45.404671 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d2c731-c2c1-4a7c-97c5-f685db13ae67 admin admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:47:45.404967 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d2c731-c2c1-4a7c-97c5-f685db13ae67 admin admin] PATH_INFO: `/v3/roles` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:47:45.405599 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9d2c731-c2c1-4a7c-97c5-f685db13ae67 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:47:45.406604 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 794/1587] 10.222.0.13 () {60 vars in 1153 bytes} [Wed Aug 31 09:47:45 2022] POST /identity/v3/projects => generated 385 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 09:47:45.407447 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48879fa1-b6d1-4290-97e2-52dc7b0d5815 admin admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:47:45.407559 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9d2c731-c2c1-4a7c-97c5-f685db13ae67 admin admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:47:45.419232 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 794/1588] 10.222.0.13 () {58 vars in 1122 bytes} [Wed Aug 31 09:47:45 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 09:47:45.423235 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d2c731-c2c1-4a7c-97c5-f685db13ae67 admin admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:47:45.433885 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d2c731-c2c1-4a7c-97c5-f685db13ae67 admin admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:47:45.479698 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48879fa1-b6d1-4290-97e2-52dc7b0d5815 admin admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:47:45.484733 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf6b71d0-199b-4b01-a872-38cb79b402c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:47:45.485432 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6b71d0-199b-4b01-a872-38cb79b402c3 admin admin] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:47:45.485742 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6b71d0-199b-4b01-a872-38cb79b402c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:47:45.486013 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6b71d0-199b-4b01-a872-38cb79b402c3 admin admin] PATH_INFO: `/v3/roles` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:47:45.486681 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf6b71d0-199b-4b01-a872-38cb79b402c3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:47:45.488311 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf6b71d0-199b-4b01-a872-38cb79b402c3 admin admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:47:45.492576 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4941ec1c-5b01-4f1c-abb4-77d69fbf14fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:47:45.493209 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4941ec1c-5b01-4f1c-abb4-77d69fbf14fa admin admin] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:47:45.493661 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4941ec1c-5b01-4f1c-abb4-77d69fbf14fa admin admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:47:45.493932 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4941ec1c-5b01-4f1c-abb4-77d69fbf14fa admin admin] PATH_INFO: `/v3/roles` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:47:45.494578 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4941ec1c-5b01-4f1c-abb4-77d69fbf14fa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:47:45.496166 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4941ec1c-5b01-4f1c-abb4-77d69fbf14fa admin admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:47:45.499330 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 795/1589] 10.222.0.13 () {58 vars in 1122 bytes} [Wed Aug 31 09:47:45 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 09:47:45.500164 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf6b71d0-199b-4b01-a872-38cb79b402c3 admin admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:47:45.507734 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 795/1590] 10.222.0.13 () {58 vars in 1122 bytes} [Wed Aug 31 09:47:45 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 09:47:45.508551 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4941ec1c-5b01-4f1c-abb4-77d69fbf14fa admin admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:47:45.512941 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf6b71d0-199b-4b01-a872-38cb79b402c3 admin admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:47:45.517665 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a278c2-aecc-48ab-a381-f9d3e1478623 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:47:45.518268 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a278c2-aecc-48ab-a381-f9d3e1478623 admin admin] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:47:45.518379 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4941ec1c-5b01-4f1c-abb4-77d69fbf14fa admin admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:47:45.518542 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a278c2-aecc-48ab-a381-f9d3e1478623 admin admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:47:45.518811 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a278c2-aecc-48ab-a381-f9d3e1478623 admin admin] PATH_INFO: `/v3/roles` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:47:45.519440 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88a278c2-aecc-48ab-a381-f9d3e1478623 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:47:45.521023 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88a278c2-aecc-48ab-a381-f9d3e1478623 admin admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:47:45.524108 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11dc7a9b-6e1c-4af3-89e3-fb392875abb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:47:45.524711 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11dc7a9b-6e1c-4af3-89e3-fb392875abb7 admin admin] REQUEST_METHOD: `POST` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:47:45.524990 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11dc7a9b-6e1c-4af3-89e3-fb392875abb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:47:45.525261 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11dc7a9b-6e1c-4af3-89e3-fb392875abb7 admin admin] PATH_INFO: `/v3/users` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:47:45.525926 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11dc7a9b-6e1c-4af3-89e3-fb392875abb7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:47:45.527844 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11dc7a9b-6e1c-4af3-89e3-fb392875abb7 admin admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:47:45.534043 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 796/1591] 10.222.0.13 () {58 vars in 1122 bytes} [Wed Aug 31 09:47:45 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 09:47:45.534888 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a278c2-aecc-48ab-a381-f9d3e1478623 admin admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:47:45.555909 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 796/1592] 10.222.0.13 () {60 vars in 1144 bytes} [Wed Aug 31 09:47:45 2022] POST /identity/v3/users => generated 282 bytes in 42 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 09:47:45.555909 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11dc7a9b-6e1c-4af3-89e3-fb392875abb7 admin admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:47:45.562603 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11dc7a9b-6e1c-4af3-89e3-fb392875abb7 admin admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:47:45.567951 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7713749-27e8-49b2-9782-32dc7e61787e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:47:45.568607 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7713749-27e8-49b2-9782-32dc7e61787e admin admin] REQUEST_METHOD: `PUT` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:47:45.568946 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7713749-27e8-49b2-9782-32dc7e61787e admin admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:47:45.569260 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7713749-27e8-49b2-9782-32dc7e61787e admin admin] PATH_INFO: `/v3/projects/c0fc6660b5494415915cdf2b325e5bd6/users/886e40cf192d46c2b2e7eb2a95ec62e1/roles/6c99f5d9f1f243eaa134d8540f21095d` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:47:45.584651 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7713749-27e8-49b2-9782-32dc7e61787e admin admin] RBAC: Authorizing `identity:create_grant(project_id=c0fc6660b5494415915cdf2b325e5bd6, user_id=886e40cf192d46c2b2e7eb2a95ec62e1, role_id=6c99f5d9f1f243eaa134d8540f21095d)` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:47:45.587435 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7713749-27e8-49b2-9782-32dc7e61787e admin admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:47:45.601574 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 797/1593] 10.222.0.13 () {60 vars in 1483 bytes} [Wed Aug 31 09:47:45 2022] PUT /identity/v3/projects/c0fc6660b5494415915cdf2b325e5bd6/users/886e40cf192d46c2b2e7eb2a95ec62e1/roles/6c99f5d9f1f243eaa134d8540f21095d => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:47:45.605593 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7713749-27e8-49b2-9782-32dc7e61787e admin admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:47:45.617410 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a278c2-aecc-48ab-a381-f9d3e1478623 admin admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:47:45.621295 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:47:45.668442 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7713749-27e8-49b2-9782-32dc7e61787e admin admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:47:45.673708 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-306437e2-98de-4f34-8e16-377ce995ab3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:47:45.674328 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306437e2-98de-4f34-8e16-377ce995ab3d admin admin] REQUEST_METHOD: `POST` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:47:45.674606 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306437e2-98de-4f34-8e16-377ce995ab3d admin admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:47:45.674879 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306437e2-98de-4f34-8e16-377ce995ab3d admin admin] PATH_INFO: `/v3/users` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:47:45.675523 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-306437e2-98de-4f34-8e16-377ce995ab3d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:47:45.677628 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-306437e2-98de-4f34-8e16-377ce995ab3d admin admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:47:45.689208 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee5d111-db8e-4b4e-9521-4b63a9721973 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc076493239d4ccdbdc37c0369c49529', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '49548eb44d62415582af820ab66decee', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:47:45.689923 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee5d111-db8e-4b4e-9521-4b63a9721973 service nova] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:47:45.690204 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee5d111-db8e-4b4e-9521-4b63a9721973 service nova] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:47:45.690557 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee5d111-db8e-4b4e-9521-4b63a9721973 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:47:45.695344 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:47:45.706540 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 798/1594] 10.222.0.13 () {60 vars in 1144 bytes} [Wed Aug 31 09:47:45 2022] POST /identity/v3/users => generated 293 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 09:47:45.706540 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-306437e2-98de-4f34-8e16-377ce995ab3d admin admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:47:45.715100 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-306437e2-98de-4f34-8e16-377ce995ab3d admin admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:47:45.719944 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad86ca1f-6c8f-4b14-b3e0-729e5ff87bda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:47:45.720626 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad86ca1f-6c8f-4b14-b3e0-729e5ff87bda admin admin] REQUEST_METHOD: `PUT` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:47:45.720935 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad86ca1f-6c8f-4b14-b3e0-729e5ff87bda admin admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:47:45.721201 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad86ca1f-6c8f-4b14-b3e0-729e5ff87bda admin admin] PATH_INFO: `/v3/projects/05357b8aa9c6411e849aab8e271b1ca1/users/9bfb4fba581c45bbb1101be63a110025/roles/6c99f5d9f1f243eaa134d8540f21095d` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:47:45.737327 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad86ca1f-6c8f-4b14-b3e0-729e5ff87bda admin admin] RBAC: Authorizing `identity:create_grant(project_id=05357b8aa9c6411e849aab8e271b1ca1, user_id=9bfb4fba581c45bbb1101be63a110025, role_id=6c99f5d9f1f243eaa134d8540f21095d)` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:47:45.740199 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad86ca1f-6c8f-4b14-b3e0-729e5ff87bda admin admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:47:45.750903 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fee5d111-db8e-4b4e-9521-4b63a9721973 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:47:45.752531 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fee5d111-db8e-4b4e-9521-4b63a9721973 service nova] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:47:45.753581 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 799/1595] 10.222.0.13 () {60 vars in 1483 bytes} [Wed Aug 31 09:47:45 2022] PUT /identity/v3/projects/05357b8aa9c6411e849aab8e271b1ca1/users/9bfb4fba581c45bbb1101be63a110025/roles/6c99f5d9f1f243eaa134d8540f21095d => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:47:45.754475 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad86ca1f-6c8f-4b14-b3e0-729e5ff87bda admin admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:47:45.762918 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 797/1596] 10.222.0.13 () {60 vars in 1365 bytes} [Wed Aug 31 09:47:45 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 228 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:47:45.763792 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee5d111-db8e-4b4e-9521-4b63a9721973 service nova] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:47:45.775736 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee5d111-db8e-4b4e-9521-4b63a9721973 service nova] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:47:45.827500 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad86ca1f-6c8f-4b14-b3e0-729e5ff87bda admin admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:47:45.832620 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba5a57c9-86d4-4018-a768-ae48c460d4ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:47:45.833558 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5a57c9-86d4-4018-a768-ae48c460d4ad admin admin] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:47:45.834536 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5a57c9-86d4-4018-a768-ae48c460d4ad admin admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:47:45.834812 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5a57c9-86d4-4018-a768-ae48c460d4ad admin admin] PATH_INFO: `/v3/roles` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:47:45.835582 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba5a57c9-86d4-4018-a768-ae48c460d4ad admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:47:45.837248 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba5a57c9-86d4-4018-a768-ae48c460d4ad admin admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:47:45.841333 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b97257-88c6-40a4-9a91-288b814e97a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:47:45.842016 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b97257-88c6-40a4-9a91-288b814e97a3 admin admin] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:47:45.842491 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b97257-88c6-40a4-9a91-288b814e97a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:47:45.842603 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b97257-88c6-40a4-9a91-288b814e97a3 admin admin] PATH_INFO: `/v3/roles` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:47:45.843383 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96b97257-88c6-40a4-9a91-288b814e97a3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:47:45.845094 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96b97257-88c6-40a4-9a91-288b814e97a3 admin admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:47:45.856802 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 800/1597] 10.222.0.13 () {58 vars in 1122 bytes} [Wed Aug 31 09:47:45 2022] GET /identity/v3/roles => generated 1420 bytes in 103 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 09:47:45.860548 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 798/1598] 10.222.0.13 () {58 vars in 1122 bytes} [Wed Aug 31 09:47:45 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 09:47:45.860701 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba5a57c9-86d4-4018-a768-ae48c460d4ad admin admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:47:45.860786 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b97257-88c6-40a4-9a91-288b814e97a3 admin admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:47:45.869922 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b97257-88c6-40a4-9a91-288b814e97a3 admin admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:47:45.871340 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba5a57c9-86d4-4018-a768-ae48c460d4ad admin admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:47:45.874848 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e01bfd-b91f-48a5-9eac-fa40a428e9a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:47:45.875464 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e01bfd-b91f-48a5-9eac-fa40a428e9a8 admin admin] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:47:45.875747 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e01bfd-b91f-48a5-9eac-fa40a428e9a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:47:45.875960 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c3be5d-b55e-4995-bedf-fd6096692f08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:47:45.876074 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e01bfd-b91f-48a5-9eac-fa40a428e9a8 admin admin] PATH_INFO: `/v3/roles` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:47:45.876558 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c3be5d-b55e-4995-bedf-fd6096692f08 admin admin] REQUEST_METHOD: `PUT` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:47:45.876692 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03e01bfd-b91f-48a5-9eac-fa40a428e9a8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:47:45.876889 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c3be5d-b55e-4995-bedf-fd6096692f08 admin admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:47:45.877101 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c3be5d-b55e-4995-bedf-fd6096692f08 admin admin] PATH_INFO: `/v3/projects/05be916a558445ee8550559d491efbd0/users/4961545b43f74fdb87d01b78600d54ed/roles/6c99f5d9f1f243eaa134d8540f21095d` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:47:45.878549 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03e01bfd-b91f-48a5-9eac-fa40a428e9a8 admin admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:47:45.893275 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 799/1599] 10.222.0.13 () {58 vars in 1122 bytes} [Wed Aug 31 09:47:45 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 09:47:45.893275 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1f520d-1102-428e-8b23-3c9a3552a803 None None] REQUEST_METHOD: `POST` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:47:45.893275 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1f520d-1102-428e-8b23-3c9a3552a803 None None] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:47:45.893275 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1f520d-1102-428e-8b23-3c9a3552a803 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:47:45.898563 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89c3be5d-b55e-4995-bedf-fd6096692f08 admin admin] RBAC: Authorizing `identity:create_grant(project_id=05be916a558445ee8550559d491efbd0, user_id=4961545b43f74fdb87d01b78600d54ed, role_id=6c99f5d9f1f243eaa134d8540f21095d)` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:47:45.902422 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89c3be5d-b55e-4995-bedf-fd6096692f08 admin admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:47:45.912910 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.auth.core [None req-ba1f520d-1102-428e-8b23-3c9a3552a803 None None] MFA Rules not processed for user `fd836cf590a145df9acf19cf3ae2ac56`. Rule list: `[]` (Enabled: `True`). {{(pid=88113) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:47:45.914103 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.fernet_utils [None req-ba1f520d-1102-428e-8b23-3c9a3552a803 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:47:45.918817 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 801/1600] 10.222.0.13 () {60 vars in 1483 bytes} [Wed Aug 31 09:47:45 2022] PUT /identity/v3/projects/05be916a558445ee8550559d491efbd0/users/4961545b43f74fdb87d01b78600d54ed/roles/6c99f5d9f1f243eaa134d8540f21095d => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:47:45.919897 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c3be5d-b55e-4995-bedf-fd6096692f08 admin admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:47:45.945739 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=100 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:47:45.987561 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 800/1601] 10.222.0.13 () {58 vars in 958 bytes} [Wed Aug 31 09:47:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:47:45.988922 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86378326-3d54-4a97-adbc-eb75ad3195db None None] REQUEST_METHOD: `POST` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:47:45.989285 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86378326-3d54-4a97-adbc-eb75ad3195db None None] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:47:45.989677 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86378326-3d54-4a97-adbc-eb75ad3195db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:47:45.994060 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c3be5d-b55e-4995-bedf-fd6096692f08 admin admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:47:46.000432 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce3edc33-8aa2-4356-a2df-294329d7a7cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:47:46.000432 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3edc33-8aa2-4356-a2df-294329d7a7cb admin admin] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:47:46.000758 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3edc33-8aa2-4356-a2df-294329d7a7cb admin admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:47:46.000758 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3edc33-8aa2-4356-a2df-294329d7a7cb admin admin] PATH_INFO: `/v3/roles` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:47:46.001769 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce3edc33-8aa2-4356-a2df-294329d7a7cb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:47:46.003322 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce3edc33-8aa2-4356-a2df-294329d7a7cb admin admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:47:46.016253 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 802/1602] 10.222.0.13 () {58 vars in 1122 bytes} [Wed Aug 31 09:47:45 2022] GET /identity/v3/roles => generated 1420 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 09:47:46.017334 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce3edc33-8aa2-4356-a2df-294329d7a7cb admin admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:47:46.028296 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce3edc33-8aa2-4356-a2df-294329d7a7cb admin admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:47:46.029703 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.auth.core [None req-86378326-3d54-4a97-adbc-eb75ad3195db None None] MFA Rules not processed for user `9bfb4fba581c45bbb1101be63a110025`. Rule list: `[]` (Enabled: `True`). {{(pid=88113) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:47:46.030963 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.fernet_utils [None req-86378326-3d54-4a97-adbc-eb75ad3195db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:47:46.033106 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d3637c-d9f4-4d8d-a7ef-71162468abb3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:47:46.033788 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d3637c-d9f4-4d8d-a7ef-71162468abb3 admin admin] REQUEST_METHOD: `PUT` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:47:46.034063 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d3637c-d9f4-4d8d-a7ef-71162468abb3 admin admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:47:46.034348 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d3637c-d9f4-4d8d-a7ef-71162468abb3 admin admin] PATH_INFO: `/v3/projects/c8c7821da6b84645985cbe9d3f3e0f04/users/f8bffa6c63644be1b1087916e000999a/roles/6c99f5d9f1f243eaa134d8540f21095d` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:47:46.050180 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11d3637c-d9f4-4d8d-a7ef-71162468abb3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c8c7821da6b84645985cbe9d3f3e0f04, user_id=f8bffa6c63644be1b1087916e000999a, role_id=6c99f5d9f1f243eaa134d8540f21095d)` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:47:46.053095 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11d3637c-d9f4-4d8d-a7ef-71162468abb3 admin admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:47:46.072879 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 803/1603] 10.222.0.13 () {60 vars in 1483 bytes} [Wed Aug 31 09:47:46 2022] PUT /identity/v3/projects/c8c7821da6b84645985cbe9d3f3e0f04/users/f8bffa6c63644be1b1087916e000999a/roles/6c99f5d9f1f243eaa134d8540f21095d => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:47:46.073620 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d3637c-d9f4-4d8d-a7ef-71162468abb3 admin admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:47:46.097660 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:47:46.112628 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 801/1604] 10.222.0.13 () {58 vars in 958 bytes} [Wed Aug 31 09:47:45 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:47:46.113669 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86378326-3d54-4a97-adbc-eb75ad3195db None None] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:47:46.135520 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86378326-3d54-4a97-adbc-eb75ad3195db None None] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:47:46.152004 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d3637c-d9f4-4d8d-a7ef-71162468abb3 admin admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:47:46.157461 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f73870f1-199a-4532-b9ea-e4802f3da76d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:47:46.158125 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73870f1-199a-4532-b9ea-e4802f3da76d admin admin] REQUEST_METHOD: `PUT` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:47:46.158541 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73870f1-199a-4532-b9ea-e4802f3da76d admin admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:47:46.158685 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73870f1-199a-4532-b9ea-e4802f3da76d admin admin] PATH_INFO: `/v3/projects/eb404a7dd529496888531e7ef00fd352/users/ae46122578a143a78f07f7f3a80ae079/roles/6c99f5d9f1f243eaa134d8540f21095d` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:47:46.176612 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f73870f1-199a-4532-b9ea-e4802f3da76d admin admin] RBAC: Authorizing `identity:create_grant(project_id=eb404a7dd529496888531e7ef00fd352, user_id=ae46122578a143a78f07f7f3a80ae079, role_id=6c99f5d9f1f243eaa134d8540f21095d)` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:47:46.179314 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f73870f1-199a-4532-b9ea-e4802f3da76d admin admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:47:46.194949 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 804/1605] 10.222.0.13 () {60 vars in 1483 bytes} [Wed Aug 31 09:47:46 2022] PUT /identity/v3/projects/eb404a7dd529496888531e7ef00fd352/users/ae46122578a143a78f07f7f3a80ae079/roles/6c99f5d9f1f243eaa134d8540f21095d => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:47:46.196428 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9a0014-aa75-48d1-90f2-3c1d320948cc None None] REQUEST_METHOD: `POST` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:47:46.196704 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9a0014-aa75-48d1-90f2-3c1d320948cc None None] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:47:46.197142 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9a0014-aa75-48d1-90f2-3c1d320948cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:47:46.205499 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c887844d-90d1-42cf-aad5-d45f28a5c609 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:47:46.206634 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c887844d-90d1-42cf-aad5-d45f28a5c609 admin admin] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:47:46.206918 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c887844d-90d1-42cf-aad5-d45f28a5c609 admin admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:47:46.207192 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c887844d-90d1-42cf-aad5-d45f28a5c609 admin admin] PATH_INFO: `/v3/roles` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:47:46.207921 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c887844d-90d1-42cf-aad5-d45f28a5c609 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:47:46.211236 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c887844d-90d1-42cf-aad5-d45f28a5c609 admin admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:47:46.217854 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.auth.core [None req-4c9a0014-aa75-48d1-90f2-3c1d320948cc None None] MFA Rules not processed for user `fd836cf590a145df9acf19cf3ae2ac56`. Rule list: `[]` (Enabled: `True`). {{(pid=88114) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:47:46.219081 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.fernet_utils [None req-4c9a0014-aa75-48d1-90f2-3c1d320948cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88114) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:47:46.222469 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 802/1606] 10.222.0.13 () {58 vars in 1122 bytes} [Wed Aug 31 09:47:46 2022] GET /identity/v3/roles => generated 1420 bytes in 109 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 09:47:46.226334 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c887844d-90d1-42cf-aad5-d45f28a5c609 admin admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:47:46.298513 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c887844d-90d1-42cf-aad5-d45f28a5c609 admin admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:47:46.305756 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 805/1607] 10.222.0.13 () {58 vars in 958 bytes} [Wed Aug 31 09:47:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:47:46.306715 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c9a0014-aa75-48d1-90f2-3c1d320948cc None None] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:47:46.308032 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=100 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:47:46.324779 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c9a0014-aa75-48d1-90f2-3c1d320948cc None None] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:47:46.329465 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f664ed-2784-4164-9877-de30e3fae18e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:47:46.330029 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f664ed-2784-4164-9877-de30e3fae18e admin admin] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:47:46.330343 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f664ed-2784-4164-9877-de30e3fae18e admin admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:47:46.330598 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f664ed-2784-4164-9877-de30e3fae18e admin admin] PATH_INFO: `/v3/roles` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:47:46.331229 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81f664ed-2784-4164-9877-de30e3fae18e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:47:46.332831 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81f664ed-2784-4164-9877-de30e3fae18e admin admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:47:46.344668 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 806/1608] 10.222.0.13 () {58 vars in 1122 bytes} [Wed Aug 31 09:47:46 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 09:47:46.346413 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705589c4-b75e-4d0b-b6c9-00ad4dffc290 None None] REQUEST_METHOD: `POST` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:47:46.346712 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705589c4-b75e-4d0b-b6c9-00ad4dffc290 None None] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:47:46.346988 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705589c4-b75e-4d0b-b6c9-00ad4dffc290 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:47:46.357181 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe99a535-4237-45d6-a5f8-f03392e5efa4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9bdaacd597442c2bcde2c555f2cc140', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '49548eb44d62415582af820ab66decee', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:47:46.357829 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe99a535-4237-45d6-a5f8-f03392e5efa4 service neutron] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:47:46.358104 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe99a535-4237-45d6-a5f8-f03392e5efa4 service neutron] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:47:46.358368 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe99a535-4237-45d6-a5f8-f03392e5efa4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:47:46.369797 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe99a535-4237-45d6-a5f8-f03392e5efa4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:47:46.371377 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe99a535-4237-45d6-a5f8-f03392e5efa4 service neutron] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:47:46.376729 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.auth.core [None req-705589c4-b75e-4d0b-b6c9-00ad4dffc290 None None] MFA Rules not processed for user `f8bffa6c63644be1b1087916e000999a`. Rule list: `[]` (Enabled: `True`). {{(pid=88114) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:47:46.378239 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.fernet_utils [None req-705589c4-b75e-4d0b-b6c9-00ad4dffc290 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88114) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:47:46.383289 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 803/1609] 10.222.0.13 () {60 vars in 1365 bytes} [Wed Aug 31 09:47:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 157 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:47:46.384288 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe99a535-4237-45d6-a5f8-f03392e5efa4 service neutron] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:47:46.464452 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 807/1610] 10.222.0.13 () {58 vars in 958 bytes} [Wed Aug 31 09:47:46 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:47:46.465829 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebe15a4-066b-4d44-9408-36689e8450ce None None] REQUEST_METHOD: `POST` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:47:46.466115 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebe15a4-066b-4d44-9408-36689e8450ce None None] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:47:46.466390 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebe15a4-066b-4d44-9408-36689e8450ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:47:46.475707 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe99a535-4237-45d6-a5f8-f03392e5efa4 service neutron] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:47:46.482403 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55dc25d6-2438-40cd-95f5-407754b67ef6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc076493239d4ccdbdc37c0369c49529', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '49548eb44d62415582af820ab66decee', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:47:46.483231 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55dc25d6-2438-40cd-95f5-407754b67ef6 service nova] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:47:46.483504 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55dc25d6-2438-40cd-95f5-407754b67ef6 service nova] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:47:46.483852 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55dc25d6-2438-40cd-95f5-407754b67ef6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:47:46.486796 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.auth.core [None req-aebe15a4-066b-4d44-9408-36689e8450ce None None] MFA Rules not processed for user `fd836cf590a145df9acf19cf3ae2ac56`. Rule list: `[]` (Enabled: `True`). {{(pid=88114) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:47:46.488161 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.fernet_utils [None req-aebe15a4-066b-4d44-9408-36689e8450ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88114) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:47:46.500363 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 808/1611] 10.222.0.13 () {58 vars in 958 bytes} [Wed Aug 31 09:47:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:47:46.500363 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aebe15a4-066b-4d44-9408-36689e8450ce None None] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:47:46.510927 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aebe15a4-066b-4d44-9408-36689e8450ce None None] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:47:46.516292 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-935d7544-f60c-4b57-9851-dbc950dd32d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:47:46.517202 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935d7544-f60c-4b57-9851-dbc950dd32d9 admin admin] REQUEST_METHOD: `PUT` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:47:46.517579 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935d7544-f60c-4b57-9851-dbc950dd32d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:47:46.517918 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935d7544-f60c-4b57-9851-dbc950dd32d9 admin admin] PATH_INFO: `/v3/projects/220b78acd5114533b881a6823728317b/users/bf564c77b7d249f9b050b2055b76e362/roles/6c99f5d9f1f243eaa134d8540f21095d` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:47:46.536208 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:47:46.541270 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-935d7544-f60c-4b57-9851-dbc950dd32d9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=220b78acd5114533b881a6823728317b, user_id=bf564c77b7d249f9b050b2055b76e362, role_id=6c99f5d9f1f243eaa134d8540f21095d)` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:47:46.544293 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-935d7544-f60c-4b57-9851-dbc950dd32d9 admin admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:47:46.563601 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 809/1612] 10.222.0.13 () {60 vars in 1483 bytes} [Wed Aug 31 09:47:46 2022] PUT /identity/v3/projects/220b78acd5114533b881a6823728317b/users/bf564c77b7d249f9b050b2055b76e362/roles/6c99f5d9f1f243eaa134d8540f21095d => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:47:46.567591 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32608191-f0e1-46f8-b01d-fbfc2ab7988b None None] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:47:46.568855 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32608191-f0e1-46f8-b01d-fbfc2ab7988b None None] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:47:46.569408 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32608191-f0e1-46f8-b01d-fbfc2ab7988b None None] PATH_INFO: `/` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:47:46.571292 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 810/1613] 10.222.0.13 () {54 vars in 962 bytes} [Wed Aug 31 09:47:46 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:47:46.571921 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32608191-f0e1-46f8-b01d-fbfc2ab7988b None None] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:47:46.578027 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55dc25d6-2438-40cd-95f5-407754b67ef6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:47:46.579742 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55dc25d6-2438-40cd-95f5-407754b67ef6 service nova] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:47:46.591659 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 804/1614] 10.222.0.13 () {60 vars in 1365 bytes} [Wed Aug 31 09:47:46 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 208 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:47:46.592538 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55dc25d6-2438-40cd-95f5-407754b67ef6 service nova] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:47:46.605123 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55dc25d6-2438-40cd-95f5-407754b67ef6 service nova] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:47:46.658010 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32608191-f0e1-46f8-b01d-fbfc2ab7988b None None] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:47:46.664166 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-684455fa-39a6-4d09-9d0b-15b0709073e6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9bdaacd597442c2bcde2c555f2cc140', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '49548eb44d62415582af820ab66decee', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:47:46.664874 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684455fa-39a6-4d09-9d0b-15b0709073e6 service neutron] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:47:46.665157 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684455fa-39a6-4d09-9d0b-15b0709073e6 service neutron] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:47:46.665462 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684455fa-39a6-4d09-9d0b-15b0709073e6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:47:46.669093 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=100 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:47:46.681933 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49775a59-44bd-4ca9-afd3-028803599a3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:47:46.682733 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49775a59-44bd-4ca9-afd3-028803599a3b admin admin] REQUEST_METHOD: `PUT` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:47:46.683130 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49775a59-44bd-4ca9-afd3-028803599a3b admin admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:47:46.683559 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49775a59-44bd-4ca9-afd3-028803599a3b admin admin] PATH_INFO: `/v3/projects/365f6597f247407da4b61f4d50aa1503/users/7ccc7825081b4ec79f4b6d9a4feb9969/roles/6c99f5d9f1f243eaa134d8540f21095d` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:47:46.703325 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49775a59-44bd-4ca9-afd3-028803599a3b admin admin] RBAC: Authorizing `identity:create_grant(project_id=365f6597f247407da4b61f4d50aa1503, user_id=7ccc7825081b4ec79f4b6d9a4feb9969, role_id=6c99f5d9f1f243eaa134d8540f21095d)` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:47:46.706814 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49775a59-44bd-4ca9-afd3-028803599a3b admin admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:47:46.739413 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-684455fa-39a6-4d09-9d0b-15b0709073e6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:47:46.741837 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-684455fa-39a6-4d09-9d0b-15b0709073e6 service neutron] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:47:46.742930 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 805/1615] 10.222.0.13 () {60 vars in 1483 bytes} [Wed Aug 31 09:47:46 2022] PUT /identity/v3/projects/365f6597f247407da4b61f4d50aa1503/users/7ccc7825081b4ec79f4b6d9a4feb9969/roles/6c99f5d9f1f243eaa134d8540f21095d => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:47:46.743857 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49775a59-44bd-4ca9-afd3-028803599a3b admin admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:47:46.758965 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 811/1616] 10.222.0.13 () {60 vars in 1365 bytes} [Wed Aug 31 09:47:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 187 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:47:46.761408 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af95e52-0196-4899-8024-c9b56cc78317 None None] REQUEST_METHOD: `POST` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:47:46.761705 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af95e52-0196-4899-8024-c9b56cc78317 None None] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:47:46.761976 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af95e52-0196-4899-8024-c9b56cc78317 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:47:46.823259 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.auth.core [None req-9af95e52-0196-4899-8024-c9b56cc78317 None None] MFA Rules not processed for user `bf564c77b7d249f9b050b2055b76e362`. Rule list: `[]` (Enabled: `True`). {{(pid=88114) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:47:46.825366 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.fernet_utils [None req-9af95e52-0196-4899-8024-c9b56cc78317 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88114) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:47:46.840508 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49775a59-44bd-4ca9-afd3-028803599a3b admin admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:47:46.849247 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e9187f-9b11-45ec-bd52-5caf56481c78 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9bdaacd597442c2bcde2c555f2cc140', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '49548eb44d62415582af820ab66decee', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:47:46.850302 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e9187f-9b11-45ec-bd52-5caf56481c78 service neutron] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:47:46.850805 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e9187f-9b11-45ec-bd52-5caf56481c78 service neutron] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:47:46.851309 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e9187f-9b11-45ec-bd52-5caf56481c78 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:47:46.971721 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9e9187f-9b11-45ec-bd52-5caf56481c78 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:47:46.979650 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9e9187f-9b11-45ec-bd52-5caf56481c78 service neutron] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:47:46.986490 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 812/1617] 10.222.0.13 () {58 vars in 958 bytes} [Wed Aug 31 09:47:46 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 227 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:47:46.987774 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03cec746-a158-449b-ac60-e3ca26ad5206 None None] REQUEST_METHOD: `POST` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:47:46.988204 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03cec746-a158-449b-ac60-e3ca26ad5206 None None] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:47:46.988577 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03cec746-a158-449b-ac60-e3ca26ad5206 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:47:46.998755 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 806/1618] 10.222.0.13 () {60 vars in 1365 bytes} [Wed Aug 31 09:47:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 255 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:47:47.002177 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa99507d-003a-4b08-ab1c-9386d015c797 None None] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:47:47.003544 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa99507d-003a-4b08-ab1c-9386d015c797 None None] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:47:47.004097 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa99507d-003a-4b08-ab1c-9386d015c797 None None] PATH_INFO: `/` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:47:47.008548 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 807/1619] 10.222.0.13 () {54 vars in 912 bytes} [Wed Aug 31 09:47:47 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:47:47.011034 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2aa8a2-c3e8-4569-9f25-7c81536b91f5 None None] REQUEST_METHOD: `POST` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:47:47.011518 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2aa8a2-c3e8-4569-9f25-7c81536b91f5 None None] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:47:47.012481 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2aa8a2-c3e8-4569-9f25-7c81536b91f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:47:47.018454 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.auth.core [None req-03cec746-a158-449b-ac60-e3ca26ad5206 None None] MFA Rules not processed for user `dc076493239d4ccdbdc37c0369c49529`. Rule list: `[]` (Enabled: `True`). {{(pid=88114) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:47:47.019965 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.fernet_utils [None req-03cec746-a158-449b-ac60-e3ca26ad5206 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88114) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:47:47.058438 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.auth.core [None req-dd2aa8a2-c3e8-4569-9f25-7c81536b91f5 None None] MFA Rules not processed for user `7ccc7825081b4ec79f4b6d9a4feb9969`. Rule list: `[]` (Enabled: `True`). {{(pid=88113) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:47:47.062883 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.fernet_utils [None req-dd2aa8a2-c3e8-4569-9f25-7c81536b91f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:47:47.064546 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:47:47.168762 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 813/1620] 10.222.0.13 () {58 vars in 1059 bytes} [Wed Aug 31 09:47:46 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:47:47.170209 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03cec746-a158-449b-ac60-e3ca26ad5206 None None] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:47:47.183946 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 808/1621] 10.222.0.13 () {58 vars in 958 bytes} [Wed Aug 31 09:47:47 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:47:47.185546 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d96ae7-36aa-4ef2-a650-cc12f1f3056c None None] REQUEST_METHOD: `POST` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:47:47.185989 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d96ae7-36aa-4ef2-a650-cc12f1f3056c None None] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:47:47.186280 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d96ae7-36aa-4ef2-a650-cc12f1f3056c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:47:47.189306 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03cec746-a158-449b-ac60-e3ca26ad5206 None None] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:47:47.207158 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.auth.core [None req-91d96ae7-36aa-4ef2-a650-cc12f1f3056c None None] MFA Rules not processed for user `dc076493239d4ccdbdc37c0369c49529`. Rule list: `[]` (Enabled: `True`). {{(pid=88113) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:47:47.208096 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-861e2a85-6d34-46f5-b716-e80b93778fcc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:47:47.208351 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.fernet_utils [None req-91d96ae7-36aa-4ef2-a650-cc12f1f3056c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:47:47.209487 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861e2a85-6d34-46f5-b716-e80b93778fcc admin admin] REQUEST_METHOD: `POST` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:47:47.210289 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861e2a85-6d34-46f5-b716-e80b93778fcc admin admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:47:47.210601 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861e2a85-6d34-46f5-b716-e80b93778fcc admin admin] PATH_INFO: `/v3/projects` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:47:47.211546 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-861e2a85-6d34-46f5-b716-e80b93778fcc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:47:47.213851 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-861e2a85-6d34-46f5-b716-e80b93778fcc admin admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:47:47.220226 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 809/1622] 10.222.0.13 () {58 vars in 1009 bytes} [Wed Aug 31 09:47:47 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:47:47.222133 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2bf2d9-1509-4bee-bf32-4b6ff64dfcb6 None None] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:47:47.222420 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2bf2d9-1509-4bee-bf32-4b6ff64dfcb6 None None] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:47:47.222699 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2bf2d9-1509-4bee-bf32-4b6ff64dfcb6 None None] PATH_INFO: `/` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:47:47.234386 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 810/1623] 10.222.0.13 () {54 vars in 912 bytes} [Wed Aug 31 09:47:47 2022] GET /identity => generated 270 bytes in 13 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:47:47.235378 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d2bf2d9-1509-4bee-bf32-4b6ff64dfcb6 None None] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:47:47.255307 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d2bf2d9-1509-4bee-bf32-4b6ff64dfcb6 None None] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:47:47.263885 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a6f4e6-08f7-4933-8755-6997398f14d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:47:47.264749 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a6f4e6-08f7-4933-8755-6997398f14d1 admin admin] REQUEST_METHOD: `POST` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:47:47.265075 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a6f4e6-08f7-4933-8755-6997398f14d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:47:47.265478 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a6f4e6-08f7-4933-8755-6997398f14d1 admin admin] PATH_INFO: `/v3/projects` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:47:47.266380 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92a6f4e6-08f7-4933-8755-6997398f14d1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:47:47.268754 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92a6f4e6-08f7-4933-8755-6997398f14d1 admin admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:47:47.275568 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:47:47.295249 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 814/1624] 10.222.0.13 () {60 vars in 1153 bytes} [Wed Aug 31 09:47:47 2022] POST /identity/v3/projects => generated 363 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 09:47:47.297069 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f40ba3-5067-4cba-affa-33e685b7cf79 None None] REQUEST_METHOD: `POST` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:47:47.297365 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f40ba3-5067-4cba-affa-33e685b7cf79 None None] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:47:47.297874 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f40ba3-5067-4cba-affa-33e685b7cf79 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:47:47.319858 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 811/1625] 10.222.0.13 () {60 vars in 1153 bytes} [Wed Aug 31 09:47:47 2022] POST /identity/v3/projects => generated 385 bytes in 85 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 09:47:47.320590 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a6f4e6-08f7-4933-8755-6997398f14d1 admin admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:47:47.330721 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.auth.core [None req-96f40ba3-5067-4cba-affa-33e685b7cf79 None None] MFA Rules not processed for user `dc076493239d4ccdbdc37c0369c49529`. Rule list: `[]` (Enabled: `True`). {{(pid=88114) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:47:47.332616 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.fernet_utils [None req-96f40ba3-5067-4cba-affa-33e685b7cf79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88114) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:47:47.410193 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a6f4e6-08f7-4933-8755-6997398f14d1 admin admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:47:47.415270 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff8f62fe-d400-4971-9b4f-5a788106f28d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d52b69c8e94441dacfcff944326f169', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '49548eb44d62415582af820ab66decee', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:47:47.415811 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8f62fe-d400-4971-9b4f-5a788106f28d service cinder] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:47:47.416209 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8f62fe-d400-4971-9b4f-5a788106f28d service cinder] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:47:47.416385 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8f62fe-d400-4971-9b4f-5a788106f28d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:47:47.476059 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 815/1626] 10.222.0.13 () {58 vars in 1009 bytes} [Wed Aug 31 09:47:47 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:47:47.477815 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f40ba3-5067-4cba-affa-33e685b7cf79 None None] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:47:47.546060 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff8f62fe-d400-4971-9b4f-5a788106f28d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:47:47.556822 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff8f62fe-d400-4971-9b4f-5a788106f28d service cinder] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:47:47.578766 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 812/1627] 10.222.0.13 () {60 vars in 1365 bytes} [Wed Aug 31 09:47:47 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 258 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:47:47.580333 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff8f62fe-d400-4971-9b4f-5a788106f28d service cinder] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:47:47.628266 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f40ba3-5067-4cba-affa-33e685b7cf79 None None] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:47:47.635139 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cfbbc46-3335-446f-a067-f81360aa5fbe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:47:47.635783 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cfbbc46-3335-446f-a067-f81360aa5fbe admin admin] REQUEST_METHOD: `POST` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:47:47.636062 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cfbbc46-3335-446f-a067-f81360aa5fbe admin admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:47:47.636336 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cfbbc46-3335-446f-a067-f81360aa5fbe admin admin] PATH_INFO: `/v3/users` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:47:47.637052 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cfbbc46-3335-446f-a067-f81360aa5fbe admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:47:47.639679 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cfbbc46-3335-446f-a067-f81360aa5fbe admin admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:47:47.677259 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 816/1628] 10.222.0.13 () {60 vars in 1144 bytes} [Wed Aug 31 09:47:47 2022] POST /identity/v3/users => generated 288 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 09:47:47.677259 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cfbbc46-3335-446f-a067-f81360aa5fbe admin admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:47:47.690103 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cfbbc46-3335-446f-a067-f81360aa5fbe admin admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:47:47.694959 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b8fbdec-6638-41a2-b930-b5e0a1d97efd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d52b69c8e94441dacfcff944326f169', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '49548eb44d62415582af820ab66decee', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:47:47.695688 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8fbdec-6638-41a2-b930-b5e0a1d97efd service cinder] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:47:47.696101 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8fbdec-6638-41a2-b930-b5e0a1d97efd service cinder] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:47:47.696377 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8fbdec-6638-41a2-b930-b5e0a1d97efd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:47:47.712228 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b8fbdec-6638-41a2-b930-b5e0a1d97efd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:47:47.714788 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b8fbdec-6638-41a2-b930-b5e0a1d97efd service cinder] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:47:47.741955 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 817/1629] 10.222.0.13 () {60 vars in 1365 bytes} [Wed Aug 31 09:47:47 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 66 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:47:47.746537 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b8fbdec-6638-41a2-b930-b5e0a1d97efd service cinder] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:47:47.775164 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b8fbdec-6638-41a2-b930-b5e0a1d97efd service cinder] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:47:47.782855 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff8f62fe-d400-4971-9b4f-5a788106f28d service cinder] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:47:47.786424 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ea4d0d-aca6-4b0f-90c9-408de378b410 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:47:47.788918 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ea4d0d-aca6-4b0f-90c9-408de378b410 admin admin] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:47:47.789041 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab5954b-a959-4baa-85f2-0db34e28351b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:47:47.789707 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab5954b-a959-4baa-85f2-0db34e28351b admin admin] REQUEST_METHOD: `POST` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:47:47.789825 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ea4d0d-aca6-4b0f-90c9-408de378b410 admin admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:47:47.790007 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab5954b-a959-4baa-85f2-0db34e28351b admin admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:47:47.790116 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ea4d0d-aca6-4b0f-90c9-408de378b410 admin admin] PATH_INFO: `/v3/roles` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:47:47.790276 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab5954b-a959-4baa-85f2-0db34e28351b admin admin] PATH_INFO: `/v3/users` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:47:47.790940 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ab5954b-a959-4baa-85f2-0db34e28351b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:47:47.791727 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87ea4d0d-aca6-4b0f-90c9-408de378b410 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:47:47.793291 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ab5954b-a959-4baa-85f2-0db34e28351b admin admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:47:47.796144 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87ea4d0d-aca6-4b0f-90c9-408de378b410 admin admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:47:47.803219 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:47:47.825893 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 813/1630] 10.222.0.13 () {60 vars in 1144 bytes} [Wed Aug 31 09:47:47 2022] POST /identity/v3/users => generated 299 bytes in 247 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 09:47:47.831760 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 818/1631] 10.222.0.13 () {58 vars in 1122 bytes} [Wed Aug 31 09:47:47 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 09:47:47.831946 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab5954b-a959-4baa-85f2-0db34e28351b admin admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:47:47.833118 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ea4d0d-aca6-4b0f-90c9-408de378b410 admin admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:47:47.847614 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab5954b-a959-4baa-85f2-0db34e28351b admin admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:47:47.854003 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a4d9cd-02d2-472a-96de-516ce1c03217 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:47:47.854633 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a4d9cd-02d2-472a-96de-516ce1c03217 admin admin] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:47:47.854911 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a4d9cd-02d2-472a-96de-516ce1c03217 admin admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:47:47.855181 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a4d9cd-02d2-472a-96de-516ce1c03217 admin admin] PATH_INFO: `/v3/roles` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:47:47.855838 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1a4d9cd-02d2-472a-96de-516ce1c03217 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:47:47.857871 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1a4d9cd-02d2-472a-96de-516ce1c03217 admin admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:47:47.858533 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ea4d0d-aca6-4b0f-90c9-408de378b410 admin admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:47:47.869122 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-340da077-92b6-4edc-8b7e-18e0e12df4fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:47:47.872057 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340da077-92b6-4edc-8b7e-18e0e12df4fe admin admin] REQUEST_METHOD: `PUT` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:47:47.872349 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340da077-92b6-4edc-8b7e-18e0e12df4fe admin admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:47:47.872631 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340da077-92b6-4edc-8b7e-18e0e12df4fe admin admin] PATH_INFO: `/v3/projects/c68a2bf348bc4da6b00e539140456f3f/users/3e4613bf47b84243bf8a2ad3aed3af03/roles/5d4ee63a3add47fc9a6b8b16d39e0a1f` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:47:47.881969 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 814/1632] 10.222.0.13 () {58 vars in 1122 bytes} [Wed Aug 31 09:47:47 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 09:47:47.887406 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a4d9cd-02d2-472a-96de-516ce1c03217 admin admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:47:47.895242 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:47:47.899191 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-340da077-92b6-4edc-8b7e-18e0e12df4fe admin admin] RBAC: Authorizing `identity:create_grant(project_id=c68a2bf348bc4da6b00e539140456f3f, user_id=3e4613bf47b84243bf8a2ad3aed3af03, role_id=5d4ee63a3add47fc9a6b8b16d39e0a1f)` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:47:47.902203 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-340da077-92b6-4edc-8b7e-18e0e12df4fe admin admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:47:47.905714 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a4d9cd-02d2-472a-96de-516ce1c03217 admin admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:47:47.915319 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52724bd5-9235-4447-b35a-9610b4549a6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:47:47.916605 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52724bd5-9235-4447-b35a-9610b4549a6b admin admin] REQUEST_METHOD: `PUT` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:47:47.917431 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52724bd5-9235-4447-b35a-9610b4549a6b admin admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:47:47.918563 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52724bd5-9235-4447-b35a-9610b4549a6b admin admin] PATH_INFO: `/v3/projects/a2e59c137309422787c48463fcb9f7c6/users/9ad4450133404a7e921fbce29fa5e809/roles/5d4ee63a3add47fc9a6b8b16d39e0a1f` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:47:47.932481 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 819/1633] 10.222.0.13 () {60 vars in 1483 bytes} [Wed Aug 31 09:47:47 2022] PUT /identity/v3/projects/c68a2bf348bc4da6b00e539140456f3f/users/3e4613bf47b84243bf8a2ad3aed3af03/roles/5d4ee63a3add47fc9a6b8b16d39e0a1f => generated 0 bytes in 97 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:47:47.934545 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-340da077-92b6-4edc-8b7e-18e0e12df4fe admin admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:47:47.967682 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52724bd5-9235-4447-b35a-9610b4549a6b admin admin] RBAC: Authorizing `identity:create_grant(project_id=a2e59c137309422787c48463fcb9f7c6, user_id=9ad4450133404a7e921fbce29fa5e809, role_id=5d4ee63a3add47fc9a6b8b16d39e0a1f)` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:47:47.973206 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52724bd5-9235-4447-b35a-9610b4549a6b admin admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:47:48.011090 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 815/1634] 10.222.0.13 () {60 vars in 1483 bytes} [Wed Aug 31 09:47:47 2022] PUT /identity/v3/projects/a2e59c137309422787c48463fcb9f7c6/users/9ad4450133404a7e921fbce29fa5e809/roles/5d4ee63a3add47fc9a6b8b16d39e0a1f => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:47:48.013595 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52724bd5-9235-4447-b35a-9610b4549a6b admin admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:47:48.109340 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-340da077-92b6-4edc-8b7e-18e0e12df4fe admin admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:47:48.148271 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52724bd5-9235-4447-b35a-9610b4549a6b admin admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:47:48.163924 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e2a4de7-5528-4146-a1e0-efd7d8105417 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:47:48.166068 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2a4de7-5528-4146-a1e0-efd7d8105417 admin admin] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:47:48.166348 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2a4de7-5528-4146-a1e0-efd7d8105417 admin admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:47:48.169369 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2a4de7-5528-4146-a1e0-efd7d8105417 admin admin] PATH_INFO: `/v3/roles` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:47:48.172144 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e2a4de7-5528-4146-a1e0-efd7d8105417 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:47:48.175452 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e2a4de7-5528-4146-a1e0-efd7d8105417 admin admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:47:48.216617 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 816/1635] 10.222.0.13 () {58 vars in 1122 bytes} [Wed Aug 31 09:47:48 2022] GET /identity/v3/roles => generated 1420 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 09:47:48.216845 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e2a4de7-5528-4146-a1e0-efd7d8105417 admin admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:47:48.229445 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-148bdd8f-134e-4503-93b5-ab4f138d74bd service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc076493239d4ccdbdc37c0369c49529', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '49548eb44d62415582af820ab66decee', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:47:48.230238 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148bdd8f-134e-4503-93b5-ab4f138d74bd service nova] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:47:48.230626 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148bdd8f-134e-4503-93b5-ab4f138d74bd service nova] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:47:48.231005 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148bdd8f-134e-4503-93b5-ab4f138d74bd service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:47:48.252529 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e2a4de7-5528-4146-a1e0-efd7d8105417 admin admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:47:48.267612 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31689f6e-bbad-4adb-833d-08cb46accf2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:47:48.270012 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31689f6e-bbad-4adb-833d-08cb46accf2e admin admin] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:47:48.274760 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31689f6e-bbad-4adb-833d-08cb46accf2e admin admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:47:48.275931 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31689f6e-bbad-4adb-833d-08cb46accf2e admin admin] PATH_INFO: `/v3/roles` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:47:48.276941 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31689f6e-bbad-4adb-833d-08cb46accf2e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:47:48.282861 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31689f6e-bbad-4adb-833d-08cb46accf2e admin admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:47:48.312426 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 817/1636] 10.222.0.13 () {58 vars in 1122 bytes} [Wed Aug 31 09:47:48 2022] GET /identity/v3/roles => generated 1420 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 09:47:48.313438 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31689f6e-bbad-4adb-833d-08cb46accf2e admin admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:47:48.339679 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31689f6e-bbad-4adb-833d-08cb46accf2e admin admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:47:48.345995 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-148bdd8f-134e-4503-93b5-ab4f138d74bd service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:47:48.352500 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-148bdd8f-134e-4503-93b5-ab4f138d74bd service nova] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:47:48.352819 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f7248a3-9c21-4fc3-bbd0-eeb6606da9a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:47:48.353339 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7248a3-9c21-4fc3-bbd0-eeb6606da9a3 admin admin] REQUEST_METHOD: `PUT` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:47:48.353647 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7248a3-9c21-4fc3-bbd0-eeb6606da9a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:47:48.353924 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7248a3-9c21-4fc3-bbd0-eeb6606da9a3 admin admin] PATH_INFO: `/v3/projects/c68a2bf348bc4da6b00e539140456f3f/users/3e4613bf47b84243bf8a2ad3aed3af03/roles/5d4ee63a3add47fc9a6b8b16d39e0a1f` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:47:48.359009 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f7248a3-9c21-4fc3-bbd0-eeb6606da9a3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c68a2bf348bc4da6b00e539140456f3f, user_id=3e4613bf47b84243bf8a2ad3aed3af03, role_id=5d4ee63a3add47fc9a6b8b16d39e0a1f)` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:47:48.366051 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f7248a3-9c21-4fc3-bbd0-eeb6606da9a3 admin admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:47:48.371421 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 820/1637] 10.222.0.13 () {60 vars in 1365 bytes} [Wed Aug 31 09:47:47 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 441 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:47:48.374308 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-148bdd8f-134e-4503-93b5-ab4f138d74bd service nova] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:47:48.416236 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 818/1638] 10.222.0.13 () {60 vars in 1483 bytes} [Wed Aug 31 09:47:48 2022] PUT /identity/v3/projects/c68a2bf348bc4da6b00e539140456f3f/users/3e4613bf47b84243bf8a2ad3aed3af03/roles/5d4ee63a3add47fc9a6b8b16d39e0a1f => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:47:48.417303 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-148bdd8f-134e-4503-93b5-ab4f138d74bd service nova] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:47:48.426151 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70603c0b-9e8c-4201-82a5-ba0f3b69afe6 None None] REQUEST_METHOD: `POST` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:47:48.426438 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70603c0b-9e8c-4201-82a5-ba0f3b69afe6 None None] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:47:48.426717 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70603c0b-9e8c-4201-82a5-ba0f3b69afe6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:47:48.437655 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:47:48.536424 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.auth.core [None req-70603c0b-9e8c-4201-82a5-ba0f3b69afe6 None None] MFA Rules not processed for user `3e4613bf47b84243bf8a2ad3aed3af03`. Rule list: `[]` (Enabled: `True`). {{(pid=88113) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:47:48.537864 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.fernet_utils [None req-70603c0b-9e8c-4201-82a5-ba0f3b69afe6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:47:48.654116 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c25b4ade-8353-4461-b501-322a66ad0d6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:47:48.656300 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25b4ade-8353-4461-b501-322a66ad0d6b admin admin] REQUEST_METHOD: `PUT` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:47:48.656610 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25b4ade-8353-4461-b501-322a66ad0d6b admin admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:47:48.657569 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25b4ade-8353-4461-b501-322a66ad0d6b admin admin] PATH_INFO: `/v3/projects/a2e59c137309422787c48463fcb9f7c6/users/9ad4450133404a7e921fbce29fa5e809/roles/5d4ee63a3add47fc9a6b8b16d39e0a1f` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:47:48.666339 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c25b4ade-8353-4461-b501-322a66ad0d6b admin admin] RBAC: Authorizing `identity:create_grant(project_id=a2e59c137309422787c48463fcb9f7c6, user_id=9ad4450133404a7e921fbce29fa5e809, role_id=5d4ee63a3add47fc9a6b8b16d39e0a1f)` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:47:48.675808 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c25b4ade-8353-4461-b501-322a66ad0d6b admin admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:47:48.725265 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 821/1639] 10.222.0.13 () {60 vars in 1483 bytes} [Wed Aug 31 09:47:48 2022] PUT /identity/v3/projects/a2e59c137309422787c48463fcb9f7c6/users/9ad4450133404a7e921fbce29fa5e809/roles/5d4ee63a3add47fc9a6b8b16d39e0a1f => generated 0 bytes in 341 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:47:48.728820 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8b935b-1477-4ed5-92d7-7329fb9c4220 None None] REQUEST_METHOD: `POST` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:47:48.729107 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8b935b-1477-4ed5-92d7-7329fb9c4220 None None] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:47:48.730867 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8b935b-1477-4ed5-92d7-7329fb9c4220 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:47:48.741596 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 819/1640] 10.222.0.13 () {58 vars in 958 bytes} [Wed Aug 31 09:47:48 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 313 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:47:48.824353 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.auth.core [None req-ad8b935b-1477-4ed5-92d7-7329fb9c4220 None None] MFA Rules not processed for user `9ad4450133404a7e921fbce29fa5e809`. Rule list: `[]` (Enabled: `True`). {{(pid=88114) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:47:48.827848 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.fernet_utils [None req-ad8b935b-1477-4ed5-92d7-7329fb9c4220 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88114) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:47:48.844388 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70603c0b-9e8c-4201-82a5-ba0f3b69afe6 None None] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:47:48.992478 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:47:49.027956 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 822/1641] 10.222.0.13 () {58 vars in 958 bytes} [Wed Aug 31 09:47:48 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 302 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:47:49.046379 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70603c0b-9e8c-4201-82a5-ba0f3b69afe6 None None] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:47:49.179483 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4886543d-db80-46f1-b7fb-fc602c1a790c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d52b69c8e94441dacfcff944326f169', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '49548eb44d62415582af820ab66decee', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:47:49.180418 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4886543d-db80-46f1-b7fb-fc602c1a790c service cinder] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:47:49.180829 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4886543d-db80-46f1-b7fb-fc602c1a790c service cinder] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:47:49.181578 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4886543d-db80-46f1-b7fb-fc602c1a790c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:47:49.283844 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4886543d-db80-46f1-b7fb-fc602c1a790c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:47:49.285605 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4886543d-db80-46f1-b7fb-fc602c1a790c service cinder] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:47:49.319393 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 820/1642] 10.222.0.13 () {60 vars in 1365 bytes} [Wed Aug 31 09:47:48 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 475 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:47:49.370123 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad8b935b-1477-4ed5-92d7-7329fb9c4220 None None] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:47:49.383262 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad8b935b-1477-4ed5-92d7-7329fb9c4220 None None] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:47:49.394125 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da50366-baa1-4582-9221-541beb42d4e4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d52b69c8e94441dacfcff944326f169', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '49548eb44d62415582af820ab66decee', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:47:49.395096 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da50366-baa1-4582-9221-541beb42d4e4 service cinder] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:47:49.395096 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da50366-baa1-4582-9221-541beb42d4e4 service cinder] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:47:49.395316 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da50366-baa1-4582-9221-541beb42d4e4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:47:49.416037 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0da50366-baa1-4582-9221-541beb42d4e4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:47:49.417840 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0da50366-baa1-4582-9221-541beb42d4e4 service cinder] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:47:49.441802 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 823/1643] 10.222.0.13 () {60 vars in 1365 bytes} [Wed Aug 31 09:47:49 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 72 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:47:49.687927 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4886543d-db80-46f1-b7fb-fc602c1a790c service cinder] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:47:49.704889 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da50366-baa1-4582-9221-541beb42d4e4 service cinder] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:47:49.718350 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da50366-baa1-4582-9221-541beb42d4e4 service cinder] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:47:49.734924 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be2d3aac-855b-4dbc-a4c3-32fac47ee277 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d52b69c8e94441dacfcff944326f169', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '49548eb44d62415582af820ab66decee', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:47:49.737961 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2d3aac-855b-4dbc-a4c3-32fac47ee277 service cinder] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:47:49.738265 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2d3aac-855b-4dbc-a4c3-32fac47ee277 service cinder] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:47:49.738543 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2d3aac-855b-4dbc-a4c3-32fac47ee277 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:47:49.860752 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4886543d-db80-46f1-b7fb-fc602c1a790c service cinder] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:47:49.862783 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be2d3aac-855b-4dbc-a4c3-32fac47ee277 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:47:49.864365 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be2d3aac-855b-4dbc-a4c3-32fac47ee277 service cinder] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:47:49.868020 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc9cb9de-3ca9-4d4f-b951-85e1c62d3383 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:47:49.869008 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9cb9de-3ca9-4d4f-b951-85e1c62d3383 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:47:49.869540 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9cb9de-3ca9-4d4f-b951-85e1c62d3383 admin admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:47:49.870056 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9cb9de-3ca9-4d4f-b951-85e1c62d3383 admin admin] PATH_INFO: `/v3/users/7ccc7825081b4ec79f4b6d9a4feb9969` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:47:49.871548 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc9cb9de-3ca9-4d4f-b951-85e1c62d3383 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7ccc7825081b4ec79f4b6d9a4feb9969)` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:47:49.872885 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: WARNING py.warnings [None req-bc9cb9de-3ca9-4d4f-b951-85e1c62d3383 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 31 09:47:49.872885 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: warnings.warn(msg) Aug 31 09:47:49.872885 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: Aug 31 09:47:49.874565 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc9cb9de-3ca9-4d4f-b951-85e1c62d3383 admin admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:47:49.888128 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 824/1644] 10.222.0.13 () {60 vars in 1365 bytes} [Wed Aug 31 09:47:49 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 184 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 09:47:49.901991 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be2d3aac-855b-4dbc-a4c3-32fac47ee277 service cinder] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:47:49.940958 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be2d3aac-855b-4dbc-a4c3-32fac47ee277 service cinder] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:47:49.957148 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-664ee9e2-99cb-48c9-8117-6a64dac18f75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:47:49.957964 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664ee9e2-99cb-48c9-8117-6a64dac18f75 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:47:49.961890 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664ee9e2-99cb-48c9-8117-6a64dac18f75 admin admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:47:49.962228 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664ee9e2-99cb-48c9-8117-6a64dac18f75 admin admin] PATH_INFO: `/v3/users/bf41ca204722491abce3c27f2ff87126` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:47:49.964551 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-664ee9e2-99cb-48c9-8117-6a64dac18f75 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bf41ca204722491abce3c27f2ff87126)` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:47:49.973110 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-664ee9e2-99cb-48c9-8117-6a64dac18f75 admin admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:47:49.994798 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:47:50.002362 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.notifications [None req-bc9cb9de-3ca9-4d4f-b951-85e1c62d3383 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7ccc7825081b4ec79f4b6d9a4feb9969'} {{(pid=88113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:47:50.009687 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:47:50.054744 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.notifications [None req-bc9cb9de-3ca9-4d4f-b951-85e1c62d3383 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7ccc7825081b4ec79f4b6d9a4feb9969'} {{(pid=88113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:47:50.155902 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.notifications [None req-664ee9e2-99cb-48c9-8117-6a64dac18f75 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bf41ca204722491abce3c27f2ff87126'} {{(pid=88114) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:47:50.176716 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.notifications [None req-bc9cb9de-3ca9-4d4f-b951-85e1c62d3383 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7ccc7825081b4ec79f4b6d9a4feb9969'} {{(pid=88113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:47:50.179095 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.notifications [None req-664ee9e2-99cb-48c9-8117-6a64dac18f75 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bf41ca204722491abce3c27f2ff87126'} {{(pid=88114) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:47:50.180192 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.notifications [None req-bc9cb9de-3ca9-4d4f-b951-85e1c62d3383 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7ccc7825081b4ec79f4b6d9a4feb9969'} {{(pid=88113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:47:50.200630 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.notifications [None req-664ee9e2-99cb-48c9-8117-6a64dac18f75 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bf41ca204722491abce3c27f2ff87126'} {{(pid=88114) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:47:50.201489 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.notifications [None req-664ee9e2-99cb-48c9-8117-6a64dac18f75 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bf41ca204722491abce3c27f2ff87126'} {{(pid=88114) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:47:50.218939 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.notifications [None req-bc9cb9de-3ca9-4d4f-b951-85e1c62d3383 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7ccc7825081b4ec79f4b6d9a4feb9969'} {{(pid=88113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:47:50.242351 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.notifications [None req-664ee9e2-99cb-48c9-8117-6a64dac18f75 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bf41ca204722491abce3c27f2ff87126'} {{(pid=88114) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:47:50.262860 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 821/1645] 10.222.0.13 () {58 vars in 1224 bytes} [Wed Aug 31 09:47:49 2022] DELETE /identity/v3/users/7ccc7825081b4ec79f4b6d9a4feb9969 => generated 0 bytes in 579 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:47:50.266520 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 825/1646] 10.222.0.13 () {58 vars in 1224 bytes} [Wed Aug 31 09:47:49 2022] DELETE /identity/v3/users/bf41ca204722491abce3c27f2ff87126 => generated 0 bytes in 378 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:47:50.269068 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb94b14-9d29-4c76-8039-a35baf549cda None None] REQUEST_METHOD: `POST` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:47:50.270727 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b4618e-2cc6-45e8-a538-4807fd2ae585 None None] REQUEST_METHOD: `POST` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:47:50.272466 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b4618e-2cc6-45e8-a538-4807fd2ae585 None None] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:47:50.272866 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b4618e-2cc6-45e8-a538-4807fd2ae585 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:47:50.273752 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb94b14-9d29-4c76-8039-a35baf549cda None None] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:47:50.274197 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb94b14-9d29-4c76-8039-a35baf549cda None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:47:50.341863 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.auth.core [None req-78b4618e-2cc6-45e8-a538-4807fd2ae585 None None] MFA Rules not processed for user `fd836cf590a145df9acf19cf3ae2ac56`. Rule list: `[]` (Enabled: `True`). {{(pid=88114) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:47:50.341863 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.fernet_utils [None req-78b4618e-2cc6-45e8-a538-4807fd2ae585 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88114) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:47:50.342895 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.auth.core [None req-ceb94b14-9d29-4c76-8039-a35baf549cda None None] MFA Rules not processed for user `fd836cf590a145df9acf19cf3ae2ac56`. Rule list: `[]` (Enabled: `True`). {{(pid=88113) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:47:50.342895 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.fernet_utils [None req-ceb94b14-9d29-4c76-8039-a35baf549cda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:47:50.501803 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 826/1647] 10.222.0.13 () {58 vars in 958 bytes} [Wed Aug 31 09:47:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 232 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:47:50.531230 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 822/1648] 10.222.0.13 () {58 vars in 958 bytes} [Wed Aug 31 09:47:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 263 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 09:47:50.557819 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b4618e-2cc6-45e8-a538-4807fd2ae585 None None] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:47:50.559394 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.fernet_utils [None req-78b4618e-2cc6-45e8-a538-4807fd2ae585 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88114) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:47:50.568207 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb94b14-9d29-4c76-8039-a35baf549cda None None] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:47:50.570537 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.fernet_utils [None req-ceb94b14-9d29-4c76-8039-a35baf549cda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:47:50.666180 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b4618e-2cc6-45e8-a538-4807fd2ae585 None None] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:47:50.670572 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a6b7ff0-453a-4278-ad16-20cedb7ac82d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9bdaacd597442c2bcde2c555f2cc140', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '49548eb44d62415582af820ab66decee', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:47:50.671371 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6b7ff0-453a-4278-ad16-20cedb7ac82d service neutron] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:47:50.671517 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6b7ff0-453a-4278-ad16-20cedb7ac82d service neutron] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:47:50.671786 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6b7ff0-453a-4278-ad16-20cedb7ac82d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:47:50.685614 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a6b7ff0-453a-4278-ad16-20cedb7ac82d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:47:50.687087 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a6b7ff0-453a-4278-ad16-20cedb7ac82d service neutron] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:47:50.699216 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 827/1649] 10.222.0.13 () {60 vars in 1365 bytes} [Wed Aug 31 09:47:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:47:50.741821 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb94b14-9d29-4c76-8039-a35baf549cda None None] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:47:50.752337 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa43c1a-e422-4417-832f-bbe58a303176 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9bdaacd597442c2bcde2c555f2cc140', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '49548eb44d62415582af820ab66decee', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:47:50.753306 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa43c1a-e422-4417-832f-bbe58a303176 service neutron] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:47:50.753693 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa43c1a-e422-4417-832f-bbe58a303176 service neutron] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:47:50.754624 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa43c1a-e422-4417-832f-bbe58a303176 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:47:50.785570 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aa43c1a-e422-4417-832f-bbe58a303176 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:47:50.789440 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aa43c1a-e422-4417-832f-bbe58a303176 service neutron] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:47:50.842485 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 823/1650] 10.222.0.13 () {60 vars in 1365 bytes} [Wed Aug 31 09:47:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 275 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 09:47:51.989206 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a6b7ff0-453a-4278-ad16-20cedb7ac82d service neutron] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:47:51.991241 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.fernet_utils [None req-2a6b7ff0-453a-4278-ad16-20cedb7ac82d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88114) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:47:52.013039 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a6b7ff0-453a-4278-ad16-20cedb7ac82d service neutron] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:47:52.029005 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f98bfe-7bf2-4a2e-b061-33074fe10253 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:47:52.029356 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f98bfe-7bf2-4a2e-b061-33074fe10253 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:47:52.029690 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f98bfe-7bf2-4a2e-b061-33074fe10253 admin admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:47:52.029972 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f98bfe-7bf2-4a2e-b061-33074fe10253 admin admin] PATH_INFO: `/v3/projects/1073c2fa1d8a4810a4846b7ccd823a19` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:47:52.045003 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6f98bfe-7bf2-4a2e-b061-33074fe10253 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1073c2fa1d8a4810a4846b7ccd823a19)` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:47:52.049209 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: WARNING py.warnings [None req-a6f98bfe-7bf2-4a2e-b061-33074fe10253 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 31 09:47:52.049209 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: warnings.warn(msg) Aug 31 09:47:52.049209 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: Aug 31 09:47:52.054307 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6f98bfe-7bf2-4a2e-b061-33074fe10253 admin admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:47:52.178843 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:47:52.270497 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.notifications [None req-a6f98bfe-7bf2-4a2e-b061-33074fe10253 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1073c2fa1d8a4810a4846b7ccd823a19', 'request_id': 'req-a6f98bfe-7bf2-4a2e-b061-33074fe10253'} {{(pid=88114) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:47:52.319039 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.notifications [None req-a6f98bfe-7bf2-4a2e-b061-33074fe10253 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1073c2fa1d8a4810a4846b7ccd823a19', 'request_id': 'req-a6f98bfe-7bf2-4a2e-b061-33074fe10253'} {{(pid=88114) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:47:52.346470 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.notifications [None req-a6f98bfe-7bf2-4a2e-b061-33074fe10253 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1073c2fa1d8a4810a4846b7ccd823a19', 'request_id': 'req-a6f98bfe-7bf2-4a2e-b061-33074fe10253'} {{(pid=88114) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:47:52.388243 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.notifications [None req-a6f98bfe-7bf2-4a2e-b061-33074fe10253 admin admin] The token cache is being invalidate because project 1073c2fa1d8a4810a4846b7ccd823a19 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88114) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 09:47:52.388554 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.notifications [None req-a6f98bfe-7bf2-4a2e-b061-33074fe10253 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88114) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:47:52.399056 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.notifications [None req-a6f98bfe-7bf2-4a2e-b061-33074fe10253 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88114) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:47:52.407898 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65262efd-8028-4182-a940-5dd3f0584023 None None] REQUEST_METHOD: `POST` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:47:52.408436 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65262efd-8028-4182-a940-5dd3f0584023 None None] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:47:52.408436 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65262efd-8028-4182-a940-5dd3f0584023 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:47:52.412177 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 828/1651] 10.222.0.13 () {58 vars in 1233 bytes} [Wed Aug 31 09:47:51 2022] DELETE /identity/v3/projects/1073c2fa1d8a4810a4846b7ccd823a19 => generated 0 bytes in 414 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 09:47:52.459511 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.auth.core [None req-65262efd-8028-4182-a940-5dd3f0584023 None None] MFA Rules not processed for user `fd836cf590a145df9acf19cf3ae2ac56`. Rule list: `[]` (Enabled: `True`). {{(pid=88113) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:47:52.460927 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.fernet_utils [None req-65262efd-8028-4182-a940-5dd3f0584023 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:47:52.598911 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 824/1652] 10.222.0.13 () {58 vars in 958 bytes} [Wed Aug 31 09:47:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:47:52.603803 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e131624b-ae68-436f-b6e7-d02b3433ea0b None None] REQUEST_METHOD: `POST` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:47:52.604093 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e131624b-ae68-436f-b6e7-d02b3433ea0b None None] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:47:52.604368 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e131624b-ae68-436f-b6e7-d02b3433ea0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:47:52.645551 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.auth.core [None req-e131624b-ae68-436f-b6e7-d02b3433ea0b None None] MFA Rules not processed for user `fd836cf590a145df9acf19cf3ae2ac56`. Rule list: `[]` (Enabled: `True`). {{(pid=88114) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:47:52.647349 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.fernet_utils [None req-e131624b-ae68-436f-b6e7-d02b3433ea0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88114) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:47:52.674103 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 829/1653] 10.222.0.13 () {58 vars in 958 bytes} [Wed Aug 31 09:47:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 71 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:47:52.675346 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65262efd-8028-4182-a940-5dd3f0584023 None None] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:47:52.689241 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65262efd-8028-4182-a940-5dd3f0584023 None None] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:47:52.701109 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcdf6c27-53de-48ae-96ae-5979ac28bf4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:47:52.703418 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcdf6c27-53de-48ae-96ae-5979ac28bf4d admin admin] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:47:52.704015 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcdf6c27-53de-48ae-96ae-5979ac28bf4d admin admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:47:52.705157 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcdf6c27-53de-48ae-96ae-5979ac28bf4d admin admin] PATH_INFO: `/v3/domains` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:47:52.707752 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcdf6c27-53de-48ae-96ae-5979ac28bf4d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88113) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:47:52.708296 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcdf6c27-53de-48ae-96ae-5979ac28bf4d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:47:52.713627 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcdf6c27-53de-48ae-96ae-5979ac28bf4d admin admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:47:52.718104 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=96,keystone:DELETE=3,keystone:INSERT=1 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:47:52.780466 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 825/1654] 10.222.0.13 () {58 vars in 1153 bytes} [Wed Aug 31 09:47:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 105 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:47:52.783577 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e131624b-ae68-436f-b6e7-d02b3433ea0b None None] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:47:52.797251 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e131624b-ae68-436f-b6e7-d02b3433ea0b None None] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:47:52.816868 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5615579-e809-4fc9-85e4-ca5b8c77d015 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:47:52.817571 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5615579-e809-4fc9-85e4-ca5b8c77d015 admin admin] REQUEST_METHOD: `POST` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:47:52.818401 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5615579-e809-4fc9-85e4-ca5b8c77d015 admin admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:47:52.818677 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5615579-e809-4fc9-85e4-ca5b8c77d015 admin admin] PATH_INFO: `/v3/projects` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:47:52.819331 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5615579-e809-4fc9-85e4-ca5b8c77d015 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:47:52.821302 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5615579-e809-4fc9-85e4-ca5b8c77d015 admin admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:47:52.871106 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcdf6c27-53de-48ae-96ae-5979ac28bf4d admin admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:47:52.875598 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.fernet_utils [None req-bcdf6c27-53de-48ae-96ae-5979ac28bf4d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:47:52.928297 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcdf6c27-53de-48ae-96ae-5979ac28bf4d admin admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:47:52.955159 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 830/1655] 10.222.0.13 () {60 vars in 1153 bytes} [Wed Aug 31 09:47:52 2022] POST /identity/v3/projects => generated 403 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 09:47:52.955656 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-743f7c79-1418-420d-a287-d1d7fb41e829 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:47:52.956131 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5615579-e809-4fc9-85e4-ca5b8c77d015 admin admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:47:52.957152 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743f7c79-1418-420d-a287-d1d7fb41e829 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:47:52.957483 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743f7c79-1418-420d-a287-d1d7fb41e829 admin admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:47:52.957763 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743f7c79-1418-420d-a287-d1d7fb41e829 admin admin] PATH_INFO: `/v3/users/9bfb4fba581c45bbb1101be63a110025` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:47:52.964195 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.fernet_utils [None req-e5615579-e809-4fc9-85e4-ca5b8c77d015 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88114) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:47:52.964998 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-743f7c79-1418-420d-a287-d1d7fb41e829 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9bfb4fba581c45bbb1101be63a110025)` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:47:52.976229 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-743f7c79-1418-420d-a287-d1d7fb41e829 admin admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:47:53.202750 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5615579-e809-4fc9-85e4-ca5b8c77d015 admin admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:47:53.218663 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.notifications [None req-743f7c79-1418-420d-a287-d1d7fb41e829 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9bfb4fba581c45bbb1101be63a110025'} {{(pid=88113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:47:53.219610 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62b471c1-9365-4eac-9483-0f70f1b66938 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:47:53.222383 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b471c1-9365-4eac-9483-0f70f1b66938 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:47:53.222383 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b471c1-9365-4eac-9483-0f70f1b66938 admin admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:47:53.222383 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b471c1-9365-4eac-9483-0f70f1b66938 admin admin] PATH_INFO: `/v3/projects/365f6597f247407da4b61f4d50aa1503` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:47:53.222903 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62b471c1-9365-4eac-9483-0f70f1b66938 admin admin] RBAC: Authorizing `identity:delete_project(project_id=365f6597f247407da4b61f4d50aa1503)` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:47:53.226144 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62b471c1-9365-4eac-9483-0f70f1b66938 admin admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:47:53.249461 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.notifications [None req-743f7c79-1418-420d-a287-d1d7fb41e829 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9bfb4fba581c45bbb1101be63a110025'} {{(pid=88113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:47:53.304643 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.notifications [None req-743f7c79-1418-420d-a287-d1d7fb41e829 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9bfb4fba581c45bbb1101be63a110025'} {{(pid=88113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:47:53.305604 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.notifications [None req-743f7c79-1418-420d-a287-d1d7fb41e829 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9bfb4fba581c45bbb1101be63a110025'} {{(pid=88113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:47:53.332713 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.notifications [None req-743f7c79-1418-420d-a287-d1d7fb41e829 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9bfb4fba581c45bbb1101be63a110025'} {{(pid=88113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:47:53.348148 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 826/1656] 10.222.0.13 () {58 vars in 1224 bytes} [Wed Aug 31 09:47:52 2022] DELETE /identity/v3/users/9bfb4fba581c45bbb1101be63a110025 => generated 0 bytes in 481 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:47:53.350328 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-743f7c79-1418-420d-a287-d1d7fb41e829 admin admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:47:53.358289 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.fernet_utils [None req-743f7c79-1418-420d-a287-d1d7fb41e829 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:47:53.373778 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.notifications [None req-62b471c1-9365-4eac-9483-0f70f1b66938 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '365f6597f247407da4b61f4d50aa1503', 'request_id': 'req-62b471c1-9365-4eac-9483-0f70f1b66938'} {{(pid=88114) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:47:53.388277 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.notifications [None req-62b471c1-9365-4eac-9483-0f70f1b66938 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '365f6597f247407da4b61f4d50aa1503', 'request_id': 'req-62b471c1-9365-4eac-9483-0f70f1b66938'} {{(pid=88114) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:47:53.403123 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.notifications [None req-62b471c1-9365-4eac-9483-0f70f1b66938 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '365f6597f247407da4b61f4d50aa1503', 'request_id': 'req-62b471c1-9365-4eac-9483-0f70f1b66938'} {{(pid=88114) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:47:53.427733 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.notifications [None req-62b471c1-9365-4eac-9483-0f70f1b66938 admin admin] The token cache is being invalidate because project 365f6597f247407da4b61f4d50aa1503 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88114) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 09:47:53.428034 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.notifications [None req-62b471c1-9365-4eac-9483-0f70f1b66938 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88114) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:47:53.429748 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.notifications [None req-62b471c1-9365-4eac-9483-0f70f1b66938 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88114) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:47:53.431762 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 831/1657] 10.222.0.13 () {58 vars in 1233 bytes} [Wed Aug 31 09:47:52 2022] DELETE /identity/v3/projects/365f6597f247407da4b61f4d50aa1503 => generated 0 bytes in 476 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:47:53.437572 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc49f4b1-9a9a-4f81-9b87-1f6827db8c4c None None] REQUEST_METHOD: `POST` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:47:53.438212 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc49f4b1-9a9a-4f81-9b87-1f6827db8c4c None None] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:47:53.438630 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc49f4b1-9a9a-4f81-9b87-1f6827db8c4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:47:53.493205 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.auth.core [None req-fc49f4b1-9a9a-4f81-9b87-1f6827db8c4c None None] MFA Rules not processed for user `fd836cf590a145df9acf19cf3ae2ac56`. Rule list: `[]` (Enabled: `True`). {{(pid=88114) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:47:53.494705 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.fernet_utils [None req-fc49f4b1-9a9a-4f81-9b87-1f6827db8c4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88114) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:47:53.512688 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=84,keystone:DELETE=13,keystone:INSERT=3 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:47:53.571636 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-743f7c79-1418-420d-a287-d1d7fb41e829 admin admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:47:53.589028 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c842dd-cd8f-4500-90bb-64a4fdd0a376 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:47:53.592712 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c842dd-cd8f-4500-90bb-64a4fdd0a376 admin admin] REQUEST_METHOD: `POST` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:47:53.593577 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c842dd-cd8f-4500-90bb-64a4fdd0a376 admin admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:47:53.595604 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c842dd-cd8f-4500-90bb-64a4fdd0a376 admin admin] PATH_INFO: `/v3/users` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:47:53.597809 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50c842dd-cd8f-4500-90bb-64a4fdd0a376 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:47:53.603259 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50c842dd-cd8f-4500-90bb-64a4fdd0a376 admin admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:47:53.648820 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 827/1658] 10.222.0.13 () {60 vars in 1144 bytes} [Wed Aug 31 09:47:53 2022] POST /identity/v3/users => generated 302 bytes in 300 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 09:47:53.649556 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 832/1659] 10.222.0.13 () {58 vars in 958 bytes} [Wed Aug 31 09:47:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 215 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:47:53.650984 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c842dd-cd8f-4500-90bb-64a4fdd0a376 admin admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:47:53.655711 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc49f4b1-9a9a-4f81-9b87-1f6827db8c4c None None] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:47:53.659572 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.fernet_utils [None req-50c842dd-cd8f-4500-90bb-64a4fdd0a376 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:47:53.662025 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.fernet_utils [None req-fc49f4b1-9a9a-4f81-9b87-1f6827db8c4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88114) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:47:53.695754 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c842dd-cd8f-4500-90bb-64a4fdd0a376 admin admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:47:53.696696 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc49f4b1-9a9a-4f81-9b87-1f6827db8c4c None None] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:47:53.709101 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e928aa0b-9ad8-463b-823e-8fbae86d9bb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:47:53.710096 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e928aa0b-9ad8-463b-823e-8fbae86d9bb2 admin admin] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:47:53.711219 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e928aa0b-9ad8-463b-823e-8fbae86d9bb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:47:53.711745 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e928aa0b-9ad8-463b-823e-8fbae86d9bb2 admin admin] PATH_INFO: `/v3/roles` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:47:53.712673 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e928aa0b-9ad8-463b-823e-8fbae86d9bb2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:47:53.714546 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e928aa0b-9ad8-463b-823e-8fbae86d9bb2 admin admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:47:53.718350 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8380daaf-7caa-492c-aab3-0323bf03db27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:47:53.719041 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8380daaf-7caa-492c-aab3-0323bf03db27 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:47:53.719316 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8380daaf-7caa-492c-aab3-0323bf03db27 admin admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:47:53.719585 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8380daaf-7caa-492c-aab3-0323bf03db27 admin admin] PATH_INFO: `/v3/users/9ad4450133404a7e921fbce29fa5e809` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:47:53.727311 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8380daaf-7caa-492c-aab3-0323bf03db27 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9ad4450133404a7e921fbce29fa5e809)` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:47:53.729285 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8380daaf-7caa-492c-aab3-0323bf03db27 admin admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:47:53.730890 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 833/1660] 10.222.0.13 () {58 vars in 1122 bytes} [Wed Aug 31 09:47:53 2022] GET /identity/v3/roles => generated 1420 bytes in 76 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 09:47:53.735380 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e928aa0b-9ad8-463b-823e-8fbae86d9bb2 admin admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:47:53.747996 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e928aa0b-9ad8-463b-823e-8fbae86d9bb2 admin admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:47:53.757887 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff41872b-5342-44b6-a3be-3546c7ca9f31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:47:53.758983 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff41872b-5342-44b6-a3be-3546c7ca9f31 admin admin] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:47:53.759548 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff41872b-5342-44b6-a3be-3546c7ca9f31 admin admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:47:53.760026 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff41872b-5342-44b6-a3be-3546c7ca9f31 admin admin] PATH_INFO: `/v3/roles` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:47:53.761066 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff41872b-5342-44b6-a3be-3546c7ca9f31 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:47:53.763842 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff41872b-5342-44b6-a3be-3546c7ca9f31 admin admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:47:53.777696 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 834/1661] 10.222.0.13 () {58 vars in 1122 bytes} [Wed Aug 31 09:47:53 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 09:47:53.781737 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff41872b-5342-44b6-a3be-3546c7ca9f31 admin admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:47:53.796404 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff41872b-5342-44b6-a3be-3546c7ca9f31 admin admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:47:53.822031 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-518bf8a0-71a8-413d-b44d-23e7c81a64f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:47:53.823307 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518bf8a0-71a8-413d-b44d-23e7c81a64f9 admin admin] REQUEST_METHOD: `PUT` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:47:53.823778 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518bf8a0-71a8-413d-b44d-23e7c81a64f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:47:53.824042 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518bf8a0-71a8-413d-b44d-23e7c81a64f9 admin admin] PATH_INFO: `/v3/projects/7274499d018e4e5ca98199f5446a19e0/users/20f156894e214714bf67b37ff94cd538/roles/6c99f5d9f1f243eaa134d8540f21095d` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:47:53.860387 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.notifications [None req-8380daaf-7caa-492c-aab3-0323bf03db27 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9ad4450133404a7e921fbce29fa5e809'} {{(pid=88113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:47:53.865833 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=79,keystone:DELETE=17,keystone:INSERT=4 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:47:53.872713 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-518bf8a0-71a8-413d-b44d-23e7c81a64f9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7274499d018e4e5ca98199f5446a19e0, user_id=20f156894e214714bf67b37ff94cd538, role_id=6c99f5d9f1f243eaa134d8540f21095d)` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:47:53.882616 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-518bf8a0-71a8-413d-b44d-23e7c81a64f9 admin admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:47:53.882807 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.notifications [None req-8380daaf-7caa-492c-aab3-0323bf03db27 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9ad4450133404a7e921fbce29fa5e809'} {{(pid=88113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:47:53.909892 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.notifications [None req-8380daaf-7caa-492c-aab3-0323bf03db27 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9ad4450133404a7e921fbce29fa5e809'} {{(pid=88113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:47:53.910768 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.notifications [None req-8380daaf-7caa-492c-aab3-0323bf03db27 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9ad4450133404a7e921fbce29fa5e809'} {{(pid=88113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:47:53.921211 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 835/1662] 10.222.0.13 () {60 vars in 1483 bytes} [Wed Aug 31 09:47:53 2022] PUT /identity/v3/projects/7274499d018e4e5ca98199f5446a19e0/users/20f156894e214714bf67b37ff94cd538/roles/6c99f5d9f1f243eaa134d8540f21095d => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:47:53.922658 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-518bf8a0-71a8-413d-b44d-23e7c81a64f9 admin admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:47:53.925534 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.fernet_utils [None req-518bf8a0-71a8-413d-b44d-23e7c81a64f9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88114) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:47:53.927971 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.notifications [None req-8380daaf-7caa-492c-aab3-0323bf03db27 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9ad4450133404a7e921fbce29fa5e809'} {{(pid=88113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:47:53.946079 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 828/1663] 10.222.0.13 () {58 vars in 1224 bytes} [Wed Aug 31 09:47:53 2022] DELETE /identity/v3/users/9ad4450133404a7e921fbce29fa5e809 => generated 0 bytes in 292 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:47:53.947667 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662bf451-a328-44ba-baf4-23d22ff02eda None None] REQUEST_METHOD: `POST` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:47:53.948023 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662bf451-a328-44ba-baf4-23d22ff02eda None None] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:47:53.948371 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662bf451-a328-44ba-baf4-23d22ff02eda None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:47:53.992481 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.auth.core [None req-662bf451-a328-44ba-baf4-23d22ff02eda None None] MFA Rules not processed for user `fd836cf590a145df9acf19cf3ae2ac56`. Rule list: `[]` (Enabled: `True`). {{(pid=88113) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:47:53.995035 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.fernet_utils [None req-662bf451-a328-44ba-baf4-23d22ff02eda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:47:54.086277 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-518bf8a0-71a8-413d-b44d-23e7c81a64f9 admin admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:47:54.141675 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 829/1664] 10.222.0.13 () {58 vars in 958 bytes} [Wed Aug 31 09:47:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:47:54.232819 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8046ba9c-4c0f-407f-9ed7-7184c50044c3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9bdaacd597442c2bcde2c555f2cc140', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '49548eb44d62415582af820ab66decee', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:47:54.235385 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8046ba9c-4c0f-407f-9ed7-7184c50044c3 service neutron] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:47:54.235679 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8046ba9c-4c0f-407f-9ed7-7184c50044c3 service neutron] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:47:54.236001 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8046ba9c-4c0f-407f-9ed7-7184c50044c3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:47:54.242874 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.fernet_utils [None req-8046ba9c-4c0f-407f-9ed7-7184c50044c3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88114) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:47:54.273767 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-662bf451-a328-44ba-baf4-23d22ff02eda None None] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:47:54.289182 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-662bf451-a328-44ba-baf4-23d22ff02eda None None] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:47:54.294188 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-720655aa-1c4e-41c9-b0d5-c7877f710d46 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9bdaacd597442c2bcde2c555f2cc140', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '49548eb44d62415582af820ab66decee', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:47:54.294833 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720655aa-1c4e-41c9-b0d5-c7877f710d46 service neutron] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:47:54.295137 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720655aa-1c4e-41c9-b0d5-c7877f710d46 service neutron] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:47:54.295394 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720655aa-1c4e-41c9-b0d5-c7877f710d46 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:47:54.296690 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8046ba9c-4c0f-407f-9ed7-7184c50044c3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:47:54.310018 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8046ba9c-4c0f-407f-9ed7-7184c50044c3 service neutron] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:47:54.324506 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-720655aa-1c4e-41c9-b0d5-c7877f710d46 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:47:54.326235 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-720655aa-1c4e-41c9-b0d5-c7877f710d46 service neutron] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:47:54.339103 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 836/1665] 10.222.0.13 () {60 vars in 1365 bytes} [Wed Aug 31 09:47:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 415 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:47:54.361242 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 830/1666] 10.222.0.13 () {60 vars in 1365 bytes} [Wed Aug 31 09:47:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 09:47:55.471483 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8046ba9c-4c0f-407f-9ed7-7184c50044c3 service neutron] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:47:55.477124 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.fernet_utils [None req-8046ba9c-4c0f-407f-9ed7-7184c50044c3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88114) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:47:55.503300 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8046ba9c-4c0f-407f-9ed7-7184c50044c3 service neutron] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:47:55.513323 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac751a70-0511-4aa6-8069-c71d8ce77e40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:47:55.514245 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac751a70-0511-4aa6-8069-c71d8ce77e40 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:47:55.514526 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac751a70-0511-4aa6-8069-c71d8ce77e40 admin admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:47:55.514797 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac751a70-0511-4aa6-8069-c71d8ce77e40 admin admin] PATH_INFO: `/v3/projects/a2e59c137309422787c48463fcb9f7c6` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:47:55.516674 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac751a70-0511-4aa6-8069-c71d8ce77e40 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a2e59c137309422787c48463fcb9f7c6)` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:47:55.522248 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac751a70-0511-4aa6-8069-c71d8ce77e40 admin admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:47:55.604229 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:47:55.688036 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.notifications [None req-ac751a70-0511-4aa6-8069-c71d8ce77e40 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a2e59c137309422787c48463fcb9f7c6', 'request_id': 'req-ac751a70-0511-4aa6-8069-c71d8ce77e40'} {{(pid=88114) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:47:55.700187 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.notifications [None req-ac751a70-0511-4aa6-8069-c71d8ce77e40 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a2e59c137309422787c48463fcb9f7c6', 'request_id': 'req-ac751a70-0511-4aa6-8069-c71d8ce77e40'} {{(pid=88114) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:47:55.720468 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.notifications [None req-ac751a70-0511-4aa6-8069-c71d8ce77e40 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a2e59c137309422787c48463fcb9f7c6', 'request_id': 'req-ac751a70-0511-4aa6-8069-c71d8ce77e40'} {{(pid=88114) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:47:55.768295 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.notifications [None req-ac751a70-0511-4aa6-8069-c71d8ce77e40 admin admin] The token cache is being invalidate because project a2e59c137309422787c48463fcb9f7c6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88114) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 09:47:55.770268 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.notifications [None req-ac751a70-0511-4aa6-8069-c71d8ce77e40 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88114) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:47:55.776800 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.notifications [None req-ac751a70-0511-4aa6-8069-c71d8ce77e40 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88114) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:47:55.785848 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 837/1667] 10.222.0.13 () {58 vars in 1233 bytes} [Wed Aug 31 09:47:55 2022] DELETE /identity/v3/projects/a2e59c137309422787c48463fcb9f7c6 => generated 0 bytes in 315 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:47:55.791612 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b87aee-5981-4dd1-b716-95d38b0df5f7 None None] REQUEST_METHOD: `POST` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:47:55.792179 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b87aee-5981-4dd1-b716-95d38b0df5f7 None None] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:47:55.792179 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b87aee-5981-4dd1-b716-95d38b0df5f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:47:55.835402 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.auth.core [None req-38b87aee-5981-4dd1-b716-95d38b0df5f7 None None] MFA Rules not processed for user `fd836cf590a145df9acf19cf3ae2ac56`. Rule list: `[]` (Enabled: `True`). {{(pid=88113) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:47:55.836654 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.fernet_utils [None req-38b87aee-5981-4dd1-b716-95d38b0df5f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:47:55.958081 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 831/1668] 10.222.0.13 () {58 vars in 958 bytes} [Wed Aug 31 09:47:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:47:55.970193 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c4bdd3-3550-4e52-8ad8-994b679d00be None None] REQUEST_METHOD: `POST` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:47:55.970762 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c4bdd3-3550-4e52-8ad8-994b679d00be None None] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:47:55.971741 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c4bdd3-3550-4e52-8ad8-994b679d00be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:47:56.012939 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.auth.core [None req-12c4bdd3-3550-4e52-8ad8-994b679d00be None None] MFA Rules not processed for user `fd836cf590a145df9acf19cf3ae2ac56`. Rule list: `[]` (Enabled: `True`). {{(pid=88114) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:47:56.014465 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.fernet_utils [None req-12c4bdd3-3550-4e52-8ad8-994b679d00be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88114) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:47:56.028547 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 838/1669] 10.222.0.13 () {58 vars in 958 bytes} [Wed Aug 31 09:47:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 61 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:47:56.032779 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b87aee-5981-4dd1-b716-95d38b0df5f7 None None] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:47:56.056864 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b87aee-5981-4dd1-b716-95d38b0df5f7 None None] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:47:56.063117 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e704fd17-5be0-4268-80d2-91e18d5b9666 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:47:56.063990 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e704fd17-5be0-4268-80d2-91e18d5b9666 admin admin] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:47:56.066711 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e704fd17-5be0-4268-80d2-91e18d5b9666 admin admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:47:56.067167 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e704fd17-5be0-4268-80d2-91e18d5b9666 admin admin] PATH_INFO: `/v3/domains` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:47:56.068540 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e704fd17-5be0-4268-80d2-91e18d5b9666 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88113) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:47:56.069031 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e704fd17-5be0-4268-80d2-91e18d5b9666 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:47:56.071419 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e704fd17-5be0-4268-80d2-91e18d5b9666 admin admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:47:56.103266 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c4bdd3-3550-4e52-8ad8-994b679d00be None None] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:47:56.103769 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 832/1670] 10.222.0.13 () {58 vars in 1153 bytes} [Wed Aug 31 09:47:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:47:56.117088 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c4bdd3-3550-4e52-8ad8-994b679d00be None None] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:47:56.136026 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b2c0df9-f503-4356-8536-301cb62eff1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:47:56.136637 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2c0df9-f503-4356-8536-301cb62eff1e admin admin] REQUEST_METHOD: `POST` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:47:56.137017 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2c0df9-f503-4356-8536-301cb62eff1e admin admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:47:56.137475 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2c0df9-f503-4356-8536-301cb62eff1e admin admin] PATH_INFO: `/v3/projects` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:47:56.139172 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b2c0df9-f503-4356-8536-301cb62eff1e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:47:56.143812 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b2c0df9-f503-4356-8536-301cb62eff1e admin admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:47:56.205951 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 839/1671] 10.222.0.13 () {60 vars in 1153 bytes} [Wed Aug 31 09:47:56 2022] POST /identity/v3/projects => generated 381 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 09:47:56.211002 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e704fd17-5be0-4268-80d2-91e18d5b9666 admin admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:47:56.230997 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e704fd17-5be0-4268-80d2-91e18d5b9666 admin admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:47:56.370429 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=96,keystone:DELETE=3,keystone:INSERT=1 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:47:56.435085 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89490789-f97a-4405-8d6d-d5d273a41373 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:47:56.435769 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89490789-f97a-4405-8d6d-d5d273a41373 admin admin] REQUEST_METHOD: `POST` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:47:56.436419 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89490789-f97a-4405-8d6d-d5d273a41373 admin admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:47:56.436702 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89490789-f97a-4405-8d6d-d5d273a41373 admin admin] PATH_INFO: `/v3/users` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:47:56.437355 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89490789-f97a-4405-8d6d-d5d273a41373 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:47:56.439337 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89490789-f97a-4405-8d6d-d5d273a41373 admin admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:47:56.504117 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b2c0df9-f503-4356-8536-301cb62eff1e admin admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:47:56.508441 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 833/1672] 10.222.0.13 () {60 vars in 1144 bytes} [Wed Aug 31 09:47:56 2022] POST /identity/v3/users => generated 291 bytes in 298 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 09:47:56.538133 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b2c0df9-f503-4356-8536-301cb62eff1e admin admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:47:56.554192 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-898a949b-df9d-49ca-ac12-a9110ef1b9ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:47:56.555116 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898a949b-df9d-49ca-ac12-a9110ef1b9ba admin admin] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:47:56.555542 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898a949b-df9d-49ca-ac12-a9110ef1b9ba admin admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:47:56.557097 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898a949b-df9d-49ca-ac12-a9110ef1b9ba admin admin] PATH_INFO: `/v3/roles` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:47:56.561037 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-898a949b-df9d-49ca-ac12-a9110ef1b9ba admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:47:56.565749 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-898a949b-df9d-49ca-ac12-a9110ef1b9ba admin admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:47:56.603158 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89490789-f97a-4405-8d6d-d5d273a41373 admin admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:47:56.617811 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 840/1673] 10.222.0.13 () {58 vars in 1122 bytes} [Wed Aug 31 09:47:56 2022] GET /identity/v3/roles => generated 1420 bytes in 113 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 09:47:56.619082 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-898a949b-df9d-49ca-ac12-a9110ef1b9ba admin admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:47:56.621107 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.fernet_utils [None req-898a949b-df9d-49ca-ac12-a9110ef1b9ba admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88114) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:47:56.633473 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89490789-f97a-4405-8d6d-d5d273a41373 admin admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:47:56.645365 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-898a949b-df9d-49ca-ac12-a9110ef1b9ba admin admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:47:56.651500 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1298dcf1-12c1-4a8c-b9aa-745e052b8449 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:47:56.652140 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1298dcf1-12c1-4a8c-b9aa-745e052b8449 admin admin] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:47:56.652424 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1298dcf1-12c1-4a8c-b9aa-745e052b8449 admin admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:47:56.652699 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1298dcf1-12c1-4a8c-b9aa-745e052b8449 admin admin] PATH_INFO: `/v3/roles` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:47:56.653535 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1298dcf1-12c1-4a8c-b9aa-745e052b8449 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:47:56.655231 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1298dcf1-12c1-4a8c-b9aa-745e052b8449 admin admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:47:56.668368 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f2b280-1b14-4385-8c0b-6de7f604299c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:47:56.669788 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f2b280-1b14-4385-8c0b-6de7f604299c admin admin] REQUEST_METHOD: `DELETE` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:47:56.670243 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f2b280-1b14-4385-8c0b-6de7f604299c admin admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:47:56.670690 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f2b280-1b14-4385-8c0b-6de7f604299c admin admin] PATH_INFO: `/v3/projects/05357b8aa9c6411e849aab8e271b1ca1` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:47:56.672279 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5f2b280-1b14-4385-8c0b-6de7f604299c admin admin] RBAC: Authorizing `identity:delete_project(project_id=05357b8aa9c6411e849aab8e271b1ca1)` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:47:56.675174 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5f2b280-1b14-4385-8c0b-6de7f604299c admin admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:47:56.690786 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 834/1674] 10.222.0.13 () {58 vars in 1122 bytes} [Wed Aug 31 09:47:56 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 09:47:56.694642 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1298dcf1-12c1-4a8c-b9aa-745e052b8449 admin admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:47:56.726106 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1298dcf1-12c1-4a8c-b9aa-745e052b8449 admin admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:47:56.749412 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec9439b-44a9-498e-9338-d600a901dde0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:47:56.752582 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec9439b-44a9-498e-9338-d600a901dde0 admin admin] REQUEST_METHOD: `PUT` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:47:56.753108 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec9439b-44a9-498e-9338-d600a901dde0 admin admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:47:56.753391 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec9439b-44a9-498e-9338-d600a901dde0 admin admin] PATH_INFO: `/v3/projects/90ac571b08da4c508ec9cf3bec5addb5/users/ea77057efa394b6ea5c25b22651ebacb/roles/6c99f5d9f1f243eaa134d8540f21095d` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:47:56.766908 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.notifications [None req-c5f2b280-1b14-4385-8c0b-6de7f604299c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '05357b8aa9c6411e849aab8e271b1ca1', 'request_id': 'req-c5f2b280-1b14-4385-8c0b-6de7f604299c'} {{(pid=88114) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:47:56.781624 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.notifications [None req-c5f2b280-1b14-4385-8c0b-6de7f604299c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '05357b8aa9c6411e849aab8e271b1ca1', 'request_id': 'req-c5f2b280-1b14-4385-8c0b-6de7f604299c'} {{(pid=88114) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:47:56.792633 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.notifications [None req-c5f2b280-1b14-4385-8c0b-6de7f604299c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '05357b8aa9c6411e849aab8e271b1ca1', 'request_id': 'req-c5f2b280-1b14-4385-8c0b-6de7f604299c'} {{(pid=88114) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:47:56.812034 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ec9439b-44a9-498e-9338-d600a901dde0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=90ac571b08da4c508ec9cf3bec5addb5, user_id=ea77057efa394b6ea5c25b22651ebacb, role_id=6c99f5d9f1f243eaa134d8540f21095d)` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:47:56.815377 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ec9439b-44a9-498e-9338-d600a901dde0 admin admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:47:56.822290 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.notifications [None req-c5f2b280-1b14-4385-8c0b-6de7f604299c admin admin] The token cache is being invalidate because project 05357b8aa9c6411e849aab8e271b1ca1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88114) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 09:47:56.822590 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.notifications [None req-c5f2b280-1b14-4385-8c0b-6de7f604299c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88114) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:47:56.832183 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.notifications [None req-c5f2b280-1b14-4385-8c0b-6de7f604299c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88114) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:47:56.836779 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 841/1675] 10.222.0.13 () {58 vars in 1233 bytes} [Wed Aug 31 09:47:56 2022] DELETE /identity/v3/projects/05357b8aa9c6411e849aab8e271b1ca1 => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:47:56.848406 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18cd655a-fc0b-45f6-8711-bceed2ed2a1d None None] REQUEST_METHOD: `POST` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:47:56.848702 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18cd655a-fc0b-45f6-8711-bceed2ed2a1d None None] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:47:56.849551 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18cd655a-fc0b-45f6-8711-bceed2ed2a1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:47:56.851913 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 835/1676] 10.222.0.13 () {60 vars in 1483 bytes} [Wed Aug 31 09:47:56 2022] PUT /identity/v3/projects/90ac571b08da4c508ec9cf3bec5addb5/users/ea77057efa394b6ea5c25b22651ebacb/roles/6c99f5d9f1f243eaa134d8540f21095d => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:47:56.862867 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d29521-4752-4a5b-91f7-34e6b677ecf7 None None] REQUEST_METHOD: `POST` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:47:56.863465 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d29521-4752-4a5b-91f7-34e6b677ecf7 None None] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:47:56.866310 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d29521-4752-4a5b-91f7-34e6b677ecf7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:47:56.934527 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.auth.core [None req-18cd655a-fc0b-45f6-8711-bceed2ed2a1d None None] MFA Rules not processed for user `fd836cf590a145df9acf19cf3ae2ac56`. Rule list: `[]` (Enabled: `True`). {{(pid=88114) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:47:56.936937 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.fernet_utils [None req-18cd655a-fc0b-45f6-8711-bceed2ed2a1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88114) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:47:56.956279 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.auth.core [None req-96d29521-4752-4a5b-91f7-34e6b677ecf7 None None] MFA Rules not processed for user `ea77057efa394b6ea5c25b22651ebacb`. Rule list: `[]` (Enabled: `True`). {{(pid=88113) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:47:56.963465 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.fernet_utils [None req-96d29521-4752-4a5b-91f7-34e6b677ecf7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:47:57.095617 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:DELETE=14,keystone:SELECT=83,keystone:INSERT=3 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:47:57.149702 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 842/1677] 10.222.0.13 () {58 vars in 958 bytes} [Wed Aug 31 09:47:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 295 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:47:57.152210 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50552682-19fe-4b3f-be7d-f99b5f02a92f None None] REQUEST_METHOD: `POST` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:47:57.152569 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50552682-19fe-4b3f-be7d-f99b5f02a92f None None] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:47:57.153016 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50552682-19fe-4b3f-be7d-f99b5f02a92f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:47:57.205140 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 836/1678] 10.222.0.13 () {58 vars in 958 bytes} [Wed Aug 31 09:47:56 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 343 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:47:57.206342 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d29521-4752-4a5b-91f7-34e6b677ecf7 None None] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:47:57.211442 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.fernet_utils [None req-96d29521-4752-4a5b-91f7-34e6b677ecf7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:47:57.219987 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.auth.core [None req-50552682-19fe-4b3f-be7d-f99b5f02a92f None None] MFA Rules not processed for user `fd836cf590a145df9acf19cf3ae2ac56`. Rule list: `[]` (Enabled: `True`). {{(pid=88114) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:47:57.227910 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.fernet_utils [None req-50552682-19fe-4b3f-be7d-f99b5f02a92f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88114) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:47:57.245619 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d29521-4752-4a5b-91f7-34e6b677ecf7 None None] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:47:57.258756 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 843/1679] 10.222.0.13 () {58 vars in 958 bytes} [Wed Aug 31 09:47:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 09:47:57.261080 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50552682-19fe-4b3f-be7d-f99b5f02a92f None None] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:47:57.269316 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99af08e5-d630-4bd7-9d58-935a15370a2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:47:57.274227 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99af08e5-d630-4bd7-9d58-935a15370a2e admin admin] REQUEST_METHOD: `POST` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:47:57.275072 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99af08e5-d630-4bd7-9d58-935a15370a2e admin admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:47:57.276190 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99af08e5-d630-4bd7-9d58-935a15370a2e admin admin] PATH_INFO: `/v3/projects` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:47:57.278433 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99af08e5-d630-4bd7-9d58-935a15370a2e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:47:57.287302 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99af08e5-d630-4bd7-9d58-935a15370a2e admin admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:47:57.296689 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50552682-19fe-4b3f-be7d-f99b5f02a92f None None] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:47:57.313345 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d4f056-3e8c-4846-8079-28c323a07be1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:47:57.314156 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d4f056-3e8c-4846-8079-28c323a07be1 admin admin] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:47:57.319333 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d4f056-3e8c-4846-8079-28c323a07be1 admin admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:47:57.319898 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d4f056-3e8c-4846-8079-28c323a07be1 admin admin] PATH_INFO: `/v3/domains` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:47:57.321645 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82d4f056-3e8c-4846-8079-28c323a07be1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88114) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:47:57.325035 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82d4f056-3e8c-4846-8079-28c323a07be1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:47:57.327677 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82d4f056-3e8c-4846-8079-28c323a07be1 admin admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:47:57.362502 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 837/1680] 10.222.0.13 () {60 vars in 1153 bytes} [Wed Aug 31 09:47:57 2022] POST /identity/v3/projects => generated 381 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 09:47:57.366984 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99af08e5-d630-4bd7-9d58-935a15370a2e admin admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:47:57.381929 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 844/1681] 10.222.0.13 () {58 vars in 1153 bytes} [Wed Aug 31 09:47:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 122 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:47:57.383038 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d4f056-3e8c-4846-8079-28c323a07be1 admin admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:47:57.411228 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d4f056-3e8c-4846-8079-28c323a07be1 admin admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:47:57.515101 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99af08e5-d630-4bd7-9d58-935a15370a2e admin admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:47:57.530713 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7eacdae-29b0-4f67-8a70-0cc18c5e5073 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:47:57.532240 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7eacdae-29b0-4f67-8a70-0cc18c5e5073 admin admin] REQUEST_METHOD: `POST` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:47:57.532240 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7eacdae-29b0-4f67-8a70-0cc18c5e5073 admin admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:47:57.532707 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7eacdae-29b0-4f67-8a70-0cc18c5e5073 admin admin] PATH_INFO: `/v3/users` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:47:57.533663 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7eacdae-29b0-4f67-8a70-0cc18c5e5073 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:47:57.535760 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7eacdae-29b0-4f67-8a70-0cc18c5e5073 admin admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:47:57.561056 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:47:57.586649 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 838/1682] 10.222.0.13 () {60 vars in 1144 bytes} [Wed Aug 31 09:47:57 2022] POST /identity/v3/users => generated 297 bytes in 220 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 09:47:57.591808 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7eacdae-29b0-4f67-8a70-0cc18c5e5073 admin admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:47:57.594412 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a5e4e32-d7cc-48d1-b9dc-8c4da8352d4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:47:57.595575 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5e4e32-d7cc-48d1-b9dc-8c4da8352d4f admin admin] REQUEST_METHOD: `POST` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:47:57.597931 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5e4e32-d7cc-48d1-b9dc-8c4da8352d4f admin admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:47:57.598481 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5e4e32-d7cc-48d1-b9dc-8c4da8352d4f admin admin] PATH_INFO: `/v3/projects` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:47:57.602732 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a5e4e32-d7cc-48d1-b9dc-8c4da8352d4f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:47:57.605097 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a5e4e32-d7cc-48d1-b9dc-8c4da8352d4f admin admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:47:57.611926 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7eacdae-29b0-4f67-8a70-0cc18c5e5073 admin admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:47:57.630871 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b6cf0e-ba9e-4957-b797-7628ded68a46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:47:57.633639 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b6cf0e-ba9e-4957-b797-7628ded68a46 admin admin] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:47:57.634326 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b6cf0e-ba9e-4957-b797-7628ded68a46 admin admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:47:57.634774 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b6cf0e-ba9e-4957-b797-7628ded68a46 admin admin] PATH_INFO: `/v3/roles` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:47:57.636841 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5b6cf0e-ba9e-4957-b797-7628ded68a46 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:47:57.641233 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5b6cf0e-ba9e-4957-b797-7628ded68a46 admin admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:47:57.675452 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 839/1683] 10.222.0.13 () {58 vars in 1122 bytes} [Wed Aug 31 09:47:57 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 09:47:57.676968 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b6cf0e-ba9e-4957-b797-7628ded68a46 admin admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:47:57.689453 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 845/1684] 10.222.0.13 () {60 vars in 1153 bytes} [Wed Aug 31 09:47:57 2022] POST /identity/v3/projects => generated 403 bytes in 307 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 09:47:57.695496 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a5e4e32-d7cc-48d1-b9dc-8c4da8352d4f admin admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:47:57.704742 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b6cf0e-ba9e-4957-b797-7628ded68a46 admin admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:47:57.842618 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a5e4e32-d7cc-48d1-b9dc-8c4da8352d4f admin admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:47:57.852178 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09816242-75d4-434c-9a00-941fc067eff5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:47:57.853062 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09816242-75d4-434c-9a00-941fc067eff5 admin admin] REQUEST_METHOD: `POST` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:47:57.854415 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09816242-75d4-434c-9a00-941fc067eff5 admin admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:47:57.854911 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09816242-75d4-434c-9a00-941fc067eff5 admin admin] PATH_INFO: `/v3/users` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:47:57.855829 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09816242-75d4-434c-9a00-941fc067eff5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:47:57.858588 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09816242-75d4-434c-9a00-941fc067eff5 admin admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:47:57.883820 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44624323-ac49-4fe4-87df-37617e82a305 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:47:57.884676 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44624323-ac49-4fe4-87df-37617e82a305 admin admin] REQUEST_METHOD: `PUT` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:47:57.885244 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44624323-ac49-4fe4-87df-37617e82a305 admin admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:47:57.885873 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44624323-ac49-4fe4-87df-37617e82a305 admin admin] PATH_INFO: `/v3/projects/0265d8f847a748519b8b74c308245784/users/59c84ddd914043f6aeaf185c51fd1f9d/roles/5d4ee63a3add47fc9a6b8b16d39e0a1f` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:47:57.901351 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 846/1685] 10.222.0.13 () {60 vars in 1144 bytes} [Wed Aug 31 09:47:57 2022] POST /identity/v3/users => generated 302 bytes in 206 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 09:47:57.905413 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09816242-75d4-434c-9a00-941fc067eff5 admin admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:47:57.911918 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44624323-ac49-4fe4-87df-37617e82a305 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0265d8f847a748519b8b74c308245784, user_id=59c84ddd914043f6aeaf185c51fd1f9d, role_id=5d4ee63a3add47fc9a6b8b16d39e0a1f)` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:47:57.917109 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44624323-ac49-4fe4-87df-37617e82a305 admin admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:47:57.936713 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09816242-75d4-434c-9a00-941fc067eff5 admin admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:47:57.949044 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 840/1686] 10.222.0.13 () {60 vars in 1483 bytes} [Wed Aug 31 09:47:57 2022] PUT /identity/v3/projects/0265d8f847a748519b8b74c308245784/users/59c84ddd914043f6aeaf185c51fd1f9d/roles/5d4ee63a3add47fc9a6b8b16d39e0a1f => generated 0 bytes in 273 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:47:57.953714 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44624323-ac49-4fe4-87df-37617e82a305 admin admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:47:58.049443 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:47:58.086083 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44624323-ac49-4fe4-87df-37617e82a305 admin admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:47:58.089931 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68390848-df2c-4cff-b5d4-3e6eefa65284 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:47:58.090548 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68390848-df2c-4cff-b5d4-3e6eefa65284 admin admin] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:47:58.090820 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68390848-df2c-4cff-b5d4-3e6eefa65284 admin admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:47:58.091092 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68390848-df2c-4cff-b5d4-3e6eefa65284 admin admin] PATH_INFO: `/v3/roles` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:47:58.096211 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68390848-df2c-4cff-b5d4-3e6eefa65284 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:47:58.097973 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68390848-df2c-4cff-b5d4-3e6eefa65284 admin admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:47:58.100649 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a877ac9a-b147-4895-a2b3-c3eac69e3447 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:47:58.101297 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a877ac9a-b147-4895-a2b3-c3eac69e3447 admin admin] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:47:58.101854 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a877ac9a-b147-4895-a2b3-c3eac69e3447 admin admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:47:58.102155 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a877ac9a-b147-4895-a2b3-c3eac69e3447 admin admin] PATH_INFO: `/v3/roles` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:47:58.102826 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a877ac9a-b147-4895-a2b3-c3eac69e3447 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:47:58.104465 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a877ac9a-b147-4895-a2b3-c3eac69e3447 admin admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:47:58.132032 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 841/1687] 10.222.0.13 () {58 vars in 1122 bytes} [Wed Aug 31 09:47:57 2022] GET /identity/v3/roles => generated 1420 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 09:47:58.144394 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a877ac9a-b147-4895-a2b3-c3eac69e3447 admin admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:47:58.147745 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a877ac9a-b147-4895-a2b3-c3eac69e3447 admin admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:47:58.150527 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 847/1688] 10.222.0.13 () {58 vars in 1122 bytes} [Wed Aug 31 09:47:57 2022] GET /identity/v3/roles => generated 1420 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 09:47:58.151923 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2389f974-752c-4539-b17c-73af5c914731 None None] REQUEST_METHOD: `POST` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:47:58.154363 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2389f974-752c-4539-b17c-73af5c914731 None None] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:47:58.154363 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2389f974-752c-4539-b17c-73af5c914731 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:47:58.158771 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c7c711-1fba-44ef-9978-0bc671b75d9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:47:58.160096 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c7c711-1fba-44ef-9978-0bc671b75d9e admin admin] REQUEST_METHOD: `PUT` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:47:58.160212 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c7c711-1fba-44ef-9978-0bc671b75d9e admin admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:47:58.160597 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c7c711-1fba-44ef-9978-0bc671b75d9e admin admin] PATH_INFO: `/v3/projects/0265d8f847a748519b8b74c308245784/users/59c84ddd914043f6aeaf185c51fd1f9d/roles/5d4ee63a3add47fc9a6b8b16d39e0a1f` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:47:58.164105 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9c7c711-1fba-44ef-9978-0bc671b75d9e admin admin] RBAC: Authorizing `identity:create_grant(project_id=0265d8f847a748519b8b74c308245784, user_id=59c84ddd914043f6aeaf185c51fd1f9d, role_id=5d4ee63a3add47fc9a6b8b16d39e0a1f)` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:47:58.171196 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9c7c711-1fba-44ef-9978-0bc671b75d9e admin admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:47:58.196533 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 842/1689] 10.222.0.13 () {60 vars in 1483 bytes} [Wed Aug 31 09:47:58 2022] PUT /identity/v3/projects/0265d8f847a748519b8b74c308245784/users/59c84ddd914043f6aeaf185c51fd1f9d/roles/5d4ee63a3add47fc9a6b8b16d39e0a1f => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:47:58.197900 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68b5c0e-f932-45f9-9209-28d686db749a None None] REQUEST_METHOD: `POST` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:47:58.198119 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68b5c0e-f932-45f9-9209-28d686db749a None None] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:47:58.198827 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68b5c0e-f932-45f9-9209-28d686db749a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:47:58.227328 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.auth.core [None req-2389f974-752c-4539-b17c-73af5c914731 None None] MFA Rules not processed for user `886e40cf192d46c2b2e7eb2a95ec62e1`. Rule list: `[]` (Enabled: `True`). {{(pid=88114) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:47:58.229913 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.fernet_utils [None req-2389f974-752c-4539-b17c-73af5c914731 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88114) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:47:58.240951 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.auth.core [None req-a68b5c0e-f932-45f9-9209-28d686db749a None None] MFA Rules not processed for user `20f156894e214714bf67b37ff94cd538`. Rule list: `[]` (Enabled: `True`). {{(pid=88113) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:47:58.242611 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.fernet_utils [None req-a68b5c0e-f932-45f9-9209-28d686db749a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:47:58.302848 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:47:58.339243 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 848/1690] 10.222.0.13 () {58 vars in 958 bytes} [Wed Aug 31 09:47:58 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:47:58.339243 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2389f974-752c-4539-b17c-73af5c914731 None None] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:47:58.350461 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 843/1691] 10.222.0.13 () {58 vars in 958 bytes} [Wed Aug 31 09:47:58 2022] POST /identity/v3/auth/tokens => generated 3280 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:47:58.351813 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d051d84-576e-4173-9b72-41fb5c3e35ec None None] REQUEST_METHOD: `POST` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:47:58.352102 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d051d84-576e-4173-9b72-41fb5c3e35ec None None] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:47:58.352368 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d051d84-576e-4173-9b72-41fb5c3e35ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:47:58.421066 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.auth.core [None req-6d051d84-576e-4173-9b72-41fb5c3e35ec None None] MFA Rules not processed for user `59c84ddd914043f6aeaf185c51fd1f9d`. Rule list: `[]` (Enabled: `True`). {{(pid=88113) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:47:58.422646 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2389f974-752c-4539-b17c-73af5c914731 None None] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:47:58.422890 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.fernet_utils [None req-6d051d84-576e-4173-9b72-41fb5c3e35ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:47:58.429783 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d49d57f-0603-4526-912a-17f4dc12e892 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:47:58.430452 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d49d57f-0603-4526-912a-17f4dc12e892 admin admin] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:47:58.436015 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d49d57f-0603-4526-912a-17f4dc12e892 admin admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:47:58.436317 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d49d57f-0603-4526-912a-17f4dc12e892 admin admin] PATH_INFO: `/v3/roles` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:47:58.437079 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d49d57f-0603-4526-912a-17f4dc12e892 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:47:58.438791 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d49d57f-0603-4526-912a-17f4dc12e892 admin admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:47:58.477567 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 849/1692] 10.222.0.13 () {58 vars in 1122 bytes} [Wed Aug 31 09:47:58 2022] GET /identity/v3/roles => generated 1420 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 09:47:58.479585 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e916756-dd0e-4917-97ec-4611b9537bae None None] REQUEST_METHOD: `POST` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:47:58.479873 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e916756-dd0e-4917-97ec-4611b9537bae None None] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:47:58.480151 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e916756-dd0e-4917-97ec-4611b9537bae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:47:58.520117 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.auth.core [None req-4e916756-dd0e-4917-97ec-4611b9537bae None None] MFA Rules not processed for user `886e40cf192d46c2b2e7eb2a95ec62e1`. Rule list: `[]` (Enabled: `True`). {{(pid=88114) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:47:58.522182 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.fernet_utils [None req-4e916756-dd0e-4917-97ec-4611b9537bae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88114) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:47:58.547100 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 850/1693] 10.222.0.13 () {58 vars in 958 bytes} [Wed Aug 31 09:47:58 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 69 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:47:58.551323 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e916756-dd0e-4917-97ec-4611b9537bae None None] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:47:58.557574 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.fernet_utils [None req-4e916756-dd0e-4917-97ec-4611b9537bae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88114) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:47:58.592184 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e916756-dd0e-4917-97ec-4611b9537bae None None] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:47:58.608703 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b98c0476-4d63-42e8-bdaa-7bddd78bf9a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:47:58.610059 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98c0476-4d63-42e8-bdaa-7bddd78bf9a3 admin admin] REQUEST_METHOD: `POST` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:47:58.610511 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98c0476-4d63-42e8-bdaa-7bddd78bf9a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:47:58.611008 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98c0476-4d63-42e8-bdaa-7bddd78bf9a3 admin admin] PATH_INFO: `/v3/projects` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:47:58.612104 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b98c0476-4d63-42e8-bdaa-7bddd78bf9a3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:47:58.612811 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 844/1694] 10.222.0.13 () {58 vars in 958 bytes} [Wed Aug 31 09:47:58 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 262 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:47:58.613933 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d051d84-576e-4173-9b72-41fb5c3e35ec None None] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:47:58.615597 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b98c0476-4d63-42e8-bdaa-7bddd78bf9a3 admin admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:47:58.652194 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d051d84-576e-4173-9b72-41fb5c3e35ec None None] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:47:58.664523 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a06e05ba-3990-4c0c-a506-cc05c5e20d89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:47:58.665563 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06e05ba-3990-4c0c-a506-cc05c5e20d89 admin admin] REQUEST_METHOD: `PUT` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:47:58.665934 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06e05ba-3990-4c0c-a506-cc05c5e20d89 admin admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:47:58.666214 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06e05ba-3990-4c0c-a506-cc05c5e20d89 admin admin] PATH_INFO: `/v3/projects/2cbde0bbab85470d870244ff1cadffd6/users/f1859851143c43539a34fb9a2c2b68c3/roles/6c99f5d9f1f243eaa134d8540f21095d` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:47:58.696848 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 851/1695] 10.222.0.13 () {60 vars in 1153 bytes} [Wed Aug 31 09:47:58 2022] POST /identity/v3/projects => generated 403 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 09:47:58.697877 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b98c0476-4d63-42e8-bdaa-7bddd78bf9a3 admin admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:47:58.704387 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.fernet_utils [None req-b98c0476-4d63-42e8-bdaa-7bddd78bf9a3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88114) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:47:58.717792 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a06e05ba-3990-4c0c-a506-cc05c5e20d89 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2cbde0bbab85470d870244ff1cadffd6, user_id=f1859851143c43539a34fb9a2c2b68c3, role_id=6c99f5d9f1f243eaa134d8540f21095d)` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:47:58.723421 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a06e05ba-3990-4c0c-a506-cc05c5e20d89 admin admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:47:58.769725 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 845/1696] 10.222.0.13 () {60 vars in 1483 bytes} [Wed Aug 31 09:47:58 2022] PUT /identity/v3/projects/2cbde0bbab85470d870244ff1cadffd6/users/f1859851143c43539a34fb9a2c2b68c3/roles/6c99f5d9f1f243eaa134d8540f21095d => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:47:58.771422 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a06e05ba-3990-4c0c-a506-cc05c5e20d89 admin admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:47:58.777036 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.fernet_utils [None req-a06e05ba-3990-4c0c-a506-cc05c5e20d89 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:47:58.828342 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:47:58.882656 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b98c0476-4d63-42e8-bdaa-7bddd78bf9a3 admin admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:47:59.008992 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a06e05ba-3990-4c0c-a506-cc05c5e20d89 admin admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:47:59.043040 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c9aaace-aaba-4b52-bc67-bf941228e788 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d52b69c8e94441dacfcff944326f169', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '49548eb44d62415582af820ab66decee', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:47:59.043996 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9aaace-aaba-4b52-bc67-bf941228e788 service cinder] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:47:59.044501 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9aaace-aaba-4b52-bc67-bf941228e788 service cinder] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:47:59.044888 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9aaace-aaba-4b52-bc67-bf941228e788 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:47:59.089124 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-281b5910-b895-44f6-bebc-fd6bd3ac09fe service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc076493239d4ccdbdc37c0369c49529', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '49548eb44d62415582af820ab66decee', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:47:59.089708 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281b5910-b895-44f6-bebc-fd6bd3ac09fe service nova] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:47:59.089987 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281b5910-b895-44f6-bebc-fd6bd3ac09fe service nova] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:47:59.090245 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281b5910-b895-44f6-bebc-fd6bd3ac09fe service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:47:59.151237 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:47:59.176670 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c9aaace-aaba-4b52-bc67-bf941228e788 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:47:59.180013 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c9aaace-aaba-4b52-bc67-bf941228e788 service cinder] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:47:59.214372 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 846/1697] 10.222.0.13 () {60 vars in 1365 bytes} [Wed Aug 31 09:47:58 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 444 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:47:59.215826 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c9aaace-aaba-4b52-bc67-bf941228e788 service cinder] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:47:59.221728 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-281b5910-b895-44f6-bebc-fd6bd3ac09fe service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:47:59.223623 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-281b5910-b895-44f6-bebc-fd6bd3ac09fe service nova] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:47:59.264824 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 852/1698] 10.222.0.13 () {60 vars in 1365 bytes} [Wed Aug 31 09:47:58 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 567 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:47:59.267435 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837cd565-89f5-457e-b31c-cee1700cb30e None None] REQUEST_METHOD: `POST` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:47:59.271217 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837cd565-89f5-457e-b31c-cee1700cb30e None None] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:47:59.271730 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837cd565-89f5-457e-b31c-cee1700cb30e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:47:59.370739 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.auth.core [None req-837cd565-89f5-457e-b31c-cee1700cb30e None None] MFA Rules not processed for user `f1859851143c43539a34fb9a2c2b68c3`. Rule list: `[]` (Enabled: `True`). {{(pid=88114) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:47:59.374534 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.fernet_utils [None req-837cd565-89f5-457e-b31c-cee1700cb30e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88114) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:47:59.434679 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c9aaace-aaba-4b52-bc67-bf941228e788 service cinder] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:47:59.450119 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58fa6e0e-b608-45e9-82bc-511979965fc1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:47:59.450859 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58fa6e0e-b608-45e9-82bc-511979965fc1 admin admin] REQUEST_METHOD: `POST` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:47:59.451053 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58fa6e0e-b608-45e9-82bc-511979965fc1 admin admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:47:59.451356 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58fa6e0e-b608-45e9-82bc-511979965fc1 admin admin] PATH_INFO: `/v3/users` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:47:59.452699 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58fa6e0e-b608-45e9-82bc-511979965fc1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:47:59.454718 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58fa6e0e-b608-45e9-82bc-511979965fc1 admin admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:47:59.525304 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 853/1699] 10.222.0.13 () {58 vars in 958 bytes} [Wed Aug 31 09:47:59 2022] POST /identity/v3/auth/tokens => generated 3280 bytes in 259 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:47:59.529691 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 847/1700] 10.222.0.13 () {60 vars in 1144 bytes} [Wed Aug 31 09:47:59 2022] POST /identity/v3/users => generated 308 bytes in 315 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 09:47:59.529883 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-837cd565-89f5-457e-b31c-cee1700cb30e None None] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:47:59.534115 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58fa6e0e-b608-45e9-82bc-511979965fc1 admin admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:47:59.557318 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58fa6e0e-b608-45e9-82bc-511979965fc1 admin admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:47:59.568132 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-837cd565-89f5-457e-b31c-cee1700cb30e None None] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:47:59.581835 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0707ffab-2b32-4aa5-bd8f-8e92f3930cf9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:47:59.583990 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0707ffab-2b32-4aa5-bd8f-8e92f3930cf9 admin admin] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:47:59.584817 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0707ffab-2b32-4aa5-bd8f-8e92f3930cf9 admin admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:47:59.585702 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0707ffab-2b32-4aa5-bd8f-8e92f3930cf9 admin admin] PATH_INFO: `/v3/roles` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:47:59.586751 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0707ffab-2b32-4aa5-bd8f-8e92f3930cf9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:47:59.587690 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad0862f7-5935-4bf3-ac9f-1980085e61e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:47:59.589565 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0707ffab-2b32-4aa5-bd8f-8e92f3930cf9 admin admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:47:59.590372 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0862f7-5935-4bf3-ac9f-1980085e61e1 admin admin] REQUEST_METHOD: `POST` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:47:59.596343 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0862f7-5935-4bf3-ac9f-1980085e61e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:47:59.596888 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0862f7-5935-4bf3-ac9f-1980085e61e1 admin admin] PATH_INFO: `/v3/projects` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:47:59.597571 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad0862f7-5935-4bf3-ac9f-1980085e61e1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:47:59.600071 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad0862f7-5935-4bf3-ac9f-1980085e61e1 admin admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:47:59.616880 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=100 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:47:59.635423 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 848/1701] 10.222.0.13 () {58 vars in 1122 bytes} [Wed Aug 31 09:47:59 2022] GET /identity/v3/roles => generated 1420 bytes in 102 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 09:47:59.639725 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0707ffab-2b32-4aa5-bd8f-8e92f3930cf9 admin admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:47:59.664615 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0707ffab-2b32-4aa5-bd8f-8e92f3930cf9 admin admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:47:59.680307 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3adf8b6-cf0d-4281-adfc-0dc7bce7a21b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:47:59.687260 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3adf8b6-cf0d-4281-adfc-0dc7bce7a21b admin admin] REQUEST_METHOD: `PUT` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:47:59.687664 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3adf8b6-cf0d-4281-adfc-0dc7bce7a21b admin admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:47:59.687941 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3adf8b6-cf0d-4281-adfc-0dc7bce7a21b admin admin] PATH_INFO: `/v3/projects/3df0a209d4c74b6390542f869999787a/users/e4f1a935c10540f58c25b26fa4403929/roles/5d4ee63a3add47fc9a6b8b16d39e0a1f` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:47:59.691234 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 854/1702] 10.222.0.13 () {60 vars in 1153 bytes} [Wed Aug 31 09:47:59 2022] POST /identity/v3/projects => generated 405 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 09:47:59.695574 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad0862f7-5935-4bf3-ac9f-1980085e61e1 admin admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:47:59.737881 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3adf8b6-cf0d-4281-adfc-0dc7bce7a21b admin admin] RBAC: Authorizing `identity:create_grant(project_id=3df0a209d4c74b6390542f869999787a, user_id=e4f1a935c10540f58c25b26fa4403929, role_id=5d4ee63a3add47fc9a6b8b16d39e0a1f)` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:47:59.741018 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3adf8b6-cf0d-4281-adfc-0dc7bce7a21b admin admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:47:59.797160 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 849/1703] 10.222.0.13 () {60 vars in 1483 bytes} [Wed Aug 31 09:47:59 2022] PUT /identity/v3/projects/3df0a209d4c74b6390542f869999787a/users/e4f1a935c10540f58c25b26fa4403929/roles/5d4ee63a3add47fc9a6b8b16d39e0a1f => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 09:47:59.800948 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3adf8b6-cf0d-4281-adfc-0dc7bce7a21b admin admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:47:59.958588 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad0862f7-5935-4bf3-ac9f-1980085e61e1 admin admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:48:00.039712 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3adf8b6-cf0d-4281-adfc-0dc7bce7a21b admin admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:48:00.056454 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde4717d-235a-42df-bf4f-7a2f6dcd955f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:48:00.057082 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde4717d-235a-42df-bf4f-7a2f6dcd955f admin admin] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:00.057357 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde4717d-235a-42df-bf4f-7a2f6dcd955f admin admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:00.057648 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde4717d-235a-42df-bf4f-7a2f6dcd955f admin admin] PATH_INFO: `/v3/roles` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:00.058299 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bde4717d-235a-42df-bf4f-7a2f6dcd955f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:48:00.064134 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bde4717d-235a-42df-bf4f-7a2f6dcd955f admin admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:48:00.088124 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 850/1704] 10.222.0.13 () {58 vars in 1122 bytes} [Wed Aug 31 09:47:59 2022] GET /identity/v3/roles => generated 1420 bytes in 288 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 09:48:00.094087 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde4717d-235a-42df-bf4f-7a2f6dcd955f admin admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:48:00.131683 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde4717d-235a-42df-bf4f-7a2f6dcd955f admin admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:48:00.132713 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e65034-e038-4f63-86ca-a9181c03c166 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:48:00.133343 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e65034-e038-4f63-86ca-a9181c03c166 admin admin] REQUEST_METHOD: `POST` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:00.133716 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e65034-e038-4f63-86ca-a9181c03c166 admin admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:00.133984 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e65034-e038-4f63-86ca-a9181c03c166 admin admin] PATH_INFO: `/v3/users` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:00.134625 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3e65034-e038-4f63-86ca-a9181c03c166 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:48:00.136571 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3e65034-e038-4f63-86ca-a9181c03c166 admin admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:48:00.156768 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce7997f5-ac87-45f4-8a34-de532fc666c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:48:00.157583 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7997f5-ac87-45f4-8a34-de532fc666c6 admin admin] REQUEST_METHOD: `PUT` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:00.157860 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7997f5-ac87-45f4-8a34-de532fc666c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:00.158129 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7997f5-ac87-45f4-8a34-de532fc666c6 admin admin] PATH_INFO: `/v3/projects/3df0a209d4c74b6390542f869999787a/users/e4f1a935c10540f58c25b26fa4403929/roles/5d4ee63a3add47fc9a6b8b16d39e0a1f` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:00.164619 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce7997f5-ac87-45f4-8a34-de532fc666c6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3df0a209d4c74b6390542f869999787a, user_id=e4f1a935c10540f58c25b26fa4403929, role_id=5d4ee63a3add47fc9a6b8b16d39e0a1f)` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:48:00.167420 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce7997f5-ac87-45f4-8a34-de532fc666c6 admin admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:48:00.221149 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 851/1706] 10.222.0.13 () {60 vars in 1483 bytes} [Wed Aug 31 09:48:00 2022] PUT /identity/v3/projects/3df0a209d4c74b6390542f869999787a/users/e4f1a935c10540f58c25b26fa4403929/roles/5d4ee63a3add47fc9a6b8b16d39e0a1f => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:48:00.222213 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce7997f5-ac87-45f4-8a34-de532fc666c6 admin admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:48:00.222623 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 855/1706] 10.222.0.13 () {60 vars in 1144 bytes} [Wed Aug 31 09:47:59 2022] POST /identity/v3/users => generated 309 bytes in 526 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 09:48:00.238151 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e65034-e038-4f63-86ca-a9181c03c166 admin admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:48:00.315669 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:48:00.421315 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce7997f5-ac87-45f4-8a34-de532fc666c6 admin admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:48:00.425529 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e65034-e038-4f63-86ca-a9181c03c166 admin admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:48:00.440835 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04caff40-a3a6-4777-8d9d-30e84189d374 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:48:00.441514 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04caff40-a3a6-4777-8d9d-30e84189d374 admin admin] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:00.444829 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04caff40-a3a6-4777-8d9d-30e84189d374 admin admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:00.445154 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04caff40-a3a6-4777-8d9d-30e84189d374 admin admin] PATH_INFO: `/v3/roles` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:00.445974 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04caff40-a3a6-4777-8d9d-30e84189d374 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:48:00.448274 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04caff40-a3a6-4777-8d9d-30e84189d374 admin admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:48:00.480189 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 852/1707] 10.222.0.13 () {58 vars in 1122 bytes} [Wed Aug 31 09:48:00 2022] GET /identity/v3/roles => generated 1420 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 09:48:00.484465 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04caff40-a3a6-4777-8d9d-30e84189d374 admin admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:48:00.490777 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:48:00.520711 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04caff40-a3a6-4777-8d9d-30e84189d374 admin admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:48:00.531084 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f404224c-8ab2-47d4-acf1-16417b16191e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:48:00.531940 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f404224c-8ab2-47d4-acf1-16417b16191e admin admin] REQUEST_METHOD: `PUT` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:00.532395 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f404224c-8ab2-47d4-acf1-16417b16191e admin admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:00.532676 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f404224c-8ab2-47d4-acf1-16417b16191e admin admin] PATH_INFO: `/v3/projects/0c7662a0f481410fa31c1ec1b888dd52/users/989378c95b6c440e9a3a6f2ed6b78478/roles/5d4ee63a3add47fc9a6b8b16d39e0a1f` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:00.567204 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9694e8f1-907c-4f15-9b75-82f3f5b3c559 tempest-VolumeQuotaClassesTest-1080223495 tempest-VolumeQuotaClassesTest-1080223495-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '59c84ddd914043f6aeaf185c51fd1f9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0265d8f847a748519b8b74c308245784', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:48:00.568890 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9694e8f1-907c-4f15-9b75-82f3f5b3c559 tempest-VolumeQuotaClassesTest-1080223495 tempest-VolumeQuotaClassesTest-1080223495-project-admin] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:00.569369 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9694e8f1-907c-4f15-9b75-82f3f5b3c559 tempest-VolumeQuotaClassesTest-1080223495 tempest-VolumeQuotaClassesTest-1080223495-project-admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:00.569850 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9694e8f1-907c-4f15-9b75-82f3f5b3c559 tempest-VolumeQuotaClassesTest-1080223495 tempest-VolumeQuotaClassesTest-1080223495-project-admin] PATH_INFO: `/v3/domains` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:00.571902 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9694e8f1-907c-4f15-9b75-82f3f5b3c559 tempest-VolumeQuotaClassesTest-1080223495 tempest-VolumeQuotaClassesTest-1080223495-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=88114) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:48:00.572422 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9694e8f1-907c-4f15-9b75-82f3f5b3c559 tempest-VolumeQuotaClassesTest-1080223495 tempest-VolumeQuotaClassesTest-1080223495-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:48:00.573625 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f404224c-8ab2-47d4-acf1-16417b16191e admin admin] RBAC: Authorizing `identity:create_grant(project_id=0c7662a0f481410fa31c1ec1b888dd52, user_id=989378c95b6c440e9a3a6f2ed6b78478, role_id=5d4ee63a3add47fc9a6b8b16d39e0a1f)` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:48:00.577704 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f404224c-8ab2-47d4-acf1-16417b16191e admin admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:48:00.582809 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9694e8f1-907c-4f15-9b75-82f3f5b3c559 tempest-VolumeQuotaClassesTest-1080223495 tempest-VolumeQuotaClassesTest-1080223495-project-admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:48:00.611393 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 853/1708] 10.222.0.13 () {60 vars in 1483 bytes} [Wed Aug 31 09:48:00 2022] PUT /identity/v3/projects/0c7662a0f481410fa31c1ec1b888dd52/users/989378c95b6c440e9a3a6f2ed6b78478/roles/5d4ee63a3add47fc9a6b8b16d39e0a1f => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:48:00.614004 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f171476-88cb-4c2c-884a-586c1ddbf38a None None] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:00.617750 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f171476-88cb-4c2c-884a-586c1ddbf38a None None] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:00.620438 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f171476-88cb-4c2c-884a-586c1ddbf38a None None] PATH_INFO: `/` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:00.628256 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 856/1709] 10.222.0.13 () {58 vars in 1153 bytes} [Wed Aug 31 09:48:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 390 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:48:00.632189 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9694e8f1-907c-4f15-9b75-82f3f5b3c559 tempest-VolumeQuotaClassesTest-1080223495 tempest-VolumeQuotaClassesTest-1080223495-project-admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:48:00.632622 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 854/1710] 10.222.0.13 () {54 vars in 912 bytes} [Wed Aug 31 09:48:00 2022] GET /identity => generated 270 bytes in 21 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:48:00.633709 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f171476-88cb-4c2c-884a-586c1ddbf38a None None] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:48:00.681163 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9694e8f1-907c-4f15-9b75-82f3f5b3c559 tempest-VolumeQuotaClassesTest-1080223495 tempest-VolumeQuotaClassesTest-1080223495-project-admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:48:00.765601 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f171476-88cb-4c2c-884a-586c1ddbf38a None None] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:48:00.781321 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92fcdb18-5f54-4000-b62b-bda8a7f26314 tempest-VolumeQuotaClassesTest-1080223495 tempest-VolumeQuotaClassesTest-1080223495-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '59c84ddd914043f6aeaf185c51fd1f9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0265d8f847a748519b8b74c308245784', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:48:00.784805 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92fcdb18-5f54-4000-b62b-bda8a7f26314 tempest-VolumeQuotaClassesTest-1080223495 tempest-VolumeQuotaClassesTest-1080223495-project-admin] REQUEST_METHOD: `POST` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:00.785638 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92fcdb18-5f54-4000-b62b-bda8a7f26314 tempest-VolumeQuotaClassesTest-1080223495 tempest-VolumeQuotaClassesTest-1080223495-project-admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:00.788017 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92fcdb18-5f54-4000-b62b-bda8a7f26314 tempest-VolumeQuotaClassesTest-1080223495 tempest-VolumeQuotaClassesTest-1080223495-project-admin] PATH_INFO: `/v3/projects` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:00.789022 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92fcdb18-5f54-4000-b62b-bda8a7f26314 tempest-VolumeQuotaClassesTest-1080223495 tempest-VolumeQuotaClassesTest-1080223495-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:48:00.795347 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92fcdb18-5f54-4000-b62b-bda8a7f26314 tempest-VolumeQuotaClassesTest-1080223495 tempest-VolumeQuotaClassesTest-1080223495-project-admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:48:00.852048 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-354fc8b8-4921-46da-98ac-9b0fc747863b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:48:00.853284 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354fc8b8-4921-46da-98ac-9b0fc747863b admin admin] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:00.853843 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354fc8b8-4921-46da-98ac-9b0fc747863b admin admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:00.854327 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354fc8b8-4921-46da-98ac-9b0fc747863b admin admin] PATH_INFO: `/v3/roles` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:00.855462 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-354fc8b8-4921-46da-98ac-9b0fc747863b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:48:00.857663 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-354fc8b8-4921-46da-98ac-9b0fc747863b admin admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:48:00.884607 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 857/1711] 10.222.0.13 () {58 vars in 1122 bytes} [Wed Aug 31 09:48:00 2022] GET /identity/v3/roles => generated 1420 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 09:48:00.893438 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8532ca-4acd-433a-b40f-ed82c861c5e8 None None] REQUEST_METHOD: `POST` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:00.893623 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 855/1712] 10.222.0.13 () {60 vars in 1153 bytes} [Wed Aug 31 09:48:00 2022] POST /identity/v3/projects => generated 354 bytes in 257 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 09:48:00.893623 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92fcdb18-5f54-4000-b62b-bda8a7f26314 tempest-VolumeQuotaClassesTest-1080223495 tempest-VolumeQuotaClassesTest-1080223495-project-admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:48:00.894693 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8532ca-4acd-433a-b40f-ed82c861c5e8 None None] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:00.895121 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8532ca-4acd-433a-b40f-ed82c861c5e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:01.019181 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.auth.core [None req-ef8532ca-4acd-433a-b40f-ed82c861c5e8 None None] MFA Rules not processed for user `a9bdaacd597442c2bcde2c555f2cc140`. Rule list: `[]` (Enabled: `True`). {{(pid=88114) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:48:01.022844 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.fernet_utils [None req-ef8532ca-4acd-433a-b40f-ed82c861c5e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88114) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:48:01.041895 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92fcdb18-5f54-4000-b62b-bda8a7f26314 tempest-VolumeQuotaClassesTest-1080223495 tempest-VolumeQuotaClassesTest-1080223495-project-admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:48:01.055656 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92133788-28c7-4eed-8c85-f6b5e0a3ca7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:48:01.056413 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92133788-28c7-4eed-8c85-f6b5e0a3ca7a admin admin] REQUEST_METHOD: `PUT` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:01.056697 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92133788-28c7-4eed-8c85-f6b5e0a3ca7a admin admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:01.056972 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92133788-28c7-4eed-8c85-f6b5e0a3ca7a admin admin] PATH_INFO: `/v3/projects/0c7662a0f481410fa31c1ec1b888dd52/users/989378c95b6c440e9a3a6f2ed6b78478/roles/5d4ee63a3add47fc9a6b8b16d39e0a1f` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:01.061758 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92133788-28c7-4eed-8c85-f6b5e0a3ca7a admin admin] RBAC: Authorizing `identity:create_grant(project_id=0c7662a0f481410fa31c1ec1b888dd52, user_id=989378c95b6c440e9a3a6f2ed6b78478, role_id=5d4ee63a3add47fc9a6b8b16d39e0a1f)` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:48:01.065517 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92133788-28c7-4eed-8c85-f6b5e0a3ca7a admin admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:48:01.096944 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 856/1713] 10.222.0.13 () {60 vars in 1483 bytes} [Wed Aug 31 09:48:00 2022] PUT /identity/v3/projects/0c7662a0f481410fa31c1ec1b888dd52/users/989378c95b6c440e9a3a6f2ed6b78478/roles/5d4ee63a3add47fc9a6b8b16d39e0a1f => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:48:01.097897 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92133788-28c7-4eed-8c85-f6b5e0a3ca7a admin admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:48:01.135940 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 858/1714] 10.222.0.13 () {58 vars in 1009 bytes} [Wed Aug 31 09:48:00 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 250 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:48:01.137673 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd3f718-eca8-49f7-a298-77b78df00057 None None] REQUEST_METHOD: `POST` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:01.138163 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd3f718-eca8-49f7-a298-77b78df00057 None None] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:01.138778 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd3f718-eca8-49f7-a298-77b78df00057 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:01.151195 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:48:01.242821 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.auth.core [None req-0cd3f718-eca8-49f7-a298-77b78df00057 None None] MFA Rules not processed for user `989378c95b6c440e9a3a6f2ed6b78478`. Rule list: `[]` (Enabled: `True`). {{(pid=88114) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:48:01.245373 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.fernet_utils [None req-0cd3f718-eca8-49f7-a298-77b78df00057 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88114) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:48:01.301538 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92133788-28c7-4eed-8c85-f6b5e0a3ca7a admin admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:48:01.321242 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-546f5abc-3ef2-4fd4-abf9-015916e020f0 tempest-VolumeQuotaClassesTest-1080223495 tempest-VolumeQuotaClassesTest-1080223495-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '59c84ddd914043f6aeaf185c51fd1f9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0265d8f847a748519b8b74c308245784', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:48:01.322391 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546f5abc-3ef2-4fd4-abf9-015916e020f0 tempest-VolumeQuotaClassesTest-1080223495 tempest-VolumeQuotaClassesTest-1080223495-project-admin] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:01.322675 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546f5abc-3ef2-4fd4-abf9-015916e020f0 tempest-VolumeQuotaClassesTest-1080223495 tempest-VolumeQuotaClassesTest-1080223495-project-admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:01.322951 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546f5abc-3ef2-4fd4-abf9-015916e020f0 tempest-VolumeQuotaClassesTest-1080223495 tempest-VolumeQuotaClassesTest-1080223495-project-admin] PATH_INFO: `/v3/domains` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:01.330933 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-546f5abc-3ef2-4fd4-abf9-015916e020f0 tempest-VolumeQuotaClassesTest-1080223495 tempest-VolumeQuotaClassesTest-1080223495-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=88113) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:48:01.330933 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-546f5abc-3ef2-4fd4-abf9-015916e020f0 tempest-VolumeQuotaClassesTest-1080223495 tempest-VolumeQuotaClassesTest-1080223495-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:48:01.333858 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-546f5abc-3ef2-4fd4-abf9-015916e020f0 tempest-VolumeQuotaClassesTest-1080223495 tempest-VolumeQuotaClassesTest-1080223495-project-admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:48:01.380294 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 857/1715] 10.222.0.13 () {58 vars in 1153 bytes} [Wed Aug 31 09:48:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 283 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:48:01.381214 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-546f5abc-3ef2-4fd4-abf9-015916e020f0 tempest-VolumeQuotaClassesTest-1080223495 tempest-VolumeQuotaClassesTest-1080223495-project-admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:48:01.384225 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.fernet_utils [None req-546f5abc-3ef2-4fd4-abf9-015916e020f0 tempest-VolumeQuotaClassesTest-1080223495 tempest-VolumeQuotaClassesTest-1080223495-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:48:01.414188 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=100 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:48:01.480166 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 859/1716] 10.222.0.13 () {58 vars in 958 bytes} [Wed Aug 31 09:48:01 2022] POST /identity/v3/auth/tokens => generated 3349 bytes in 343 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:48:01.481194 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd3f718-eca8-49f7-a298-77b78df00057 None None] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:48:01.495063 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd3f718-eca8-49f7-a298-77b78df00057 None None] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:48:01.507802 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1925b94-2daa-4df4-8d76-a4999447cdda tempest-VolumeQuotaClassesTest-1080223495 tempest-VolumeQuotaClassesTest-1080223495-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '59c84ddd914043f6aeaf185c51fd1f9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0265d8f847a748519b8b74c308245784', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:48:01.508584 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1925b94-2daa-4df4-8d76-a4999447cdda tempest-VolumeQuotaClassesTest-1080223495 tempest-VolumeQuotaClassesTest-1080223495-project-admin] REQUEST_METHOD: `DELETE` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:01.508803 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1925b94-2daa-4df4-8d76-a4999447cdda tempest-VolumeQuotaClassesTest-1080223495 tempest-VolumeQuotaClassesTest-1080223495-project-admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:01.509127 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1925b94-2daa-4df4-8d76-a4999447cdda tempest-VolumeQuotaClassesTest-1080223495 tempest-VolumeQuotaClassesTest-1080223495-project-admin] PATH_INFO: `/v3/projects/8bac1ecb1ec74746a08db507f152f3b1` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:01.510987 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1925b94-2daa-4df4-8d76-a4999447cdda tempest-VolumeQuotaClassesTest-1080223495 tempest-VolumeQuotaClassesTest-1080223495-project-admin] RBAC: Authorizing `identity:delete_project(project_id=8bac1ecb1ec74746a08db507f152f3b1)` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:48:01.513332 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1925b94-2daa-4df4-8d76-a4999447cdda tempest-VolumeQuotaClassesTest-1080223495 tempest-VolumeQuotaClassesTest-1080223495-project-admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:48:01.545536 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-546f5abc-3ef2-4fd4-abf9-015916e020f0 tempest-VolumeQuotaClassesTest-1080223495 tempest-VolumeQuotaClassesTest-1080223495-project-admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:48:01.554120 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-320bf5f2-38d3-4765-a42e-2e2c0e960663 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9bdaacd597442c2bcde2c555f2cc140', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '49548eb44d62415582af820ab66decee', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:48:01.554757 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320bf5f2-38d3-4765-a42e-2e2c0e960663 service neutron] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:01.555035 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320bf5f2-38d3-4765-a42e-2e2c0e960663 service neutron] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:01.555301 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320bf5f2-38d3-4765-a42e-2e2c0e960663 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:01.571823 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-320bf5f2-38d3-4765-a42e-2e2c0e960663 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:48:01.573214 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-320bf5f2-38d3-4765-a42e-2e2c0e960663 service neutron] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:48:01.595635 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 858/1717] 10.222.0.13 () {60 vars in 1365 bytes} [Wed Aug 31 09:48:01 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 215 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:48:01.596526 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-320bf5f2-38d3-4765-a42e-2e2c0e960663 service neutron] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:48:01.612939 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.notifications [None req-f1925b94-2daa-4df4-8d76-a4999447cdda tempest-VolumeQuotaClassesTest-1080223495 tempest-VolumeQuotaClassesTest-1080223495-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8bac1ecb1ec74746a08db507f152f3b1', 'request_id': 'req-f1925b94-2daa-4df4-8d76-a4999447cdda'} {{(pid=88114) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:48:01.622226 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.notifications [None req-f1925b94-2daa-4df4-8d76-a4999447cdda tempest-VolumeQuotaClassesTest-1080223495 tempest-VolumeQuotaClassesTest-1080223495-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8bac1ecb1ec74746a08db507f152f3b1', 'request_id': 'req-f1925b94-2daa-4df4-8d76-a4999447cdda'} {{(pid=88114) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:48:01.631519 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.notifications [None req-f1925b94-2daa-4df4-8d76-a4999447cdda tempest-VolumeQuotaClassesTest-1080223495 tempest-VolumeQuotaClassesTest-1080223495-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8bac1ecb1ec74746a08db507f152f3b1', 'request_id': 'req-f1925b94-2daa-4df4-8d76-a4999447cdda'} {{(pid=88114) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:48:01.655392 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.notifications [None req-f1925b94-2daa-4df4-8d76-a4999447cdda tempest-VolumeQuotaClassesTest-1080223495 tempest-VolumeQuotaClassesTest-1080223495-project-admin] The token cache is being invalidate because project 8bac1ecb1ec74746a08db507f152f3b1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88114) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 09:48:01.655829 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.notifications [None req-f1925b94-2daa-4df4-8d76-a4999447cdda tempest-VolumeQuotaClassesTest-1080223495 tempest-VolumeQuotaClassesTest-1080223495-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88114) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:48:01.656819 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.notifications [None req-f1925b94-2daa-4df4-8d76-a4999447cdda tempest-VolumeQuotaClassesTest-1080223495 tempest-VolumeQuotaClassesTest-1080223495-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88114) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:48:01.666295 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 860/1718] 10.222.0.13 () {58 vars in 1233 bytes} [Wed Aug 31 09:48:01 2022] DELETE /identity/v3/projects/8bac1ecb1ec74746a08db507f152f3b1 => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:48:01.715752 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-320bf5f2-38d3-4765-a42e-2e2c0e960663 service neutron] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:48:01.809146 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64bf6560-75c4-4465-b83d-44c24ed3ae81 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d52b69c8e94441dacfcff944326f169', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '49548eb44d62415582af820ab66decee', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:48:01.811438 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64bf6560-75c4-4465-b83d-44c24ed3ae81 service cinder] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:01.811886 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64bf6560-75c4-4465-b83d-44c24ed3ae81 service cinder] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:01.812288 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64bf6560-75c4-4465-b83d-44c24ed3ae81 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:01.817150 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.fernet_utils [None req-64bf6560-75c4-4465-b83d-44c24ed3ae81 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:48:01.883301 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=100 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:48:01.962958 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64bf6560-75c4-4465-b83d-44c24ed3ae81 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:48:01.964775 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64bf6560-75c4-4465-b83d-44c24ed3ae81 service cinder] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:48:01.990470 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 859/1719] 10.222.0.13 () {60 vars in 1365 bytes} [Wed Aug 31 09:48:01 2022] GET /identity/v3/auth/tokens => generated 3349 bytes in 394 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:48:02.116072 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1925b94-2daa-4df4-8d76-a4999447cdda tempest-VolumeQuotaClassesTest-1080223495 tempest-VolumeQuotaClassesTest-1080223495-project-admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:48:02.117671 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.fernet_utils [None req-f1925b94-2daa-4df4-8d76-a4999447cdda tempest-VolumeQuotaClassesTest-1080223495 tempest-VolumeQuotaClassesTest-1080223495-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88114) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:48:02.240183 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1925b94-2daa-4df4-8d76-a4999447cdda tempest-VolumeQuotaClassesTest-1080223495 tempest-VolumeQuotaClassesTest-1080223495-project-admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:48:02.265918 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3ad4d8-9b1e-4a1d-b201-7df4e9482978 None None] REQUEST_METHOD: `POST` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:02.266866 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3ad4d8-9b1e-4a1d-b201-7df4e9482978 None None] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:02.267292 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3ad4d8-9b1e-4a1d-b201-7df4e9482978 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:02.297165 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d6ff07-1179-47e9-807a-573602c09049 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:48:02.298286 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d6ff07-1179-47e9-807a-573602c09049 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:02.298716 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d6ff07-1179-47e9-807a-573602c09049 admin admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:02.299189 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d6ff07-1179-47e9-807a-573602c09049 admin admin] PATH_INFO: `/v3/users/ea77057efa394b6ea5c25b22651ebacb` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:02.301216 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1d6ff07-1179-47e9-807a-573602c09049 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ea77057efa394b6ea5c25b22651ebacb)` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:48:02.304653 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1d6ff07-1179-47e9-807a-573602c09049 admin admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:48:02.365557 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.auth.core [None req-1b3ad4d8-9b1e-4a1d-b201-7df4e9482978 None None] MFA Rules not processed for user `4961545b43f74fdb87d01b78600d54ed`. Rule list: `[]` (Enabled: `True`). {{(pid=88113) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:48:02.369741 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.fernet_utils [None req-1b3ad4d8-9b1e-4a1d-b201-7df4e9482978 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:48:02.419397 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.notifications [None req-c1d6ff07-1179-47e9-807a-573602c09049 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ea77057efa394b6ea5c25b22651ebacb'} {{(pid=88114) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:48:02.441982 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.notifications [None req-c1d6ff07-1179-47e9-807a-573602c09049 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ea77057efa394b6ea5c25b22651ebacb'} {{(pid=88114) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:48:02.480994 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.notifications [None req-c1d6ff07-1179-47e9-807a-573602c09049 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ea77057efa394b6ea5c25b22651ebacb'} {{(pid=88114) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:48:02.481819 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.notifications [None req-c1d6ff07-1179-47e9-807a-573602c09049 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ea77057efa394b6ea5c25b22651ebacb'} {{(pid=88114) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:48:02.488844 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:48:02.539646 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.notifications [None req-c1d6ff07-1179-47e9-807a-573602c09049 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ea77057efa394b6ea5c25b22651ebacb'} {{(pid=88114) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:48:02.581023 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 861/1720] 10.222.0.13 () {58 vars in 1224 bytes} [Wed Aug 31 09:48:02 2022] DELETE /identity/v3/users/ea77057efa394b6ea5c25b22651ebacb => generated 0 bytes in 462 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:48:02.587784 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f99d007-bcee-4aa3-bf21-ae5c2a1b63d5 None None] REQUEST_METHOD: `POST` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:02.588263 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f99d007-bcee-4aa3-bf21-ae5c2a1b63d5 None None] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:02.589541 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f99d007-bcee-4aa3-bf21-ae5c2a1b63d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:02.652534 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.auth.core [None req-1f99d007-bcee-4aa3-bf21-ae5c2a1b63d5 None None] MFA Rules not processed for user `fd836cf590a145df9acf19cf3ae2ac56`. Rule list: `[]` (Enabled: `True`). {{(pid=88114) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:48:02.654594 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.fernet_utils [None req-1f99d007-bcee-4aa3-bf21-ae5c2a1b63d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88114) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:48:02.669563 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 860/1721] 10.222.0.13 () {58 vars in 958 bytes} [Wed Aug 31 09:48:02 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 401 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:48:02.669563 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b3ad4d8-9b1e-4a1d-b201-7df4e9482978 None None] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:48:02.669563 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.fernet_utils [None req-1b3ad4d8-9b1e-4a1d-b201-7df4e9482978 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:48:02.877856 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 862/1722] 10.222.0.13 () {58 vars in 958 bytes} [Wed Aug 31 09:48:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 295 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:48:02.878873 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f99d007-bcee-4aa3-bf21-ae5c2a1b63d5 None None] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:48:02.885542 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.fernet_utils [None req-1f99d007-bcee-4aa3-bf21-ae5c2a1b63d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88114) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:48:02.897613 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b3ad4d8-9b1e-4a1d-b201-7df4e9482978 None None] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:48:03.012781 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=100 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:48:03.031216 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b67407cc-0710-429c-8d73-faec1402ca31 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ff3aff5ef444910be6dbb989d0e3194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '49548eb44d62415582af820ab66decee', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:48:03.031860 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67407cc-0710-429c-8d73-faec1402ca31 service placement] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:03.032143 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67407cc-0710-429c-8d73-faec1402ca31 service placement] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:03.032418 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67407cc-0710-429c-8d73-faec1402ca31 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:03.034613 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.fernet_utils [None req-b67407cc-0710-429c-8d73-faec1402ca31 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:48:03.043709 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f99d007-bcee-4aa3-bf21-ae5c2a1b63d5 None None] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:48:03.049574 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b67407cc-0710-429c-8d73-faec1402ca31 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:48:03.051432 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b67407cc-0710-429c-8d73-faec1402ca31 service placement] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:48:03.054834 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1110bc3-caeb-46fc-ad5a-b9c79746ed21 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc076493239d4ccdbdc37c0369c49529', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '49548eb44d62415582af820ab66decee', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:48:03.055884 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1110bc3-caeb-46fc-ad5a-b9c79746ed21 service nova] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:03.056376 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1110bc3-caeb-46fc-ad5a-b9c79746ed21 service nova] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:03.057054 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1110bc3-caeb-46fc-ad5a-b9c79746ed21 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:03.065927 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 861/1723] 10.222.0.13 () {60 vars in 1365 bytes} [Wed Aug 31 09:48:02 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 399 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:48:03.068047 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02a2af8-725f-4b3c-96a8-b8faf9c09863 None None] REQUEST_METHOD: `POST` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:03.068341 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02a2af8-725f-4b3c-96a8-b8faf9c09863 None None] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:03.068668 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02a2af8-725f-4b3c-96a8-b8faf9c09863 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:03.095773 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.auth.core [None req-a02a2af8-725f-4b3c-96a8-b8faf9c09863 None None] MFA Rules not processed for user `886e40cf192d46c2b2e7eb2a95ec62e1`. Rule list: `[]` (Enabled: `True`). {{(pid=88113) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:48:03.097283 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.fernet_utils [None req-a02a2af8-725f-4b3c-96a8-b8faf9c09863 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:48:03.138071 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1110bc3-caeb-46fc-ad5a-b9c79746ed21 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:48:03.140211 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1110bc3-caeb-46fc-ad5a-b9c79746ed21 service nova] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:48:03.153961 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 863/1724] 10.222.0.13 () {60 vars in 1365 bytes} [Wed Aug 31 09:48:02 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 275 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:48:03.155765 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c286d1-8f5b-4163-b54b-db11bf2ced8b None None] REQUEST_METHOD: `POST` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:03.156153 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c286d1-8f5b-4163-b54b-db11bf2ced8b None None] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:03.156614 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c286d1-8f5b-4163-b54b-db11bf2ced8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:03.232217 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.auth.core [None req-95c286d1-8f5b-4163-b54b-db11bf2ced8b None None] MFA Rules not processed for user `ae46122578a143a78f07f7f3a80ae079`. Rule list: `[]` (Enabled: `True`). {{(pid=88114) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:48:03.233648 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.fernet_utils [None req-95c286d1-8f5b-4163-b54b-db11bf2ced8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88114) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:48:03.238653 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 862/1725] 10.222.0.13 () {58 vars in 958 bytes} [Wed Aug 31 09:48:03 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:48:03.244464 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a02a2af8-725f-4b3c-96a8-b8faf9c09863 None None] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:48:03.247054 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:48:03.248843 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.fernet_utils [None req-a02a2af8-725f-4b3c-96a8-b8faf9c09863 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:48:03.324578 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 864/1726] 10.222.0.13 () {58 vars in 958 bytes} [Wed Aug 31 09:48:03 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:48:03.326337 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f457edb-473b-4e0d-8434-64712e69ac9e None None] REQUEST_METHOD: `POST` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:03.326750 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f457edb-473b-4e0d-8434-64712e69ac9e None None] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:03.327579 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f457edb-473b-4e0d-8434-64712e69ac9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:03.371507 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.auth.core [None req-2f457edb-473b-4e0d-8434-64712e69ac9e None None] MFA Rules not processed for user `e4f1a935c10540f58c25b26fa4403929`. Rule list: `[]` (Enabled: `True`). {{(pid=88114) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:48:03.373377 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.fernet_utils [None req-2f457edb-473b-4e0d-8434-64712e69ac9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88114) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:48:03.456820 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a02a2af8-725f-4b3c-96a8-b8faf9c09863 None None] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:48:03.471610 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8dc664e-4c6c-4a3c-9dd5-0d653f58c33b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9bdaacd597442c2bcde2c555f2cc140', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '49548eb44d62415582af820ab66decee', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:48:03.472659 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8dc664e-4c6c-4a3c-9dd5-0d653f58c33b service neutron] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:03.473266 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8dc664e-4c6c-4a3c-9dd5-0d653f58c33b service neutron] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:03.474309 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8dc664e-4c6c-4a3c-9dd5-0d653f58c33b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:03.501819 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8dc664e-4c6c-4a3c-9dd5-0d653f58c33b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:48:03.504107 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8dc664e-4c6c-4a3c-9dd5-0d653f58c33b service neutron] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:48:03.519700 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 863/1727] 10.222.0.13 () {60 vars in 1365 bytes} [Wed Aug 31 09:48:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 279 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:48:03.520334 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08286cd1-0d26-470e-a643-2478c4173658 None None] REQUEST_METHOD: `POST` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:03.521297 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08286cd1-0d26-470e-a643-2478c4173658 None None] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:03.521996 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08286cd1-0d26-470e-a643-2478c4173658 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:03.548154 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.auth.core [None req-08286cd1-0d26-470e-a643-2478c4173658 None None] MFA Rules not processed for user `ae46122578a143a78f07f7f3a80ae079`. Rule list: `[]` (Enabled: `True`). {{(pid=88113) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:48:03.549815 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.fernet_utils [None req-08286cd1-0d26-470e-a643-2478c4173658 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:48:03.561553 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 864/1728] 10.222.0.13 () {58 vars in 958 bytes} [Wed Aug 31 09:48:03 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:48:03.577305 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08286cd1-0d26-470e-a643-2478c4173658 None None] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:48:03.594326 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 865/1729] 10.222.0.13 () {58 vars in 958 bytes} [Wed Aug 31 09:48:03 2022] POST /identity/v3/auth/tokens => generated 3347 bytes in 269 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:48:03.623542 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08286cd1-0d26-470e-a643-2478c4173658 None None] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:48:03.633042 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed7f86e-37fe-413a-a22f-633ca8faba12 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc076493239d4ccdbdc37c0369c49529', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '49548eb44d62415582af820ab66decee', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:48:03.633739 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed7f86e-37fe-413a-a22f-633ca8faba12 service nova] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:03.634018 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed7f86e-37fe-413a-a22f-633ca8faba12 service nova] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:03.634471 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed7f86e-37fe-413a-a22f-633ca8faba12 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:03.648142 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aed7f86e-37fe-413a-a22f-633ca8faba12 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:48:03.649914 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aed7f86e-37fe-413a-a22f-633ca8faba12 service nova] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:48:03.660733 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 865/1730] 10.222.0.13 () {60 vars in 1365 bytes} [Wed Aug 31 09:48:03 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 09:48:04.171722 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f64c694-3839-42e1-bb48-9cabe5e74d1a None None] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:04.172229 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f64c694-3839-42e1-bb48-9cabe5e74d1a None None] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:04.172698 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f64c694-3839-42e1-bb48-9cabe5e74d1a None None] PATH_INFO: `/` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:04.175667 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 866/1731] 10.222.0.13 () {54 vars in 912 bytes} [Wed Aug 31 09:48:04 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:48:04.182603 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b0524c-a35c-4df3-8d20-42e8a0f96065 None None] REQUEST_METHOD: `POST` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:04.183143 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b0524c-a35c-4df3-8d20-42e8a0f96065 None None] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:04.183630 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b0524c-a35c-4df3-8d20-42e8a0f96065 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:04.257959 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.auth.core [None req-d3b0524c-a35c-4df3-8d20-42e8a0f96065 None None] MFA Rules not processed for user `a9bdaacd597442c2bcde2c555f2cc140`. Rule list: `[]` (Enabled: `True`). {{(pid=88113) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:48:04.260321 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.fernet_utils [None req-d3b0524c-a35c-4df3-8d20-42e8a0f96065 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:48:04.293794 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 866/1732] 10.222.0.13 () {58 vars in 1009 bytes} [Wed Aug 31 09:48:04 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:48:04.349046 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f64c694-3839-42e1-bb48-9cabe5e74d1a None None] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:48:04.351101 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.fernet_utils [None req-7f64c694-3839-42e1-bb48-9cabe5e74d1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88114) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:48:04.378691 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f64c694-3839-42e1-bb48-9cabe5e74d1a None None] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:48:04.389761 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1213a8a1-2d8e-45c3-ab66-02f2b673a013 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9bdaacd597442c2bcde2c555f2cc140', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '49548eb44d62415582af820ab66decee', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:48:04.390420 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1213a8a1-2d8e-45c3-ab66-02f2b673a013 service neutron] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:04.390703 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1213a8a1-2d8e-45c3-ab66-02f2b673a013 service neutron] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:04.390980 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1213a8a1-2d8e-45c3-ab66-02f2b673a013 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:04.405512 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1213a8a1-2d8e-45c3-ab66-02f2b673a013 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:48:04.407004 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1213a8a1-2d8e-45c3-ab66-02f2b673a013 service neutron] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:48:04.432414 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 867/1733] 10.222.0.13 () {60 vars in 1365 bytes} [Wed Aug 31 09:48:04 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:48:04.511856 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b0524c-a35c-4df3-8d20-42e8a0f96065 None None] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:48:04.513711 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.fernet_utils [None req-d3b0524c-a35c-4df3-8d20-42e8a0f96065 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:48:04.551729 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b0524c-a35c-4df3-8d20-42e8a0f96065 None None] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:48:04.560775 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1213a8a1-2d8e-45c3-ab66-02f2b673a013 service neutron] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:48:04.562331 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.fernet_utils [None req-1213a8a1-2d8e-45c3-ab66-02f2b673a013 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88114) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:48:04.585673 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2801bf6f-6048-4708-a2dc-93ba39c399ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:48:04.588613 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2801bf6f-6048-4708-a2dc-93ba39c399ff admin admin] REQUEST_METHOD: `DELETE` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:04.588987 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2801bf6f-6048-4708-a2dc-93ba39c399ff admin admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:04.589956 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2801bf6f-6048-4708-a2dc-93ba39c399ff admin admin] PATH_INFO: `/v3/users/f1859851143c43539a34fb9a2c2b68c3` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:04.593690 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2801bf6f-6048-4708-a2dc-93ba39c399ff admin admin] RBAC: Authorizing `identity:delete_user(user_id=f1859851143c43539a34fb9a2c2b68c3)` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:48:04.602873 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2801bf6f-6048-4708-a2dc-93ba39c399ff admin admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:48:04.605678 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1213a8a1-2d8e-45c3-ab66-02f2b673a013 service neutron] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:48:04.631598 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec0b8a39-f04c-4c66-9ed8-1fed92a1853b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:48:04.632479 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0b8a39-f04c-4c66-9ed8-1fed92a1853b admin admin] REQUEST_METHOD: `DELETE` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:04.634044 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0b8a39-f04c-4c66-9ed8-1fed92a1853b admin admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:04.637127 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0b8a39-f04c-4c66-9ed8-1fed92a1853b admin admin] PATH_INFO: `/v3/projects/90ac571b08da4c508ec9cf3bec5addb5` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:04.640898 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec0b8a39-f04c-4c66-9ed8-1fed92a1853b admin admin] RBAC: Authorizing `identity:delete_project(project_id=90ac571b08da4c508ec9cf3bec5addb5)` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:48:04.644211 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec0b8a39-f04c-4c66-9ed8-1fed92a1853b admin admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:48:04.783644 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.notifications [None req-2801bf6f-6048-4708-a2dc-93ba39c399ff admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f1859851143c43539a34fb9a2c2b68c3'} {{(pid=88113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:48:04.803652 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.notifications [None req-2801bf6f-6048-4708-a2dc-93ba39c399ff admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f1859851143c43539a34fb9a2c2b68c3'} {{(pid=88113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:48:04.808378 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:48:04.814084 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.notifications [None req-ec0b8a39-f04c-4c66-9ed8-1fed92a1853b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '90ac571b08da4c508ec9cf3bec5addb5', 'request_id': 'req-ec0b8a39-f04c-4c66-9ed8-1fed92a1853b'} {{(pid=88114) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:48:04.826262 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.notifications [None req-ec0b8a39-f04c-4c66-9ed8-1fed92a1853b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '90ac571b08da4c508ec9cf3bec5addb5', 'request_id': 'req-ec0b8a39-f04c-4c66-9ed8-1fed92a1853b'} {{(pid=88114) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:48:04.845252 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.notifications [None req-ec0b8a39-f04c-4c66-9ed8-1fed92a1853b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '90ac571b08da4c508ec9cf3bec5addb5', 'request_id': 'req-ec0b8a39-f04c-4c66-9ed8-1fed92a1853b'} {{(pid=88114) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:48:04.865308 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.notifications [None req-ec0b8a39-f04c-4c66-9ed8-1fed92a1853b admin admin] The token cache is being invalidate because project 90ac571b08da4c508ec9cf3bec5addb5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88114) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 09:48:04.865785 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.notifications [None req-ec0b8a39-f04c-4c66-9ed8-1fed92a1853b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88114) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:48:04.870862 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.notifications [None req-2801bf6f-6048-4708-a2dc-93ba39c399ff admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f1859851143c43539a34fb9a2c2b68c3'} {{(pid=88113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:48:04.871474 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.notifications [None req-ec0b8a39-f04c-4c66-9ed8-1fed92a1853b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88114) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:48:04.876284 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 868/1734] 10.222.0.13 () {58 vars in 1233 bytes} [Wed Aug 31 09:48:04 2022] DELETE /identity/v3/projects/90ac571b08da4c508ec9cf3bec5addb5 => generated 0 bytes in 313 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:48:04.876475 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.notifications [None req-2801bf6f-6048-4708-a2dc-93ba39c399ff admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f1859851143c43539a34fb9a2c2b68c3'} {{(pid=88113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:48:04.877921 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec0b8a39-f04c-4c66-9ed8-1fed92a1853b admin admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:48:04.879524 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.fernet_utils [None req-ec0b8a39-f04c-4c66-9ed8-1fed92a1853b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88114) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:48:04.893941 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:48:04.910237 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.notifications [None req-2801bf6f-6048-4708-a2dc-93ba39c399ff admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f1859851143c43539a34fb9a2c2b68c3'} {{(pid=88113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:48:04.936899 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 867/1735] 10.222.0.13 () {58 vars in 1224 bytes} [Wed Aug 31 09:48:04 2022] DELETE /identity/v3/users/f1859851143c43539a34fb9a2c2b68c3 => generated 0 bytes in 426 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:48:04.944106 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653fc177-b14f-42ad-93a1-fad1eec2b48d None None] REQUEST_METHOD: `POST` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:04.944391 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653fc177-b14f-42ad-93a1-fad1eec2b48d None None] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:04.947531 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653fc177-b14f-42ad-93a1-fad1eec2b48d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:04.984955 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.auth.core [None req-653fc177-b14f-42ad-93a1-fad1eec2b48d None None] MFA Rules not processed for user `fd836cf590a145df9acf19cf3ae2ac56`. Rule list: `[]` (Enabled: `True`). {{(pid=88113) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:48:04.986846 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.fernet_utils [None req-653fc177-b14f-42ad-93a1-fad1eec2b48d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:48:05.030011 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec0b8a39-f04c-4c66-9ed8-1fed92a1853b admin admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:48:05.144721 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 868/1736] 10.222.0.13 () {58 vars in 958 bytes} [Wed Aug 31 09:48:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:48:05.170925 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ccb41e-b25f-41b1-bf12-cee53613b29b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:48:05.171414 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ccb41e-b25f-41b1-bf12-cee53613b29b admin admin] REQUEST_METHOD: `DELETE` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:05.171795 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ccb41e-b25f-41b1-bf12-cee53613b29b admin admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:05.172338 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ccb41e-b25f-41b1-bf12-cee53613b29b admin admin] PATH_INFO: `/v3/users/59c84ddd914043f6aeaf185c51fd1f9d` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:05.176706 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9ccb41e-b25f-41b1-bf12-cee53613b29b admin admin] RBAC: Authorizing `identity:delete_user(user_id=59c84ddd914043f6aeaf185c51fd1f9d)` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:48:05.179636 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9ccb41e-b25f-41b1-bf12-cee53613b29b admin admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:48:05.287101 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.notifications [None req-e9ccb41e-b25f-41b1-bf12-cee53613b29b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '59c84ddd914043f6aeaf185c51fd1f9d'} {{(pid=88114) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:48:05.321670 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.notifications [None req-e9ccb41e-b25f-41b1-bf12-cee53613b29b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '59c84ddd914043f6aeaf185c51fd1f9d'} {{(pid=88114) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:48:05.339553 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.notifications [None req-e9ccb41e-b25f-41b1-bf12-cee53613b29b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '59c84ddd914043f6aeaf185c51fd1f9d'} {{(pid=88114) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:48:05.340493 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.notifications [None req-e9ccb41e-b25f-41b1-bf12-cee53613b29b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '59c84ddd914043f6aeaf185c51fd1f9d'} {{(pid=88114) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:48:05.391351 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.notifications [None req-e9ccb41e-b25f-41b1-bf12-cee53613b29b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '59c84ddd914043f6aeaf185c51fd1f9d'} {{(pid=88114) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:48:05.428977 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 869/1737] 10.222.0.13 () {58 vars in 1224 bytes} [Wed Aug 31 09:48:04 2022] DELETE /identity/v3/users/59c84ddd914043f6aeaf185c51fd1f9d => generated 0 bytes in 548 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:48:05.429197 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-653fc177-b14f-42ad-93a1-fad1eec2b48d None None] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:48:05.429653 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.fernet_utils [None req-653fc177-b14f-42ad-93a1-fad1eec2b48d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:48:05.588653 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-653fc177-b14f-42ad-93a1-fad1eec2b48d None None] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:48:05.662876 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8eb9c4c-7b9a-4250-bf0e-45fb4c168d96 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9bdaacd597442c2bcde2c555f2cc140', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '49548eb44d62415582af820ab66decee', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:48:05.663607 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8eb9c4c-7b9a-4250-bf0e-45fb4c168d96 service neutron] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:05.663887 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8eb9c4c-7b9a-4250-bf0e-45fb4c168d96 service neutron] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:05.664158 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8eb9c4c-7b9a-4250-bf0e-45fb4c168d96 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:05.666868 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.fernet_utils [None req-c8eb9c4c-7b9a-4250-bf0e-45fb4c168d96 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:48:05.785115 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=96,keystone:DELETE=3,keystone:INSERT=1 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:48:05.828865 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8eb9c4c-7b9a-4250-bf0e-45fb4c168d96 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:48:05.830595 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8eb9c4c-7b9a-4250-bf0e-45fb4c168d96 service neutron] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:48:05.862818 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 869/1738] 10.222.0.13 () {60 vars in 1365 bytes} [Wed Aug 31 09:48:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 435 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:48:06.623871 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827eb881-3d79-43fc-aa65-4fe07e128005 None None] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:06.624757 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827eb881-3d79-43fc-aa65-4fe07e128005 None None] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:06.625192 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827eb881-3d79-43fc-aa65-4fe07e128005 None None] PATH_INFO: `/` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:06.628030 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 870/1739] 10.222.0.13 () {54 vars in 921 bytes} [Wed Aug 31 09:48:06 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:48:06.669102 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c431b988-08e8-4cb7-bd39-19794eefe7e0 None None] REQUEST_METHOD: `POST` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:06.669664 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c431b988-08e8-4cb7-bd39-19794eefe7e0 None None] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:06.669664 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c431b988-08e8-4cb7-bd39-19794eefe7e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:06.736555 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.auth.core [None req-c431b988-08e8-4cb7-bd39-19794eefe7e0 None None] MFA Rules not processed for user `1ff3aff5ef444910be6dbb989d0e3194`. Rule list: `[]` (Enabled: `True`). {{(pid=88113) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:48:06.746424 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.fernet_utils [None req-c431b988-08e8-4cb7-bd39-19794eefe7e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:48:07.065616 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 870/1740] 10.222.0.13 () {58 vars in 1018 bytes} [Wed Aug 31 09:48:06 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 397 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:48:07.135840 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-827eb881-3d79-43fc-aa65-4fe07e128005 None None] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:48:07.137781 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.fernet_utils [None req-827eb881-3d79-43fc-aa65-4fe07e128005 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88114) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:48:07.181903 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-827eb881-3d79-43fc-aa65-4fe07e128005 None None] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:48:07.200496 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6828e4e4-3aaa-4da6-961d-bae076e55958 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ff3aff5ef444910be6dbb989d0e3194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '49548eb44d62415582af820ab66decee', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:48:07.202029 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6828e4e4-3aaa-4da6-961d-bae076e55958 service placement] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:07.202797 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6828e4e4-3aaa-4da6-961d-bae076e55958 service placement] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:07.203413 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6828e4e4-3aaa-4da6-961d-bae076e55958 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:07.216147 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6828e4e4-3aaa-4da6-961d-bae076e55958 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:48:07.218317 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6828e4e4-3aaa-4da6-961d-bae076e55958 service placement] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:48:07.232080 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 871/1741] 10.222.0.13 () {60 vars in 1365 bytes} [Wed Aug 31 09:48:07 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:48:07.383332 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c431b988-08e8-4cb7-bd39-19794eefe7e0 None None] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:48:07.385770 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.fernet_utils [None req-c431b988-08e8-4cb7-bd39-19794eefe7e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:48:07.433204 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45da8d4f-ef06-48d2-9eef-2ba32e13a7f3 None None] REQUEST_METHOD: `POST` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:07.433476 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45da8d4f-ef06-48d2-9eef-2ba32e13a7f3 None None] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:07.433704 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45da8d4f-ef06-48d2-9eef-2ba32e13a7f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:07.462698 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c431b988-08e8-4cb7-bd39-19794eefe7e0 None None] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:48:07.480570 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a48445c-cdd4-4101-8392-8d25dde3e383 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:48:07.483147 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a48445c-cdd4-4101-8392-8d25dde3e383 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:07.484295 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a48445c-cdd4-4101-8392-8d25dde3e383 admin admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:07.484658 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a48445c-cdd4-4101-8392-8d25dde3e383 admin admin] PATH_INFO: `/v3/projects/0265d8f847a748519b8b74c308245784` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:07.486388 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a48445c-cdd4-4101-8392-8d25dde3e383 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0265d8f847a748519b8b74c308245784)` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:48:07.490268 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a48445c-cdd4-4101-8392-8d25dde3e383 admin admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:48:07.526171 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:48:07.576858 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.auth.core [None req-45da8d4f-ef06-48d2-9eef-2ba32e13a7f3 None None] MFA Rules not processed for user `768cb67c1ef24251aaa113c12d81a250`. Rule list: `[]` (Enabled: `True`). {{(pid=88114) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:48:07.578155 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.fernet_utils [None req-45da8d4f-ef06-48d2-9eef-2ba32e13a7f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88114) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:48:07.626357 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.notifications [None req-0a48445c-cdd4-4101-8392-8d25dde3e383 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0265d8f847a748519b8b74c308245784', 'request_id': 'req-0a48445c-cdd4-4101-8392-8d25dde3e383'} {{(pid=88113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:48:07.638757 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.notifications [None req-0a48445c-cdd4-4101-8392-8d25dde3e383 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0265d8f847a748519b8b74c308245784', 'request_id': 'req-0a48445c-cdd4-4101-8392-8d25dde3e383'} {{(pid=88113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:48:07.673278 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.notifications [None req-0a48445c-cdd4-4101-8392-8d25dde3e383 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0265d8f847a748519b8b74c308245784', 'request_id': 'req-0a48445c-cdd4-4101-8392-8d25dde3e383'} {{(pid=88113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:48:07.724332 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.notifications [None req-0a48445c-cdd4-4101-8392-8d25dde3e383 admin admin] The token cache is being invalidate because project 0265d8f847a748519b8b74c308245784 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88113) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 09:48:07.724510 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.notifications [None req-0a48445c-cdd4-4101-8392-8d25dde3e383 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:48:07.734158 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.notifications [None req-0a48445c-cdd4-4101-8392-8d25dde3e383 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:48:07.736538 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 871/1742] 10.222.0.13 () {58 vars in 1233 bytes} [Wed Aug 31 09:48:07 2022] DELETE /identity/v3/projects/0265d8f847a748519b8b74c308245784 => generated 0 bytes in 354 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:48:07.741462 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2f00af-094f-4c43-a57e-91dc883a475a None None] REQUEST_METHOD: `POST` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:07.742343 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2f00af-094f-4c43-a57e-91dc883a475a None None] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:07.742800 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2f00af-094f-4c43-a57e-91dc883a475a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:07.788117 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 872/1743] 10.222.0.13 () {58 vars in 1009 bytes} [Wed Aug 31 09:48:07 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 356 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:48:07.848329 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.auth.core [None req-7e2f00af-094f-4c43-a57e-91dc883a475a None None] MFA Rules not processed for user `fd836cf590a145df9acf19cf3ae2ac56`. Rule list: `[]` (Enabled: `True`). {{(pid=88113) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:48:07.859732 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.fernet_utils [None req-7e2f00af-094f-4c43-a57e-91dc883a475a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:48:08.030215 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 872/1744] 10.222.0.13 () {58 vars in 958 bytes} [Wed Aug 31 09:48:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 290 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 09:48:08.040426 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80317aa1-5293-4cbc-a384-3b98a0b87caf None None] REQUEST_METHOD: `POST` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:08.040706 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80317aa1-5293-4cbc-a384-3b98a0b87caf None None] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:08.041154 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80317aa1-5293-4cbc-a384-3b98a0b87caf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:08.065724 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e2f00af-094f-4c43-a57e-91dc883a475a None None] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:48:08.068340 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.fernet_utils [None req-7e2f00af-094f-4c43-a57e-91dc883a475a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:48:08.077951 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:48:08.080111 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.auth.core [None req-80317aa1-5293-4cbc-a384-3b98a0b87caf None None] MFA Rules not processed for user `fd836cf590a145df9acf19cf3ae2ac56`. Rule list: `[]` (Enabled: `True`). {{(pid=88114) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:48:08.088638 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.fernet_utils [None req-80317aa1-5293-4cbc-a384-3b98a0b87caf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88114) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:48:08.109311 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 873/1745] 10.222.0.13 () {58 vars in 958 bytes} [Wed Aug 31 09:48:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 70 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:48:08.113783 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80317aa1-5293-4cbc-a384-3b98a0b87caf None None] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:48:08.134117 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80317aa1-5293-4cbc-a384-3b98a0b87caf None None] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:48:08.139540 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01173984-c152-4fb7-ae87-f805703f4ade admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:48:08.140543 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01173984-c152-4fb7-ae87-f805703f4ade admin admin] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:08.141030 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01173984-c152-4fb7-ae87-f805703f4ade admin admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:08.141772 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01173984-c152-4fb7-ae87-f805703f4ade admin admin] PATH_INFO: `/v3/domains` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:08.143266 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01173984-c152-4fb7-ae87-f805703f4ade admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88114) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:48:08.144275 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01173984-c152-4fb7-ae87-f805703f4ade admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:48:08.146948 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01173984-c152-4fb7-ae87-f805703f4ade admin admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:48:08.206726 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 874/1746] 10.222.0.13 () {58 vars in 1153 bytes} [Wed Aug 31 09:48:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:48:08.208342 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01173984-c152-4fb7-ae87-f805703f4ade admin admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:48:08.231255 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01173984-c152-4fb7-ae87-f805703f4ade admin admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:48:08.242463 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c24796-d93b-4d47-a182-d1e0c9ed35ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:48:08.243253 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c24796-d93b-4d47-a182-d1e0c9ed35ed admin admin] REQUEST_METHOD: `POST` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:08.245772 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c24796-d93b-4d47-a182-d1e0c9ed35ed admin admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:08.246187 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c24796-d93b-4d47-a182-d1e0c9ed35ed admin admin] PATH_INFO: `/v3/projects` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:08.247086 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84c24796-d93b-4d47-a182-d1e0c9ed35ed admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:48:08.254188 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84c24796-d93b-4d47-a182-d1e0c9ed35ed admin admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:48:08.378803 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 875/1747] 10.222.0.13 () {60 vars in 1153 bytes} [Wed Aug 31 09:48:08 2022] POST /identity/v3/projects => generated 367 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 09:48:08.378803 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c24796-d93b-4d47-a182-d1e0c9ed35ed admin admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:48:08.390229 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e2f00af-094f-4c43-a57e-91dc883a475a None None] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:48:08.624244 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8657f129-5d8d-47ae-8722-524c43b30efb service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '202bb9db7a8a4977ba788a290ed9a3fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '49548eb44d62415582af820ab66decee', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:48:08.625269 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8657f129-5d8d-47ae-8722-524c43b30efb service swift] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:08.625721 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8657f129-5d8d-47ae-8722-524c43b30efb service swift] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:08.628568 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8657f129-5d8d-47ae-8722-524c43b30efb service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:08.628771 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c24796-d93b-4d47-a182-d1e0c9ed35ed admin admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:48:08.643880 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4606c436-e8ae-47b1-91b6-3c2f0ccb6a07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:48:08.644598 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4606c436-e8ae-47b1-91b6-3c2f0ccb6a07 admin admin] REQUEST_METHOD: `POST` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:08.644990 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4606c436-e8ae-47b1-91b6-3c2f0ccb6a07 admin admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:08.645349 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4606c436-e8ae-47b1-91b6-3c2f0ccb6a07 admin admin] PATH_INFO: `/v3/users` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:08.647378 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4606c436-e8ae-47b1-91b6-3c2f0ccb6a07 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:48:08.655129 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4606c436-e8ae-47b1-91b6-3c2f0ccb6a07 admin admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:48:08.722074 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 876/1748] 10.222.0.13 () {60 vars in 1144 bytes} [Wed Aug 31 09:48:08 2022] POST /identity/v3/users => generated 284 bytes in 348 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 09:48:08.726758 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4606c436-e8ae-47b1-91b6-3c2f0ccb6a07 admin admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:48:08.733915 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.fernet_utils [None req-4606c436-e8ae-47b1-91b6-3c2f0ccb6a07 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88114) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:48:08.773826 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4606c436-e8ae-47b1-91b6-3c2f0ccb6a07 admin admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:48:08.786799 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2872b18d-a43e-4947-8689-a33db22fcf14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:48:08.787462 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2872b18d-a43e-4947-8689-a33db22fcf14 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:08.787741 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2872b18d-a43e-4947-8689-a33db22fcf14 admin admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:08.788013 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2872b18d-a43e-4947-8689-a33db22fcf14 admin admin] PATH_INFO: `/v3/users/f8bffa6c63644be1b1087916e000999a` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:08.790116 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8657f129-5d8d-47ae-8722-524c43b30efb service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:48:08.790905 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2872b18d-a43e-4947-8689-a33db22fcf14 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f8bffa6c63644be1b1087916e000999a)` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:48:08.793314 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8657f129-5d8d-47ae-8722-524c43b30efb service swift] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:48:08.796691 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2872b18d-a43e-4947-8689-a33db22fcf14 admin admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:48:08.815261 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 873/1749] 10.222.0.13 () {60 vars in 1384 bytes} [Wed Aug 31 09:48:08 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 750 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 31 09:48:08.817622 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8657f129-5d8d-47ae-8722-524c43b30efb service swift] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:48:08.837812 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8657f129-5d8d-47ae-8722-524c43b30efb service swift] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:48:08.861554 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9afaf0e5-cd86-4183-88b4-ca2de1db7250 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:48:08.862344 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9afaf0e5-cd86-4183-88b4-ca2de1db7250 admin admin] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:08.862629 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9afaf0e5-cd86-4183-88b4-ca2de1db7250 admin admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:08.862895 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9afaf0e5-cd86-4183-88b4-ca2de1db7250 admin admin] PATH_INFO: `/v3/roles` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:08.869497 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9afaf0e5-cd86-4183-88b4-ca2de1db7250 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:48:08.873504 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9afaf0e5-cd86-4183-88b4-ca2de1db7250 admin admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:48:08.938891 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=91,keystone:INSERT=4,keystone:DELETE=5 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:48:08.941855 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 874/1750] 10.222.0.13 () {58 vars in 1122 bytes} [Wed Aug 31 09:48:08 2022] GET /identity/v3/roles => generated 1420 bytes in 124 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 09:48:08.943309 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9afaf0e5-cd86-4183-88b4-ca2de1db7250 admin admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:48:08.986509 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9afaf0e5-cd86-4183-88b4-ca2de1db7250 admin admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:48:08.999893 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.notifications [None req-2872b18d-a43e-4947-8689-a33db22fcf14 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f8bffa6c63644be1b1087916e000999a'} {{(pid=88114) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:48:09.019702 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8241e3f-f5cd-4428-b0c8-d4b0d536a36d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:48:09.021129 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8241e3f-f5cd-4428-b0c8-d4b0d536a36d admin admin] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:09.029925 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8241e3f-f5cd-4428-b0c8-d4b0d536a36d admin admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:09.030196 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8241e3f-f5cd-4428-b0c8-d4b0d536a36d admin admin] PATH_INFO: `/v3/roles` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:09.031816 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8241e3f-f5cd-4428-b0c8-d4b0d536a36d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:48:09.032074 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.notifications [None req-2872b18d-a43e-4947-8689-a33db22fcf14 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f8bffa6c63644be1b1087916e000999a'} {{(pid=88114) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:48:09.050656 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8241e3f-f5cd-4428-b0c8-d4b0d536a36d admin admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:48:09.054015 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.notifications [None req-2872b18d-a43e-4947-8689-a33db22fcf14 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f8bffa6c63644be1b1087916e000999a'} {{(pid=88114) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:48:09.060855 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.notifications [None req-2872b18d-a43e-4947-8689-a33db22fcf14 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f8bffa6c63644be1b1087916e000999a'} {{(pid=88114) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:48:09.090714 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 875/1751] 10.222.0.13 () {58 vars in 1122 bytes} [Wed Aug 31 09:48:08 2022] GET /identity/v3/roles => generated 1420 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 09:48:09.096380 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8241e3f-f5cd-4428-b0c8-d4b0d536a36d admin admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:48:09.103946 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.fernet_utils [None req-f8241e3f-f5cd-4428-b0c8-d4b0d536a36d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:48:09.138776 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=100 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:48:09.167295 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8241e3f-f5cd-4428-b0c8-d4b0d536a36d admin admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:48:09.174200 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e54c1e1-909d-4676-9f2a-94916e44b06e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:48:09.174879 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e54c1e1-909d-4676-9f2a-94916e44b06e admin admin] REQUEST_METHOD: `DELETE` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:09.175186 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e54c1e1-909d-4676-9f2a-94916e44b06e admin admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:09.175475 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e54c1e1-909d-4676-9f2a-94916e44b06e admin admin] PATH_INFO: `/v3/projects/2cbde0bbab85470d870244ff1cadffd6` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:09.176888 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e54c1e1-909d-4676-9f2a-94916e44b06e admin admin] RBAC: Authorizing `identity:delete_project(project_id=2cbde0bbab85470d870244ff1cadffd6)` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:48:09.179109 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e54c1e1-909d-4676-9f2a-94916e44b06e admin admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:48:09.187135 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.notifications [None req-2872b18d-a43e-4947-8689-a33db22fcf14 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f8bffa6c63644be1b1087916e000999a'} {{(pid=88114) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:48:09.237479 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 877/1752] 10.222.0.13 () {58 vars in 1224 bytes} [Wed Aug 31 09:48:08 2022] DELETE /identity/v3/users/f8bffa6c63644be1b1087916e000999a => generated 0 bytes in 512 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:48:09.244990 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2872b18d-a43e-4947-8689-a33db22fcf14 admin admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:48:09.257661 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.fernet_utils [None req-2872b18d-a43e-4947-8689-a33db22fcf14 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88114) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:48:09.347321 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.notifications [None req-8e54c1e1-909d-4676-9f2a-94916e44b06e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2cbde0bbab85470d870244ff1cadffd6', 'request_id': 'req-8e54c1e1-909d-4676-9f2a-94916e44b06e'} {{(pid=88113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:48:09.378697 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.notifications [None req-8e54c1e1-909d-4676-9f2a-94916e44b06e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2cbde0bbab85470d870244ff1cadffd6', 'request_id': 'req-8e54c1e1-909d-4676-9f2a-94916e44b06e'} {{(pid=88113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:48:09.407609 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.notifications [None req-8e54c1e1-909d-4676-9f2a-94916e44b06e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2cbde0bbab85470d870244ff1cadffd6', 'request_id': 'req-8e54c1e1-909d-4676-9f2a-94916e44b06e'} {{(pid=88113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:48:09.477245 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.notifications [None req-8e54c1e1-909d-4676-9f2a-94916e44b06e admin admin] The token cache is being invalidate because project 2cbde0bbab85470d870244ff1cadffd6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88113) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 09:48:09.479652 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.notifications [None req-8e54c1e1-909d-4676-9f2a-94916e44b06e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:48:09.482746 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.notifications [None req-8e54c1e1-909d-4676-9f2a-94916e44b06e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:48:09.498865 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 876/1753] 10.222.0.13 () {58 vars in 1233 bytes} [Wed Aug 31 09:48:09 2022] DELETE /identity/v3/projects/2cbde0bbab85470d870244ff1cadffd6 => generated 0 bytes in 406 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:48:09.514792 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c73942-2bd1-471c-b456-02823252fd9b None None] REQUEST_METHOD: `POST` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:09.518591 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c73942-2bd1-471c-b456-02823252fd9b None None] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:09.519615 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c73942-2bd1-471c-b456-02823252fd9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:09.695488 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.auth.core [None req-b3c73942-2bd1-471c-b456-02823252fd9b None None] MFA Rules not processed for user `fd836cf590a145df9acf19cf3ae2ac56`. Rule list: `[]` (Enabled: `True`). {{(pid=88113) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:48:09.702255 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.fernet_utils [None req-b3c73942-2bd1-471c-b456-02823252fd9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:48:09.757665 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2872b18d-a43e-4947-8689-a33db22fcf14 admin admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:48:10.026393 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 877/1754] 10.222.0.13 () {58 vars in 958 bytes} [Wed Aug 31 09:48:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 525 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:48:10.026944 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c73942-2bd1-471c-b456-02823252fd9b None None] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:48:10.031935 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.fernet_utils [None req-b3c73942-2bd1-471c-b456-02823252fd9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:48:10.039922 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eaaae8e-dca6-43b7-b628-9d65d244e959 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:48:10.045884 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eaaae8e-dca6-43b7-b628-9d65d244e959 admin admin] REQUEST_METHOD: `PUT` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:10.046434 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eaaae8e-dca6-43b7-b628-9d65d244e959 admin admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:10.047235 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eaaae8e-dca6-43b7-b628-9d65d244e959 admin admin] PATH_INFO: `/v3/projects/6f2fae641f6c4f8188a8f8828248936f/users/68b68dcf9382455a926ae2be6c0f5279/roles/6c99f5d9f1f243eaa134d8540f21095d` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:10.058944 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c73942-2bd1-471c-b456-02823252fd9b None None] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:48:10.085890 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a55f5b0f-1aad-45a9-a3e5-e5b7052ec6b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:48:10.086555 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55f5b0f-1aad-45a9-a3e5-e5b7052ec6b3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:10.086972 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55f5b0f-1aad-45a9-a3e5-e5b7052ec6b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:10.087098 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55f5b0f-1aad-45a9-a3e5-e5b7052ec6b3 admin admin] PATH_INFO: `/v3/users/989378c95b6c440e9a3a6f2ed6b78478` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:10.090047 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a55f5b0f-1aad-45a9-a3e5-e5b7052ec6b3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=989378c95b6c440e9a3a6f2ed6b78478)` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:48:10.092116 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a55f5b0f-1aad-45a9-a3e5-e5b7052ec6b3 admin admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:48:10.131474 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eaaae8e-dca6-43b7-b628-9d65d244e959 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6f2fae641f6c4f8188a8f8828248936f, user_id=68b68dcf9382455a926ae2be6c0f5279, role_id=6c99f5d9f1f243eaa134d8540f21095d)` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:48:10.139503 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eaaae8e-dca6-43b7-b628-9d65d244e959 admin admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:48:10.205424 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 878/1755] 10.222.0.13 () {60 vars in 1483 bytes} [Wed Aug 31 09:48:09 2022] PUT /identity/v3/projects/6f2fae641f6c4f8188a8f8828248936f/users/68b68dcf9382455a926ae2be6c0f5279/roles/6c99f5d9f1f243eaa134d8540f21095d => generated 0 bytes in 968 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:48:10.215263 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.notifications [None req-a55f5b0f-1aad-45a9-a3e5-e5b7052ec6b3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '989378c95b6c440e9a3a6f2ed6b78478'} {{(pid=88113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:48:10.222769 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea32c57-c9f3-48f0-95a5-9ead4145d08a None None] REQUEST_METHOD: `POST` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:10.228785 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea32c57-c9f3-48f0-95a5-9ead4145d08a None None] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:10.228785 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea32c57-c9f3-48f0-95a5-9ead4145d08a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:10.237604 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.notifications [None req-a55f5b0f-1aad-45a9-a3e5-e5b7052ec6b3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '989378c95b6c440e9a3a6f2ed6b78478'} {{(pid=88113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:48:10.299113 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.auth.core [None req-cea32c57-c9f3-48f0-95a5-9ead4145d08a None None] MFA Rules not processed for user `4961545b43f74fdb87d01b78600d54ed`. Rule list: `[]` (Enabled: `True`). {{(pid=88114) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:48:10.300462 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.fernet_utils [None req-cea32c57-c9f3-48f0-95a5-9ead4145d08a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88114) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:48:10.310112 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.notifications [None req-a55f5b0f-1aad-45a9-a3e5-e5b7052ec6b3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '989378c95b6c440e9a3a6f2ed6b78478'} {{(pid=88113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:48:10.315097 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.notifications [None req-a55f5b0f-1aad-45a9-a3e5-e5b7052ec6b3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '989378c95b6c440e9a3a6f2ed6b78478'} {{(pid=88113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:48:10.347683 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.notifications [None req-a55f5b0f-1aad-45a9-a3e5-e5b7052ec6b3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '989378c95b6c440e9a3a6f2ed6b78478'} {{(pid=88113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:48:10.354408 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:48:10.390846 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 878/1756] 10.222.0.13 () {58 vars in 1224 bytes} [Wed Aug 31 09:48:10 2022] DELETE /identity/v3/users/989378c95b6c440e9a3a6f2ed6b78478 => generated 0 bytes in 360 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:48:10.391990 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a55f5b0f-1aad-45a9-a3e5-e5b7052ec6b3 admin admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:48:10.394707 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.fernet_utils [None req-a55f5b0f-1aad-45a9-a3e5-e5b7052ec6b3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:48:10.418018 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=93,keystone:DELETE=5,keystone:INSERT=2 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:48:10.563306 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 879/1757] 10.222.0.13 () {58 vars in 958 bytes} [Wed Aug 31 09:48:10 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 357 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:48:10.564804 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f58150-c39c-46c3-bbda-bfe2e618c56b None None] REQUEST_METHOD: `POST` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:10.573709 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f58150-c39c-46c3-bbda-bfe2e618c56b None None] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:10.573961 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f58150-c39c-46c3-bbda-bfe2e618c56b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:10.737747 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.auth.core [None req-06f58150-c39c-46c3-bbda-bfe2e618c56b None None] MFA Rules not processed for user `68b68dcf9382455a926ae2be6c0f5279`. Rule list: `[]` (Enabled: `True`). {{(pid=88114) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:48:10.739478 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.fernet_utils [None req-06f58150-c39c-46c3-bbda-bfe2e618c56b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88114) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:48:10.741662 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a55f5b0f-1aad-45a9-a3e5-e5b7052ec6b3 admin admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:48:10.774920 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfb2c330-032b-453b-9a41-105d9d67cc1d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9bdaacd597442c2bcde2c555f2cc140', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '49548eb44d62415582af820ab66decee', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:48:10.777869 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb2c330-032b-453b-9a41-105d9d67cc1d service neutron] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:10.782151 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb2c330-032b-453b-9a41-105d9d67cc1d service neutron] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:10.782950 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb2c330-032b-453b-9a41-105d9d67cc1d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:10.798745 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.fernet_utils [None req-cfb2c330-032b-453b-9a41-105d9d67cc1d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:48:11.103910 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 880/1758] 10.222.0.13 () {58 vars in 958 bytes} [Wed Aug 31 09:48:10 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 540 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:48:11.110891 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f58150-c39c-46c3-bbda-bfe2e618c56b None None] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:48:11.120994 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.fernet_utils [None req-06f58150-c39c-46c3-bbda-bfe2e618c56b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88114) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:48:11.219406 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfb2c330-032b-453b-9a41-105d9d67cc1d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:48:11.221338 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfb2c330-032b-453b-9a41-105d9d67cc1d service neutron] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:48:11.236137 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 879/1759] 10.222.0.13 () {60 vars in 1365 bytes} [Wed Aug 31 09:48:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 845 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:48:11.237113 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfb2c330-032b-453b-9a41-105d9d67cc1d service neutron] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:48:11.242548 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.fernet_utils [None req-cfb2c330-032b-453b-9a41-105d9d67cc1d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:48:11.287037 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfb2c330-032b-453b-9a41-105d9d67cc1d service neutron] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:48:11.302894 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf177a03-c475-4f5d-8de5-8a1f70bce85f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:48:11.303573 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf177a03-c475-4f5d-8de5-8a1f70bce85f admin admin] REQUEST_METHOD: `POST` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:11.303953 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf177a03-c475-4f5d-8de5-8a1f70bce85f admin admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:11.307337 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf177a03-c475-4f5d-8de5-8a1f70bce85f admin admin] PATH_INFO: `/v3/projects` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:11.310019 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf177a03-c475-4f5d-8de5-8a1f70bce85f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:48:11.317958 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf177a03-c475-4f5d-8de5-8a1f70bce85f admin admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:48:11.409658 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 880/1760] 10.222.0.13 () {60 vars in 1153 bytes} [Wed Aug 31 09:48:11 2022] POST /identity/v3/projects => generated 367 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 09:48:11.415706 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf177a03-c475-4f5d-8de5-8a1f70bce85f admin admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:48:11.461848 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f58150-c39c-46c3-bbda-bfe2e618c56b None None] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:48:11.731398 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f432f36d-4b0f-44bc-9377-5a1bc6df6cd1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:48:11.731923 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f432f36d-4b0f-44bc-9377-5a1bc6df6cd1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:11.732205 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f432f36d-4b0f-44bc-9377-5a1bc6df6cd1 admin admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:11.732479 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f432f36d-4b0f-44bc-9377-5a1bc6df6cd1 admin admin] PATH_INFO: `/v3/users/bf564c77b7d249f9b050b2055b76e362` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:11.736745 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f432f36d-4b0f-44bc-9377-5a1bc6df6cd1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bf564c77b7d249f9b050b2055b76e362)` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:48:11.748903 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf177a03-c475-4f5d-8de5-8a1f70bce85f admin admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:48:11.749138 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f432f36d-4b0f-44bc-9377-5a1bc6df6cd1 admin admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:48:11.759012 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce3ba823-5e01-4c0b-bf89-b40c31e374a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:48:11.759959 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3ba823-5e01-4c0b-bf89-b40c31e374a8 admin admin] REQUEST_METHOD: `POST` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:11.760705 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3ba823-5e01-4c0b-bf89-b40c31e374a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:11.761228 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3ba823-5e01-4c0b-bf89-b40c31e374a8 admin admin] PATH_INFO: `/v3/users` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:11.762450 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce3ba823-5e01-4c0b-bf89-b40c31e374a8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:48:11.767631 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce3ba823-5e01-4c0b-bf89-b40c31e374a8 admin admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:48:11.816212 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:48:11.846809 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 881/1761] 10.222.0.13 () {60 vars in 1144 bytes} [Wed Aug 31 09:48:11 2022] POST /identity/v3/users => generated 290 bytes in 432 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 09:48:11.852137 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce3ba823-5e01-4c0b-bf89-b40c31e374a8 admin admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:48:11.905236 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.notifications [None req-f432f36d-4b0f-44bc-9377-5a1bc6df6cd1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bf564c77b7d249f9b050b2055b76e362'} {{(pid=88114) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:48:11.906146 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce3ba823-5e01-4c0b-bf89-b40c31e374a8 admin admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:48:11.925696 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de1a6280-64fc-4bd2-bf28-453f58cc4045 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:48:11.932195 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1a6280-64fc-4bd2-bf28-453f58cc4045 admin admin] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:11.933025 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1a6280-64fc-4bd2-bf28-453f58cc4045 admin admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:11.934541 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1a6280-64fc-4bd2-bf28-453f58cc4045 admin admin] PATH_INFO: `/v3/roles` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:11.941569 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de1a6280-64fc-4bd2-bf28-453f58cc4045 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:48:11.948302 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.notifications [None req-f432f36d-4b0f-44bc-9377-5a1bc6df6cd1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bf564c77b7d249f9b050b2055b76e362'} {{(pid=88114) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:48:11.952333 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de1a6280-64fc-4bd2-bf28-453f58cc4045 admin admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:48:11.992593 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.notifications [None req-f432f36d-4b0f-44bc-9377-5a1bc6df6cd1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bf564c77b7d249f9b050b2055b76e362'} {{(pid=88114) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:48:11.994689 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.notifications [None req-f432f36d-4b0f-44bc-9377-5a1bc6df6cd1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bf564c77b7d249f9b050b2055b76e362'} {{(pid=88114) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:48:12.028461 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 882/1762] 10.222.0.13 () {58 vars in 1122 bytes} [Wed Aug 31 09:48:11 2022] GET /identity/v3/roles => generated 1420 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 09:48:12.036218 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de1a6280-64fc-4bd2-bf28-453f58cc4045 admin admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:48:12.040300 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.fernet_utils [None req-de1a6280-64fc-4bd2-bf28-453f58cc4045 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:48:12.075030 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.notifications [None req-f432f36d-4b0f-44bc-9377-5a1bc6df6cd1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bf564c77b7d249f9b050b2055b76e362'} {{(pid=88114) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:48:12.077237 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de1a6280-64fc-4bd2-bf28-453f58cc4045 admin admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:48:12.088828 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f8c508-7dae-4224-bf68-e7b0ef8f914a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:48:12.091605 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 881/1763] 10.222.0.13 () {58 vars in 1224 bytes} [Wed Aug 31 09:48:11 2022] DELETE /identity/v3/users/bf564c77b7d249f9b050b2055b76e362 => generated 0 bytes in 986 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:48:12.107464 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f8c508-7dae-4224-bf68-e7b0ef8f914a admin admin] REQUEST_METHOD: `PUT` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:12.108077 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332ecf4f-8df8-4101-a97f-ab6496ab3c2b None None] REQUEST_METHOD: `POST` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:12.108793 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332ecf4f-8df8-4101-a97f-ab6496ab3c2b None None] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:12.108793 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332ecf4f-8df8-4101-a97f-ab6496ab3c2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:12.109031 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f8c508-7dae-4224-bf68-e7b0ef8f914a admin admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:12.120768 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f8c508-7dae-4224-bf68-e7b0ef8f914a admin admin] PATH_INFO: `/v3/projects/96994ecf01df44a398f67e4ada14f7f7/users/086fbacfad4a42ec9be55fcb49f80d9d/roles/5d4ee63a3add47fc9a6b8b16d39e0a1f` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:12.129072 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:48:12.181318 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90f8c508-7dae-4224-bf68-e7b0ef8f914a admin admin] RBAC: Authorizing `identity:create_grant(project_id=96994ecf01df44a398f67e4ada14f7f7, user_id=086fbacfad4a42ec9be55fcb49f80d9d, role_id=5d4ee63a3add47fc9a6b8b16d39e0a1f)` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:48:12.184252 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90f8c508-7dae-4224-bf68-e7b0ef8f914a admin admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:48:12.196380 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.auth.core [None req-332ecf4f-8df8-4101-a97f-ab6496ab3c2b None None] MFA Rules not processed for user `fd836cf590a145df9acf19cf3ae2ac56`. Rule list: `[]` (Enabled: `True`). {{(pid=88114) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:48:12.197916 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.fernet_utils [None req-332ecf4f-8df8-4101-a97f-ab6496ab3c2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88114) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:48:12.253744 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 883/1764] 10.222.0.13 () {60 vars in 1483 bytes} [Wed Aug 31 09:48:12 2022] PUT /identity/v3/projects/96994ecf01df44a398f67e4ada14f7f7/users/086fbacfad4a42ec9be55fcb49f80d9d/roles/5d4ee63a3add47fc9a6b8b16d39e0a1f => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:48:12.260140 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f8c508-7dae-4224-bf68-e7b0ef8f914a admin admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:48:12.463706 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 882/1765] 10.222.0.13 () {58 vars in 958 bytes} [Wed Aug 31 09:48:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 352 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:48:12.619346 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eefd3631-11ed-4e8d-89cf-dee2f0e5c06a None None] REQUEST_METHOD: `POST` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:12.619984 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eefd3631-11ed-4e8d-89cf-dee2f0e5c06a None None] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:12.620458 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eefd3631-11ed-4e8d-89cf-dee2f0e5c06a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:12.655309 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f8c508-7dae-4224-bf68-e7b0ef8f914a admin admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:48:12.672807 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.auth.core [None req-eefd3631-11ed-4e8d-89cf-dee2f0e5c06a None None] MFA Rules not processed for user `a9bdaacd597442c2bcde2c555f2cc140`. Rule list: `[]` (Enabled: `True`). {{(pid=88114) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:48:12.676788 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.fernet_utils [None req-eefd3631-11ed-4e8d-89cf-dee2f0e5c06a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88114) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:48:12.681961 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-755caa24-bfa7-4005-a444-09a50d9e5aff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:48:12.684622 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755caa24-bfa7-4005-a444-09a50d9e5aff admin admin] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:12.685103 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755caa24-bfa7-4005-a444-09a50d9e5aff admin admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:12.685611 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755caa24-bfa7-4005-a444-09a50d9e5aff admin admin] PATH_INFO: `/v3/roles` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:12.691058 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-755caa24-bfa7-4005-a444-09a50d9e5aff admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:48:12.700170 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-755caa24-bfa7-4005-a444-09a50d9e5aff admin admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:48:12.763234 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 884/1766] 10.222.0.13 () {58 vars in 1122 bytes} [Wed Aug 31 09:48:12 2022] GET /identity/v3/roles => generated 1420 bytes in 506 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 09:48:12.772788 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-755caa24-bfa7-4005-a444-09a50d9e5aff admin admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:48:12.788069 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.fernet_utils [None req-755caa24-bfa7-4005-a444-09a50d9e5aff admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:48:12.854647 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 883/1767] 10.222.0.64 () {58 vars in 1015 bytes} [Wed Aug 31 09:48:12 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 237 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 09:48:12.857697 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eefd3631-11ed-4e8d-89cf-dee2f0e5c06a None None] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:48:12.864750 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.fernet_utils [None req-eefd3631-11ed-4e8d-89cf-dee2f0e5c06a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88114) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:48:12.905880 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eefd3631-11ed-4e8d-89cf-dee2f0e5c06a None None] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:48:12.929870 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd31314-09f1-461d-bc17-eba49e13ff7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:48:12.930674 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd31314-09f1-461d-bc17-eba49e13ff7e admin admin] REQUEST_METHOD: `DELETE` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:12.932356 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd31314-09f1-461d-bc17-eba49e13ff7e admin admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:12.932799 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd31314-09f1-461d-bc17-eba49e13ff7e admin admin] PATH_INFO: `/v3/users/20f156894e214714bf67b37ff94cd538` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:12.934542 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fd31314-09f1-461d-bc17-eba49e13ff7e admin admin] RBAC: Authorizing `identity:delete_user(user_id=20f156894e214714bf67b37ff94cd538)` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:48:12.936628 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fd31314-09f1-461d-bc17-eba49e13ff7e admin admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:48:13.030172 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-755caa24-bfa7-4005-a444-09a50d9e5aff admin admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:48:13.040217 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2920e2be-f89e-4235-95da-96c4e6edcb80 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9bdaacd597442c2bcde2c555f2cc140', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '49548eb44d62415582af820ab66decee', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:48:13.048506 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2920e2be-f89e-4235-95da-96c4e6edcb80 service neutron] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:13.053014 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2920e2be-f89e-4235-95da-96c4e6edcb80 service neutron] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:13.053599 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2920e2be-f89e-4235-95da-96c4e6edcb80 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:13.069200 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:48:13.078981 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.notifications [None req-4fd31314-09f1-461d-bc17-eba49e13ff7e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '20f156894e214714bf67b37ff94cd538'} {{(pid=88114) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:48:13.100117 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2920e2be-f89e-4235-95da-96c4e6edcb80 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:48:13.102919 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.notifications [None req-4fd31314-09f1-461d-bc17-eba49e13ff7e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '20f156894e214714bf67b37ff94cd538'} {{(pid=88114) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:48:13.104289 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2920e2be-f89e-4235-95da-96c4e6edcb80 service neutron] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:48:13.130485 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 885/1768] 10.222.0.13 () {60 vars in 1365 bytes} [Wed Aug 31 09:48:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 366 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:48:13.132218 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2920e2be-f89e-4235-95da-96c4e6edcb80 service neutron] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:48:13.133370 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.notifications [None req-4fd31314-09f1-461d-bc17-eba49e13ff7e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '20f156894e214714bf67b37ff94cd538'} {{(pid=88114) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:48:13.134943 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.notifications [None req-4fd31314-09f1-461d-bc17-eba49e13ff7e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '20f156894e214714bf67b37ff94cd538'} {{(pid=88114) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:48:13.157517 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2920e2be-f89e-4235-95da-96c4e6edcb80 service neutron] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:48:13.158863 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.notifications [None req-4fd31314-09f1-461d-bc17-eba49e13ff7e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '20f156894e214714bf67b37ff94cd538'} {{(pid=88114) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:48:13.171600 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 884/1769] 10.222.0.13 () {58 vars in 1224 bytes} [Wed Aug 31 09:48:12 2022] DELETE /identity/v3/users/20f156894e214714bf67b37ff94cd538 => generated 0 bytes in 316 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:48:13.174944 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd31314-09f1-461d-bc17-eba49e13ff7e admin admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:48:13.176841 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.fernet_utils [None req-4fd31314-09f1-461d-bc17-eba49e13ff7e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88114) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:48:13.196890 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a01839a-f261-4aab-984d-7b223c2a8ee1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:48:13.203837 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a01839a-f261-4aab-984d-7b223c2a8ee1 admin admin] REQUEST_METHOD: `PUT` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:13.208881 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a01839a-f261-4aab-984d-7b223c2a8ee1 admin admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:13.218264 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a01839a-f261-4aab-984d-7b223c2a8ee1 admin admin] PATH_INFO: `/v3/projects/96994ecf01df44a398f67e4ada14f7f7/users/086fbacfad4a42ec9be55fcb49f80d9d/roles/5d4ee63a3add47fc9a6b8b16d39e0a1f` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:13.229418 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a01839a-f261-4aab-984d-7b223c2a8ee1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=96994ecf01df44a398f67e4ada14f7f7, user_id=086fbacfad4a42ec9be55fcb49f80d9d, role_id=5d4ee63a3add47fc9a6b8b16d39e0a1f)` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:48:13.245819 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a01839a-f261-4aab-984d-7b223c2a8ee1 admin admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:48:13.375412 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 886/1770] 10.222.0.13 () {60 vars in 1483 bytes} [Wed Aug 31 09:48:13 2022] PUT /identity/v3/projects/96994ecf01df44a398f67e4ada14f7f7/users/086fbacfad4a42ec9be55fcb49f80d9d/roles/5d4ee63a3add47fc9a6b8b16d39e0a1f => generated 0 bytes in 243 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:48:13.394845 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e1f5aa-8468-4528-b12e-fce637990d9c None None] REQUEST_METHOD: `POST` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:13.396627 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e1f5aa-8468-4528-b12e-fce637990d9c None None] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:13.405246 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e1f5aa-8468-4528-b12e-fce637990d9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:13.428151 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd31314-09f1-461d-bc17-eba49e13ff7e admin admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:48:13.586225 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.auth.core [None req-f1e1f5aa-8468-4528-b12e-fce637990d9c None None] MFA Rules not processed for user `086fbacfad4a42ec9be55fcb49f80d9d`. Rule list: `[]` (Enabled: `True`). {{(pid=88113) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:48:13.587424 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.fernet_utils [None req-f1e1f5aa-8468-4528-b12e-fce637990d9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:48:13.678488 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e1d53e6-c5b0-4b4d-a1b2-fc25164d9707 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9bdaacd597442c2bcde2c555f2cc140', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '49548eb44d62415582af820ab66decee', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:48:13.679135 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1d53e6-c5b0-4b4d-a1b2-fc25164d9707 service neutron] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:13.679416 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1d53e6-c5b0-4b4d-a1b2-fc25164d9707 service neutron] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:13.679679 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1d53e6-c5b0-4b4d-a1b2-fc25164d9707 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:13.686532 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.fernet_utils [None req-4e1d53e6-c5b0-4b4d-a1b2-fc25164d9707 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88114) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:48:13.719642 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e1d53e6-c5b0-4b4d-a1b2-fc25164d9707 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:48:13.721116 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e1d53e6-c5b0-4b4d-a1b2-fc25164d9707 service neutron] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:48:13.750339 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 885/1771] 10.222.0.13 () {60 vars in 1365 bytes} [Wed Aug 31 09:48:13 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 578 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:48:13.806606 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:48:13.865007 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 887/1772] 10.222.0.13 () {58 vars in 958 bytes} [Wed Aug 31 09:48:13 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 472 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:48:13.896158 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e1d53e6-c5b0-4b4d-a1b2-fc25164d9707 service neutron] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:48:13.897684 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.fernet_utils [None req-4e1d53e6-c5b0-4b4d-a1b2-fc25164d9707 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88114) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:48:14.097225 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e1d53e6-c5b0-4b4d-a1b2-fc25164d9707 service neutron] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:48:14.119300 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72685f3f-8601-4a7e-8f4d-c09b0fff6a7e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d52b69c8e94441dacfcff944326f169', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '49548eb44d62415582af820ab66decee', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:48:14.120326 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72685f3f-8601-4a7e-8f4d-c09b0fff6a7e service cinder] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:14.120628 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72685f3f-8601-4a7e-8f4d-c09b0fff6a7e service cinder] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:14.120902 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72685f3f-8601-4a7e-8f4d-c09b0fff6a7e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:14.154912 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72685f3f-8601-4a7e-8f4d-c09b0fff6a7e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:48:14.156125 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72685f3f-8601-4a7e-8f4d-c09b0fff6a7e service cinder] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:48:14.197757 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 886/1773] 10.222.0.13 () {60 vars in 1365 bytes} [Wed Aug 31 09:48:13 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 303 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 09:48:14.666014 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e1f5aa-8468-4528-b12e-fce637990d9c None None] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:48:14.685284 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e1f5aa-8468-4528-b12e-fce637990d9c None None] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:48:14.720205 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e6ebe53-4652-48ae-8845-fb1de244c50a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d52b69c8e94441dacfcff944326f169', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '49548eb44d62415582af820ab66decee', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:48:14.722952 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6ebe53-4652-48ae-8845-fb1de244c50a service cinder] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:14.725311 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6ebe53-4652-48ae-8845-fb1de244c50a service cinder] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:14.728610 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6ebe53-4652-48ae-8845-fb1de244c50a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:14.733242 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.fernet_utils [None req-7e6ebe53-4652-48ae-8845-fb1de244c50a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:48:15.018900 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e6ebe53-4652-48ae-8845-fb1de244c50a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:48:15.024972 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e6ebe53-4652-48ae-8845-fb1de244c50a service cinder] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:48:15.095071 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 888/1774] 10.222.0.13 () {60 vars in 1365 bytes} [Wed Aug 31 09:48:14 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 430 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:48:15.315230 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72685f3f-8601-4a7e-8f4d-c09b0fff6a7e service cinder] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:48:15.320150 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.fernet_utils [None req-72685f3f-8601-4a7e-8f4d-c09b0fff6a7e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88114) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:48:15.457723 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=96,keystone:INSERT=1,keystone:DELETE=3 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:48:15.586248 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72685f3f-8601-4a7e-8f4d-c09b0fff6a7e service cinder] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:48:15.605480 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65fbcbcc-d1ba-43a4-a61e-5cfe7774415d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:48:15.606166 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65fbcbcc-d1ba-43a4-a61e-5cfe7774415d admin admin] REQUEST_METHOD: `DELETE` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:15.606543 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65fbcbcc-d1ba-43a4-a61e-5cfe7774415d admin admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:15.606826 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65fbcbcc-d1ba-43a4-a61e-5cfe7774415d admin admin] PATH_INFO: `/v3/projects/0c7662a0f481410fa31c1ec1b888dd52` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:15.608457 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65fbcbcc-d1ba-43a4-a61e-5cfe7774415d admin admin] RBAC: Authorizing `identity:delete_project(project_id=0c7662a0f481410fa31c1ec1b888dd52)` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:48:15.617944 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65fbcbcc-d1ba-43a4-a61e-5cfe7774415d admin admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:48:15.861312 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.notifications [None req-65fbcbcc-d1ba-43a4-a61e-5cfe7774415d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0c7662a0f481410fa31c1ec1b888dd52', 'request_id': 'req-65fbcbcc-d1ba-43a4-a61e-5cfe7774415d'} {{(pid=88114) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:48:15.882504 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.notifications [None req-65fbcbcc-d1ba-43a4-a61e-5cfe7774415d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0c7662a0f481410fa31c1ec1b888dd52', 'request_id': 'req-65fbcbcc-d1ba-43a4-a61e-5cfe7774415d'} {{(pid=88114) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:48:15.938337 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.notifications [None req-65fbcbcc-d1ba-43a4-a61e-5cfe7774415d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0c7662a0f481410fa31c1ec1b888dd52', 'request_id': 'req-65fbcbcc-d1ba-43a4-a61e-5cfe7774415d'} {{(pid=88114) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:48:15.968995 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.notifications [None req-65fbcbcc-d1ba-43a4-a61e-5cfe7774415d admin admin] The token cache is being invalidate because project 0c7662a0f481410fa31c1ec1b888dd52 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88114) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 09:48:15.969420 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.notifications [None req-65fbcbcc-d1ba-43a4-a61e-5cfe7774415d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88114) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:48:15.970336 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.notifications [None req-65fbcbcc-d1ba-43a4-a61e-5cfe7774415d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88114) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:48:15.972106 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 887/1775] 10.222.0.13 () {58 vars in 1233 bytes} [Wed Aug 31 09:48:15 2022] DELETE /identity/v3/projects/0c7662a0f481410fa31c1ec1b888dd52 => generated 0 bytes in 657 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 09:48:15.978951 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b8e225-14b0-4bd5-b3ce-6dd5673e0c73 None None] REQUEST_METHOD: `POST` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:15.979404 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b8e225-14b0-4bd5-b3ce-6dd5673e0c73 None None] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:15.979634 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b8e225-14b0-4bd5-b3ce-6dd5673e0c73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:16.029890 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.auth.core [None req-08b8e225-14b0-4bd5-b3ce-6dd5673e0c73 None None] MFA Rules not processed for user `fd836cf590a145df9acf19cf3ae2ac56`. Rule list: `[]` (Enabled: `True`). {{(pid=88113) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:48:16.032283 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.fernet_utils [None req-08b8e225-14b0-4bd5-b3ce-6dd5673e0c73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:48:16.144375 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 889/1776] 10.222.0.13 () {58 vars in 958 bytes} [Wed Aug 31 09:48:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:48:16.146871 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4fb478-bab4-4d3d-8b69-d5deb2213517 None None] REQUEST_METHOD: `POST` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:16.147436 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4fb478-bab4-4d3d-8b69-d5deb2213517 None None] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:16.147944 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4fb478-bab4-4d3d-8b69-d5deb2213517 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:16.179752 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.auth.core [None req-8e4fb478-bab4-4d3d-8b69-d5deb2213517 None None] MFA Rules not processed for user `fd836cf590a145df9acf19cf3ae2ac56`. Rule list: `[]` (Enabled: `True`). {{(pid=88114) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:48:16.181552 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.fernet_utils [None req-8e4fb478-bab4-4d3d-8b69-d5deb2213517 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88114) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:48:16.194392 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 888/1777] 10.222.0.13 () {58 vars in 958 bytes} [Wed Aug 31 09:48:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:48:16.214965 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b8e225-14b0-4bd5-b3ce-6dd5673e0c73 None None] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:48:16.241671 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b8e225-14b0-4bd5-b3ce-6dd5673e0c73 None None] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:48:16.262663 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce567d5a-f67d-4506-82bf-e02427ce5ee9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:48:16.263601 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce567d5a-f67d-4506-82bf-e02427ce5ee9 admin admin] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:16.263969 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce567d5a-f67d-4506-82bf-e02427ce5ee9 admin admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:16.264233 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce567d5a-f67d-4506-82bf-e02427ce5ee9 admin admin] PATH_INFO: `/v3/domains` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:16.265115 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce567d5a-f67d-4506-82bf-e02427ce5ee9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88113) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:48:16.266517 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce567d5a-f67d-4506-82bf-e02427ce5ee9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:48:16.271421 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce567d5a-f67d-4506-82bf-e02427ce5ee9 admin admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:48:16.313565 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 890/1778] 10.222.0.13 () {58 vars in 1153 bytes} [Wed Aug 31 09:48:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 09:48:16.319802 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e4fb478-bab4-4d3d-8b69-d5deb2213517 None None] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:48:16.368704 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e4fb478-bab4-4d3d-8b69-d5deb2213517 None None] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:48:16.392549 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf050e6-c99b-40f9-8c5c-f98e113dadae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:48:16.400551 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf050e6-c99b-40f9-8c5c-f98e113dadae admin admin] REQUEST_METHOD: `POST` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:16.401754 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf050e6-c99b-40f9-8c5c-f98e113dadae admin admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:16.402038 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf050e6-c99b-40f9-8c5c-f98e113dadae admin admin] PATH_INFO: `/v3/projects` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:16.404531 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcf050e6-c99b-40f9-8c5c-f98e113dadae admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:48:16.420415 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcf050e6-c99b-40f9-8c5c-f98e113dadae admin admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:48:16.502046 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 889/1779] 10.222.0.13 () {60 vars in 1153 bytes} [Wed Aug 31 09:48:16 2022] POST /identity/v3/projects => generated 381 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 09:48:16.509073 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce567d5a-f67d-4506-82bf-e02427ce5ee9 admin admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:48:16.532682 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce567d5a-f67d-4506-82bf-e02427ce5ee9 admin admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:48:16.706503 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf050e6-c99b-40f9-8c5c-f98e113dadae admin admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:48:16.710619 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.fernet_utils [None req-fcf050e6-c99b-40f9-8c5c-f98e113dadae admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88114) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:48:16.829342 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77080bcb-390a-4635-bd4b-ba905f428eb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:48:16.834466 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77080bcb-390a-4635-bd4b-ba905f428eb6 admin admin] REQUEST_METHOD: `POST` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:16.834962 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77080bcb-390a-4635-bd4b-ba905f428eb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:16.835454 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77080bcb-390a-4635-bd4b-ba905f428eb6 admin admin] PATH_INFO: `/v3/users` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:16.837212 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77080bcb-390a-4635-bd4b-ba905f428eb6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:48:16.853177 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77080bcb-390a-4635-bd4b-ba905f428eb6 admin admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:48:16.916504 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf050e6-c99b-40f9-8c5c-f98e113dadae admin admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:48:16.922872 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f4b8e4-afbd-474b-908b-61c298642526 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:48:16.923695 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f4b8e4-afbd-474b-908b-61c298642526 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:16.924031 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f4b8e4-afbd-474b-908b-61c298642526 admin admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:16.927824 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f4b8e4-afbd-474b-908b-61c298642526 admin admin] PATH_INFO: `/v3/projects/c8c7821da6b84645985cbe9d3f3e0f04` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:16.927824 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2f4b8e4-afbd-474b-908b-61c298642526 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c8c7821da6b84645985cbe9d3f3e0f04)` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:48:16.932988 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2f4b8e4-afbd-474b-908b-61c298642526 admin admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:48:17.026902 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 891/1780] 10.222.0.13 () {60 vars in 1144 bytes} [Wed Aug 31 09:48:16 2022] POST /identity/v3/users => generated 291 bytes in 518 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 09:48:17.028823 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77080bcb-390a-4635-bd4b-ba905f428eb6 admin admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:48:17.045712 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.fernet_utils [None req-77080bcb-390a-4635-bd4b-ba905f428eb6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:48:17.051851 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=89,keystone:DELETE=9,keystone:INSERT=2 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:48:17.108053 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77080bcb-390a-4635-bd4b-ba905f428eb6 admin admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:48:17.128220 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.notifications [None req-f2f4b8e4-afbd-474b-908b-61c298642526 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c8c7821da6b84645985cbe9d3f3e0f04', 'request_id': 'req-f2f4b8e4-afbd-474b-908b-61c298642526'} {{(pid=88114) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:48:17.157905 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.notifications [None req-f2f4b8e4-afbd-474b-908b-61c298642526 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c8c7821da6b84645985cbe9d3f3e0f04', 'request_id': 'req-f2f4b8e4-afbd-474b-908b-61c298642526'} {{(pid=88114) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:48:17.164849 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:48:17.178751 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.notifications [None req-f2f4b8e4-afbd-474b-908b-61c298642526 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c8c7821da6b84645985cbe9d3f3e0f04', 'request_id': 'req-f2f4b8e4-afbd-474b-908b-61c298642526'} {{(pid=88114) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:48:17.206609 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.notifications [None req-f2f4b8e4-afbd-474b-908b-61c298642526 admin admin] The token cache is being invalidate because project c8c7821da6b84645985cbe9d3f3e0f04 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88114) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 09:48:17.207079 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.notifications [None req-f2f4b8e4-afbd-474b-908b-61c298642526 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88114) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:48:17.208015 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.notifications [None req-f2f4b8e4-afbd-474b-908b-61c298642526 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88114) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:48:17.210619 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 890/1781] 10.222.0.13 () {58 vars in 1233 bytes} [Wed Aug 31 09:48:16 2022] DELETE /identity/v3/projects/c8c7821da6b84645985cbe9d3f3e0f04 => generated 0 bytes in 505 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 09:48:17.212227 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f4b8e4-afbd-474b-908b-61c298642526 admin admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:48:17.215375 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.fernet_utils [None req-f2f4b8e4-afbd-474b-908b-61c298642526 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88114) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:48:17.408829 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f4b8e4-afbd-474b-908b-61c298642526 admin admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:48:17.453553 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b11da64b-6c0b-417d-bc56-f9ee702c1d44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:48:17.453553 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11da64b-6c0b-417d-bc56-f9ee702c1d44 admin admin] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:17.453553 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11da64b-6c0b-417d-bc56-f9ee702c1d44 admin admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:17.453553 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11da64b-6c0b-417d-bc56-f9ee702c1d44 admin admin] PATH_INFO: `/v3/roles` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:17.454256 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b11da64b-6c0b-417d-bc56-f9ee702c1d44 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:48:17.461096 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b11da64b-6c0b-417d-bc56-f9ee702c1d44 admin admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:48:17.505327 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 891/1782] 10.222.0.13 () {58 vars in 1122 bytes} [Wed Aug 31 09:48:17 2022] GET /identity/v3/roles => generated 1420 bytes in 294 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 09:48:17.506843 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7520ed31-398d-41f5-bcd3-98ca8fc15331 None None] REQUEST_METHOD: `POST` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:17.507113 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7520ed31-398d-41f5-bcd3-98ca8fc15331 None None] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:17.507387 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7520ed31-398d-41f5-bcd3-98ca8fc15331 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:17.568391 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e9b5ced-b046-4864-821d-1be8cb176acb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:48:17.571504 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9b5ced-b046-4864-821d-1be8cb176acb admin admin] REQUEST_METHOD: `DELETE` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:17.572430 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9b5ced-b046-4864-821d-1be8cb176acb admin admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:17.573910 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9b5ced-b046-4864-821d-1be8cb176acb admin admin] PATH_INFO: `/v3/projects/7274499d018e4e5ca98199f5446a19e0` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:17.577098 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e9b5ced-b046-4864-821d-1be8cb176acb admin admin] RBAC: Authorizing `identity:delete_project(project_id=7274499d018e4e5ca98199f5446a19e0)` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:48:17.583392 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e9b5ced-b046-4864-821d-1be8cb176acb admin admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:48:17.596180 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.auth.core [None req-7520ed31-398d-41f5-bcd3-98ca8fc15331 None None] MFA Rules not processed for user `fd836cf590a145df9acf19cf3ae2ac56`. Rule list: `[]` (Enabled: `True`). {{(pid=88114) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:48:17.615124 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.fernet_utils [None req-7520ed31-398d-41f5-bcd3-98ca8fc15331 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88114) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:48:17.651493 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 892/1783] 10.222.0.13 () {58 vars in 958 bytes} [Wed Aug 31 09:48:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:48:17.657757 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7520ed31-398d-41f5-bcd3-98ca8fc15331 None None] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:48:17.698826 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7520ed31-398d-41f5-bcd3-98ca8fc15331 None None] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:48:17.718532 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ea5fe6-0885-4665-a745-f556496f0461 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:48:17.719168 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ea5fe6-0885-4665-a745-f556496f0461 admin admin] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:17.719474 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ea5fe6-0885-4665-a745-f556496f0461 admin admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:17.719957 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ea5fe6-0885-4665-a745-f556496f0461 admin admin] PATH_INFO: `/v3/roles` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:17.735250 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3ea5fe6-0885-4665-a745-f556496f0461 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:48:17.736948 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3ea5fe6-0885-4665-a745-f556496f0461 admin admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:48:17.812560 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 893/1784] 10.222.0.13 () {58 vars in 1122 bytes} [Wed Aug 31 09:48:17 2022] GET /identity/v3/roles => generated 1420 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 09:48:17.820499 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a170497f-6f5c-4161-9365-4f4ffe626896 None None] REQUEST_METHOD: `POST` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:17.821983 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a170497f-6f5c-4161-9365-4f4ffe626896 None None] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:17.823021 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a170497f-6f5c-4161-9365-4f4ffe626896 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:17.835922 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.notifications [None req-1e9b5ced-b046-4864-821d-1be8cb176acb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7274499d018e4e5ca98199f5446a19e0', 'request_id': 'req-1e9b5ced-b046-4864-821d-1be8cb176acb'} {{(pid=88113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:48:17.868047 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.notifications [None req-1e9b5ced-b046-4864-821d-1be8cb176acb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7274499d018e4e5ca98199f5446a19e0', 'request_id': 'req-1e9b5ced-b046-4864-821d-1be8cb176acb'} {{(pid=88113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:48:17.904521 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.notifications [None req-1e9b5ced-b046-4864-821d-1be8cb176acb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7274499d018e4e5ca98199f5446a19e0', 'request_id': 'req-1e9b5ced-b046-4864-821d-1be8cb176acb'} {{(pid=88113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:48:17.953370 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.auth.core [None req-a170497f-6f5c-4161-9365-4f4ffe626896 None None] MFA Rules not processed for user `fd836cf590a145df9acf19cf3ae2ac56`. Rule list: `[]` (Enabled: `True`). {{(pid=88114) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:48:17.964575 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.fernet_utils [None req-a170497f-6f5c-4161-9365-4f4ffe626896 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88114) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:48:17.995946 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.notifications [None req-1e9b5ced-b046-4864-821d-1be8cb176acb admin admin] The token cache is being invalidate because project 7274499d018e4e5ca98199f5446a19e0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88113) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 09:48:17.999073 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.notifications [None req-1e9b5ced-b046-4864-821d-1be8cb176acb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:48:18.003658 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.notifications [None req-1e9b5ced-b046-4864-821d-1be8cb176acb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:48:18.013491 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 892/1785] 10.222.0.13 () {58 vars in 1233 bytes} [Wed Aug 31 09:48:17 2022] DELETE /identity/v3/projects/7274499d018e4e5ca98199f5446a19e0 => generated 0 bytes in 983 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:48:18.015208 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e9b5ced-b046-4864-821d-1be8cb176acb admin admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:48:18.028596 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.fernet_utils [None req-1e9b5ced-b046-4864-821d-1be8cb176acb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:48:18.500013 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 894/1786] 10.222.0.13 () {58 vars in 958 bytes} [Wed Aug 31 09:48:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 687 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:48:18.502570 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a170497f-6f5c-4161-9365-4f4ffe626896 None None] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:48:18.508568 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.fernet_utils [None req-a170497f-6f5c-4161-9365-4f4ffe626896 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88114) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:48:18.552315 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a170497f-6f5c-4161-9365-4f4ffe626896 None None] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:48:18.570214 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9df8746-cb5b-4568-8f4b-81f1d6e2d6c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:48:18.579451 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9df8746-cb5b-4568-8f4b-81f1d6e2d6c2 admin admin] REQUEST_METHOD: `PUT` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:18.580360 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9df8746-cb5b-4568-8f4b-81f1d6e2d6c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:18.585228 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9df8746-cb5b-4568-8f4b-81f1d6e2d6c2 admin admin] PATH_INFO: `/v3/projects/3611fa772bfc4e1daadff496ca269d7f/users/41c204e476a1402ba8b056065184dd15/roles/6c99f5d9f1f243eaa134d8540f21095d` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:18.636930 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9df8746-cb5b-4568-8f4b-81f1d6e2d6c2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3611fa772bfc4e1daadff496ca269d7f, user_id=41c204e476a1402ba8b056065184dd15, role_id=6c99f5d9f1f243eaa134d8540f21095d)` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:48:18.643494 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e9b5ced-b046-4864-821d-1be8cb176acb admin admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:48:18.647207 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9df8746-cb5b-4568-8f4b-81f1d6e2d6c2 admin admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:48:18.666957 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ddbb121-040d-4a68-b274-523bad93d049 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:48:18.668925 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ddbb121-040d-4a68-b274-523bad93d049 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:18.669234 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ddbb121-040d-4a68-b274-523bad93d049 admin admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:18.678470 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ddbb121-040d-4a68-b274-523bad93d049 admin admin] PATH_INFO: `/v3/projects/220b78acd5114533b881a6823728317b` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:18.682715 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ddbb121-040d-4a68-b274-523bad93d049 admin admin] RBAC: Authorizing `identity:delete_project(project_id=220b78acd5114533b881a6823728317b)` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:48:18.698288 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ddbb121-040d-4a68-b274-523bad93d049 admin admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:48:18.702520 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 895/1787] 10.222.0.13 () {60 vars in 1483 bytes} [Wed Aug 31 09:48:18 2022] PUT /identity/v3/projects/3611fa772bfc4e1daadff496ca269d7f/users/41c204e476a1402ba8b056065184dd15/roles/6c99f5d9f1f243eaa134d8540f21095d => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:48:18.703590 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9df8746-cb5b-4568-8f4b-81f1d6e2d6c2 admin admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:48:18.709818 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.fernet_utils [None req-d9df8746-cb5b-4568-8f4b-81f1d6e2d6c2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88114) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:48:18.756664 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=93,keystone:DELETE=5,keystone:INSERT=2 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:48:18.862367 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9df8746-cb5b-4568-8f4b-81f1d6e2d6c2 admin admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:48:18.869401 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e5c426-0a8b-4819-ae64-941f7cb05678 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:48:18.870139 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e5c426-0a8b-4819-ae64-941f7cb05678 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:18.870427 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e5c426-0a8b-4819-ae64-941f7cb05678 admin admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:18.870702 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e5c426-0a8b-4819-ae64-941f7cb05678 admin admin] PATH_INFO: `/v3/users/e4f1a935c10540f58c25b26fa4403929` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:18.872022 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05e5c426-0a8b-4819-ae64-941f7cb05678 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e4f1a935c10540f58c25b26fa4403929)` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:48:18.874515 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05e5c426-0a8b-4819-ae64-941f7cb05678 admin admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:48:19.062468 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.notifications [None req-05e5c426-0a8b-4819-ae64-941f7cb05678 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e4f1a935c10540f58c25b26fa4403929'} {{(pid=88114) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:48:19.079741 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:48:19.100411 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.notifications [None req-5ddbb121-040d-4a68-b274-523bad93d049 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '220b78acd5114533b881a6823728317b', 'request_id': 'req-5ddbb121-040d-4a68-b274-523bad93d049'} {{(pid=88113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:48:19.120344 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.notifications [None req-5ddbb121-040d-4a68-b274-523bad93d049 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '220b78acd5114533b881a6823728317b', 'request_id': 'req-5ddbb121-040d-4a68-b274-523bad93d049'} {{(pid=88113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:48:19.130955 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.notifications [None req-05e5c426-0a8b-4819-ae64-941f7cb05678 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e4f1a935c10540f58c25b26fa4403929'} {{(pid=88114) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:48:19.142540 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.notifications [None req-5ddbb121-040d-4a68-b274-523bad93d049 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '220b78acd5114533b881a6823728317b', 'request_id': 'req-5ddbb121-040d-4a68-b274-523bad93d049'} {{(pid=88113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:48:19.150766 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.notifications [None req-05e5c426-0a8b-4819-ae64-941f7cb05678 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e4f1a935c10540f58c25b26fa4403929'} {{(pid=88114) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:48:19.163557 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.notifications [None req-05e5c426-0a8b-4819-ae64-941f7cb05678 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e4f1a935c10540f58c25b26fa4403929'} {{(pid=88114) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:48:19.189026 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.notifications [None req-05e5c426-0a8b-4819-ae64-941f7cb05678 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e4f1a935c10540f58c25b26fa4403929'} {{(pid=88114) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:48:19.194207 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.notifications [None req-5ddbb121-040d-4a68-b274-523bad93d049 admin admin] The token cache is being invalidate because project 220b78acd5114533b881a6823728317b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88113) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 09:48:19.198407 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 896/1788] 10.222.0.13 () {58 vars in 1224 bytes} [Wed Aug 31 09:48:18 2022] DELETE /identity/v3/users/e4f1a935c10540f58c25b26fa4403929 => generated 0 bytes in 494 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:48:19.199805 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e5c426-0a8b-4819-ae64-941f7cb05678 admin admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:48:19.201605 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.fernet_utils [None req-05e5c426-0a8b-4819-ae64-941f7cb05678 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88114) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:48:19.202833 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.notifications [None req-5ddbb121-040d-4a68-b274-523bad93d049 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:48:19.206131 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.notifications [None req-5ddbb121-040d-4a68-b274-523bad93d049 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:48:19.213849 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 893/1789] 10.222.0.13 () {58 vars in 1233 bytes} [Wed Aug 31 09:48:18 2022] DELETE /identity/v3/projects/220b78acd5114533b881a6823728317b => generated 0 bytes in 1199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:48:19.219318 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00b9e89-a1b6-40cf-aa5f-1405673aee66 None None] REQUEST_METHOD: `POST` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:19.219996 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00b9e89-a1b6-40cf-aa5f-1405673aee66 None None] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:19.221508 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00b9e89-a1b6-40cf-aa5f-1405673aee66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:19.337943 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.auth.core [None req-e00b9e89-a1b6-40cf-aa5f-1405673aee66 None None] MFA Rules not processed for user `41c204e476a1402ba8b056065184dd15`. Rule list: `[]` (Enabled: `True`). {{(pid=88113) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:48:19.346453 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.fernet_utils [None req-e00b9e89-a1b6-40cf-aa5f-1405673aee66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:48:19.364497 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e5c426-0a8b-4819-ae64-941f7cb05678 admin admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:48:19.390139 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-030dd13b-e1b0-45cc-b9c9-55f2cf86d618 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:48:19.391233 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030dd13b-e1b0-45cc-b9c9-55f2cf86d618 admin admin] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:19.391652 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030dd13b-e1b0-45cc-b9c9-55f2cf86d618 admin admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:19.392004 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030dd13b-e1b0-45cc-b9c9-55f2cf86d618 admin admin] PATH_INFO: `/v3/domains` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:19.393177 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-030dd13b-e1b0-45cc-b9c9-55f2cf86d618 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88114) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:48:19.393720 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-030dd13b-e1b0-45cc-b9c9-55f2cf86d618 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:48:19.397883 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-030dd13b-e1b0-45cc-b9c9-55f2cf86d618 admin admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:48:19.474447 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 897/1790] 10.222.0.13 () {58 vars in 1153 bytes} [Wed Aug 31 09:48:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:48:19.476558 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-030dd13b-e1b0-45cc-b9c9-55f2cf86d618 admin admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:48:19.488604 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.fernet_utils [None req-030dd13b-e1b0-45cc-b9c9-55f2cf86d618 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88114) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:48:19.548766 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-030dd13b-e1b0-45cc-b9c9-55f2cf86d618 admin admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:48:19.572640 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e93e3975-015c-4634-b6e0-4af13c26096a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:48:19.573764 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93e3975-015c-4634-b6e0-4af13c26096a admin admin] REQUEST_METHOD: `DELETE` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:19.574209 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93e3975-015c-4634-b6e0-4af13c26096a admin admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:19.574793 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93e3975-015c-4634-b6e0-4af13c26096a admin admin] PATH_INFO: `/v3/users/3e4613bf47b84243bf8a2ad3aed3af03` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:19.585561 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e93e3975-015c-4634-b6e0-4af13c26096a admin admin] RBAC: Authorizing `identity:delete_user(user_id=3e4613bf47b84243bf8a2ad3aed3af03)` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:48:19.589310 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e93e3975-015c-4634-b6e0-4af13c26096a admin admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:48:19.711447 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 894/1791] 10.222.0.13 () {58 vars in 958 bytes} [Wed Aug 31 09:48:19 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 493 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:48:19.711447 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e00b9e89-a1b6-40cf-aa5f-1405673aee66 None None] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:48:19.726937 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.fernet_utils [None req-e00b9e89-a1b6-40cf-aa5f-1405673aee66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:48:19.734153 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:48:19.807180 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e00b9e89-a1b6-40cf-aa5f-1405673aee66 None None] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:48:19.821696 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8024eb81-98ff-47e7-af7b-1dc5d11937c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:48:19.822717 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8024eb81-98ff-47e7-af7b-1dc5d11937c5 admin admin] REQUEST_METHOD: `POST` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:19.823068 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8024eb81-98ff-47e7-af7b-1dc5d11937c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:19.823418 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8024eb81-98ff-47e7-af7b-1dc5d11937c5 admin admin] PATH_INFO: `/v3/projects` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:19.825281 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8024eb81-98ff-47e7-af7b-1dc5d11937c5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:48:19.829128 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8024eb81-98ff-47e7-af7b-1dc5d11937c5 admin admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:48:19.831312 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.notifications [None req-e93e3975-015c-4634-b6e0-4af13c26096a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3e4613bf47b84243bf8a2ad3aed3af03'} {{(pid=88114) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:48:19.844453 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.notifications [None req-e93e3975-015c-4634-b6e0-4af13c26096a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3e4613bf47b84243bf8a2ad3aed3af03'} {{(pid=88114) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:48:19.875543 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.notifications [None req-e93e3975-015c-4634-b6e0-4af13c26096a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3e4613bf47b84243bf8a2ad3aed3af03'} {{(pid=88114) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:48:19.881547 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.notifications [None req-e93e3975-015c-4634-b6e0-4af13c26096a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3e4613bf47b84243bf8a2ad3aed3af03'} {{(pid=88114) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:48:19.905806 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 895/1792] 10.222.0.13 () {60 vars in 1153 bytes} [Wed Aug 31 09:48:19 2022] POST /identity/v3/projects => generated 367 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 09:48:19.907471 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8024eb81-98ff-47e7-af7b-1dc5d11937c5 admin admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:48:19.911147 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.fernet_utils [None req-8024eb81-98ff-47e7-af7b-1dc5d11937c5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:48:19.926850 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.notifications [None req-e93e3975-015c-4634-b6e0-4af13c26096a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3e4613bf47b84243bf8a2ad3aed3af03'} {{(pid=88114) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:48:19.940060 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 898/1793] 10.222.0.13 () {58 vars in 1224 bytes} [Wed Aug 31 09:48:19 2022] DELETE /identity/v3/users/3e4613bf47b84243bf8a2ad3aed3af03 => generated 0 bytes in 465 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:48:19.941506 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e93e3975-015c-4634-b6e0-4af13c26096a admin admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:48:19.944123 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.fernet_utils [None req-e93e3975-015c-4634-b6e0-4af13c26096a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88114) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:48:20.222539 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8024eb81-98ff-47e7-af7b-1dc5d11937c5 admin admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:48:20.258536 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e93e3975-015c-4634-b6e0-4af13c26096a admin admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:48:20.295677 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07209548-8614-45cf-84fb-164693d1be94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:48:20.303555 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07209548-8614-45cf-84fb-164693d1be94 admin admin] REQUEST_METHOD: `POST` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:20.304311 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07209548-8614-45cf-84fb-164693d1be94 admin admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:20.305062 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07209548-8614-45cf-84fb-164693d1be94 admin admin] PATH_INFO: `/v3/users` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:20.308020 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07209548-8614-45cf-84fb-164693d1be94 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:48:20.317962 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07209548-8614-45cf-84fb-164693d1be94 admin admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:48:20.393995 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 899/1794] 10.222.0.13 () {60 vars in 1144 bytes} [Wed Aug 31 09:48:19 2022] POST /identity/v3/users => generated 284 bytes in 453 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 09:48:20.409681 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=94,keystone:DELETE=4,keystone:INSERT=2 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:48:20.412879 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6898726a-1011-4e96-a011-dc895696cec1 None None] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:20.415690 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6898726a-1011-4e96-a011-dc895696cec1 None None] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:20.416633 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6898726a-1011-4e96-a011-dc895696cec1 None None] PATH_INFO: `/` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:20.428630 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 900/1795] 10.222.0.13 () {54 vars in 921 bytes} [Wed Aug 31 09:48:20 2022] GET /identity => generated 270 bytes in 34 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:48:20.431168 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6898726a-1011-4e96-a011-dc895696cec1 None None] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:48:20.491141 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6898726a-1011-4e96-a011-dc895696cec1 None None] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:48:20.497178 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd1cd68-e3c9-4c65-8fa5-487f5340b75b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:48:20.501089 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd1cd68-e3c9-4c65-8fa5-487f5340b75b admin admin] REQUEST_METHOD: `POST` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:20.501981 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd1cd68-e3c9-4c65-8fa5-487f5340b75b admin admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:20.503533 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd1cd68-e3c9-4c65-8fa5-487f5340b75b admin admin] PATH_INFO: `/v3/projects` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:20.506717 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dd1cd68-e3c9-4c65-8fa5-487f5340b75b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:48:20.507070 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb0c558-2a4c-4496-9f36-080e89103099 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:48:20.510243 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb0c558-2a4c-4496-9f36-080e89103099 admin admin] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:20.511843 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb0c558-2a4c-4496-9f36-080e89103099 admin admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:20.511843 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb0c558-2a4c-4496-9f36-080e89103099 admin admin] PATH_INFO: `/v3/roles` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:20.512322 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdb0c558-2a4c-4496-9f36-080e89103099 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:48:20.513819 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dd1cd68-e3c9-4c65-8fa5-487f5340b75b admin admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:48:20.531228 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdb0c558-2a4c-4496-9f36-080e89103099 admin admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:48:20.588795 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 901/1796] 10.222.0.13 () {58 vars in 1122 bytes} [Wed Aug 31 09:48:20 2022] GET /identity/v3/roles => generated 1420 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 09:48:20.591479 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3d4731-610f-4257-a668-9681934dbe75 None None] REQUEST_METHOD: `POST` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:20.593443 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3d4731-610f-4257-a668-9681934dbe75 None None] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:20.593731 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3d4731-610f-4257-a668-9681934dbe75 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:20.610925 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 896/1797] 10.222.0.13 () {60 vars in 1153 bytes} [Wed Aug 31 09:48:19 2022] POST /identity/v3/projects => generated 381 bytes in 704 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 09:48:20.615255 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd1cd68-e3c9-4c65-8fa5-487f5340b75b admin admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:48:20.653724 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.auth.core [None req-bc3d4731-610f-4257-a668-9681934dbe75 None None] MFA Rules not processed for user `dc076493239d4ccdbdc37c0369c49529`. Rule list: `[]` (Enabled: `True`). {{(pid=88114) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:48:20.656200 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.fernet_utils [None req-bc3d4731-610f-4257-a668-9681934dbe75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88114) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:48:20.826154 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 902/1798] 10.222.0.13 () {58 vars in 1018 bytes} [Wed Aug 31 09:48:20 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 237 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:48:20.827123 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc3d4731-610f-4257-a668-9681934dbe75 None None] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:48:20.829356 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd1cd68-e3c9-4c65-8fa5-487f5340b75b admin admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:48:20.858336 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc3d4731-610f-4257-a668-9681934dbe75 None None] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:48:20.863260 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eee31b4-c564-4b82-a41f-344efdf17507 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:48:20.866309 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eee31b4-c564-4b82-a41f-344efdf17507 admin admin] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:20.867486 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eee31b4-c564-4b82-a41f-344efdf17507 admin admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:20.868284 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eee31b4-c564-4b82-a41f-344efdf17507 admin admin] PATH_INFO: `/v3/roles` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:20.870106 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eee31b4-c564-4b82-a41f-344efdf17507 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:48:20.875191 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eee31b4-c564-4b82-a41f-344efdf17507 admin admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:48:20.884977 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa7e228-786c-4ea1-a0b3-df1497452c29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:48:20.885732 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa7e228-786c-4ea1-a0b3-df1497452c29 admin admin] REQUEST_METHOD: `POST` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:20.886083 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa7e228-786c-4ea1-a0b3-df1497452c29 admin admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:20.886355 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa7e228-786c-4ea1-a0b3-df1497452c29 admin admin] PATH_INFO: `/v3/users` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:20.887502 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fa7e228-786c-4ea1-a0b3-df1497452c29 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:48:20.891424 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fa7e228-786c-4ea1-a0b3-df1497452c29 admin admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:48:20.901904 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 897/1799] 10.222.0.13 () {58 vars in 1122 bytes} [Wed Aug 31 09:48:20 2022] GET /identity/v3/roles => generated 1420 bytes in 290 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 09:48:20.908636 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eee31b4-c564-4b82-a41f-344efdf17507 admin admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:48:20.912007 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:DELETE=8,keystone:SELECT=88,keystone:INSERT=4 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:48:20.931082 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eee31b4-c564-4b82-a41f-344efdf17507 admin admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:48:20.948414 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f98ba5f4-5613-4f2b-95fc-14b085baab4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:48:20.948414 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98ba5f4-5613-4f2b-95fc-14b085baab4a admin admin] REQUEST_METHOD: `PUT` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:20.953339 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98ba5f4-5613-4f2b-95fc-14b085baab4a admin admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:20.953870 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98ba5f4-5613-4f2b-95fc-14b085baab4a admin admin] PATH_INFO: `/v3/projects/854d843d800944a7859e4927293eb5cb/users/f1bb1e059d26454a88ec518cda057bfc/roles/6c99f5d9f1f243eaa134d8540f21095d` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:20.956071 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 903/1800] 10.222.0.13 () {60 vars in 1144 bytes} [Wed Aug 31 09:48:20 2022] POST /identity/v3/users => generated 297 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 09:48:20.959338 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa7e228-786c-4ea1-a0b3-df1497452c29 admin admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:48:20.988118 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa7e228-786c-4ea1-a0b3-df1497452c29 admin admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:48:20.997720 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c19e41d7-e6b2-4ecc-a311-92b23a46ef3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:48:20.998848 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19e41d7-e6b2-4ecc-a311-92b23a46ef3a admin admin] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:20.999133 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19e41d7-e6b2-4ecc-a311-92b23a46ef3a admin admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:20.999439 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19e41d7-e6b2-4ecc-a311-92b23a46ef3a admin admin] PATH_INFO: `/v3/roles` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:21.000268 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c19e41d7-e6b2-4ecc-a311-92b23a46ef3a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:48:21.002867 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c19e41d7-e6b2-4ecc-a311-92b23a46ef3a admin admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:48:21.014020 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f98ba5f4-5613-4f2b-95fc-14b085baab4a admin admin] RBAC: Authorizing `identity:create_grant(project_id=854d843d800944a7859e4927293eb5cb, user_id=f1bb1e059d26454a88ec518cda057bfc, role_id=6c99f5d9f1f243eaa134d8540f21095d)` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:48:21.033655 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f98ba5f4-5613-4f2b-95fc-14b085baab4a admin admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:48:21.054328 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 904/1801] 10.222.0.13 () {58 vars in 1122 bytes} [Wed Aug 31 09:48:20 2022] GET /identity/v3/roles => generated 1420 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 09:48:21.068931 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c19e41d7-e6b2-4ecc-a311-92b23a46ef3a admin admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:48:21.071880 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 898/1802] 10.222.0.13 () {60 vars in 1483 bytes} [Wed Aug 31 09:48:20 2022] PUT /identity/v3/projects/854d843d800944a7859e4927293eb5cb/users/f1bb1e059d26454a88ec518cda057bfc/roles/6c99f5d9f1f243eaa134d8540f21095d => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:48:21.085234 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c19e41d7-e6b2-4ecc-a311-92b23a46ef3a admin admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:48:21.100700 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2cd707-7101-48ed-8f5d-417d5f49ec31 None None] REQUEST_METHOD: `POST` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:21.101208 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2cd707-7101-48ed-8f5d-417d5f49ec31 None None] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:21.101636 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2cd707-7101-48ed-8f5d-417d5f49ec31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:21.173630 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.auth.core [None req-5b2cd707-7101-48ed-8f5d-417d5f49ec31 None None] MFA Rules not processed for user `f1bb1e059d26454a88ec518cda057bfc`. Rule list: `[]` (Enabled: `True`). {{(pid=88113) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:48:21.175096 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.fernet_utils [None req-5b2cd707-7101-48ed-8f5d-417d5f49ec31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:48:21.381466 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86970c7e-8b1d-4044-b26f-1f545a8477e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:48:21.382170 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86970c7e-8b1d-4044-b26f-1f545a8477e5 admin admin] REQUEST_METHOD: `PUT` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:21.382452 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86970c7e-8b1d-4044-b26f-1f545a8477e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:21.382704 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86970c7e-8b1d-4044-b26f-1f545a8477e5 admin admin] PATH_INFO: `/v3/projects/d8376c848fa04ebca5b4e9e8351bd363/users/6794803d8b2d470cb5d924011d565670/roles/5d4ee63a3add47fc9a6b8b16d39e0a1f` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:21.429708 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86970c7e-8b1d-4044-b26f-1f545a8477e5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d8376c848fa04ebca5b4e9e8351bd363, user_id=6794803d8b2d470cb5d924011d565670, role_id=5d4ee63a3add47fc9a6b8b16d39e0a1f)` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:48:21.433027 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86970c7e-8b1d-4044-b26f-1f545a8477e5 admin admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:48:21.477518 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 905/1803] 10.222.0.13 () {60 vars in 1483 bytes} [Wed Aug 31 09:48:21 2022] PUT /identity/v3/projects/d8376c848fa04ebca5b4e9e8351bd363/users/6794803d8b2d470cb5d924011d565670/roles/5d4ee63a3add47fc9a6b8b16d39e0a1f => generated 0 bytes in 419 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:48:21.478375 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86970c7e-8b1d-4044-b26f-1f545a8477e5 admin admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:48:21.486033 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.fernet_utils [None req-86970c7e-8b1d-4044-b26f-1f545a8477e5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88114) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:48:21.531966 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 899/1804] 10.222.0.13 () {58 vars in 958 bytes} [Wed Aug 31 09:48:21 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 436 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:48:21.533359 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b2cd707-7101-48ed-8f5d-417d5f49ec31 None None] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:48:21.566412 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b2cd707-7101-48ed-8f5d-417d5f49ec31 None None] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:48:21.640369 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:48:21.713667 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-482fe258-911a-4759-b5c1-1d4b3a465990 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:48:21.714488 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482fe258-911a-4759-b5c1-1d4b3a465990 admin admin] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:21.714888 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482fe258-911a-4759-b5c1-1d4b3a465990 admin admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:21.717057 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482fe258-911a-4759-b5c1-1d4b3a465990 admin admin] PATH_INFO: `/v3/roles` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:21.717057 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-482fe258-911a-4759-b5c1-1d4b3a465990 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:48:21.721477 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-482fe258-911a-4759-b5c1-1d4b3a465990 admin admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:48:21.749797 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 900/1805] 10.222.0.13 () {58 vars in 1122 bytes} [Wed Aug 31 09:48:21 2022] GET /identity/v3/roles => generated 1420 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 09:48:21.754853 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-482fe258-911a-4759-b5c1-1d4b3a465990 admin admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:48:21.762220 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.fernet_utils [None req-482fe258-911a-4759-b5c1-1d4b3a465990 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:48:21.927365 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86970c7e-8b1d-4044-b26f-1f545a8477e5 admin admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:48:21.948449 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-590f7f69-9286-43fb-bb84-b0dbb13dd6a4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc076493239d4ccdbdc37c0369c49529', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '49548eb44d62415582af820ab66decee', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:48:21.949117 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590f7f69-9286-43fb-bb84-b0dbb13dd6a4 service nova] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:21.949649 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590f7f69-9286-43fb-bb84-b0dbb13dd6a4 service nova] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:21.950404 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590f7f69-9286-43fb-bb84-b0dbb13dd6a4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:22.001839 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-590f7f69-9286-43fb-bb84-b0dbb13dd6a4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:48:22.008365 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-590f7f69-9286-43fb-bb84-b0dbb13dd6a4 service nova] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:48:22.049933 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-482fe258-911a-4759-b5c1-1d4b3a465990 admin admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:48:22.069099 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0296fc48-4074-4f60-88ae-a2d5da8e9816 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d52b69c8e94441dacfcff944326f169', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '49548eb44d62415582af820ab66decee', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:48:22.084663 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 906/1806] 10.222.0.13 () {60 vars in 1365 bytes} [Wed Aug 31 09:48:21 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 593 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:48:22.084841 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0296fc48-4074-4f60-88ae-a2d5da8e9816 service cinder] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:22.085302 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0296fc48-4074-4f60-88ae-a2d5da8e9816 service cinder] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:22.087687 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-590f7f69-9286-43fb-bb84-b0dbb13dd6a4 service nova] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:48:22.090059 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0296fc48-4074-4f60-88ae-a2d5da8e9816 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:22.157298 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-590f7f69-9286-43fb-bb84-b0dbb13dd6a4 service nova] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:48:22.189294 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-244015bd-20d4-4658-97e1-e62395acc503 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:48:22.196776 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244015bd-20d4-4658-97e1-e62395acc503 admin admin] REQUEST_METHOD: `PUT` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:22.202292 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244015bd-20d4-4658-97e1-e62395acc503 admin admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:22.203064 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244015bd-20d4-4658-97e1-e62395acc503 admin admin] PATH_INFO: `/v3/projects/d8376c848fa04ebca5b4e9e8351bd363/users/6794803d8b2d470cb5d924011d565670/roles/5d4ee63a3add47fc9a6b8b16d39e0a1f` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:22.220495 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-244015bd-20d4-4658-97e1-e62395acc503 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d8376c848fa04ebca5b4e9e8351bd363, user_id=6794803d8b2d470cb5d924011d565670, role_id=5d4ee63a3add47fc9a6b8b16d39e0a1f)` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:48:22.240706 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-244015bd-20d4-4658-97e1-e62395acc503 admin admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:48:22.372082 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 907/1807] 10.222.0.13 () {60 vars in 1483 bytes} [Wed Aug 31 09:48:22 2022] PUT /identity/v3/projects/d8376c848fa04ebca5b4e9e8351bd363/users/6794803d8b2d470cb5d924011d565670/roles/5d4ee63a3add47fc9a6b8b16d39e0a1f => generated 0 bytes in 286 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:48:22.378024 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-244015bd-20d4-4658-97e1-e62395acc503 admin admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:48:22.388984 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.fernet_utils [None req-244015bd-20d4-4658-97e1-e62395acc503 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88114) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:48:22.616204 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0296fc48-4074-4f60-88ae-a2d5da8e9816 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:48:22.618045 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0296fc48-4074-4f60-88ae-a2d5da8e9816 service cinder] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:48:22.689634 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 901/1808] 10.222.0.13 () {60 vars in 1365 bytes} [Wed Aug 31 09:48:21 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 935 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:48:22.691946 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:INSERT=5,keystone:SELECT=95 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:48:22.694688 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8602d67-d59a-4dd9-a75a-8520b89dca1d None None] REQUEST_METHOD: `POST` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:22.697064 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8602d67-d59a-4dd9-a75a-8520b89dca1d None None] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:22.697600 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8602d67-d59a-4dd9-a75a-8520b89dca1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:22.869770 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-244015bd-20d4-4658-97e1-e62395acc503 admin admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:48:22.914160 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.auth.core [None req-e8602d67-d59a-4dd9-a75a-8520b89dca1d None None] MFA Rules not processed for user `6794803d8b2d470cb5d924011d565670`. Rule list: `[]` (Enabled: `True`). {{(pid=88113) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:48:22.915664 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.fernet_utils [None req-e8602d67-d59a-4dd9-a75a-8520b89dca1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:48:22.917703 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f80cca-15fb-4419-90d6-d7b2e222e6d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:48:22.922372 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f80cca-15fb-4419-90d6-d7b2e222e6d7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:22.934800 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f80cca-15fb-4419-90d6-d7b2e222e6d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:22.938648 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f80cca-15fb-4419-90d6-d7b2e222e6d7 admin admin] PATH_INFO: `/v3/projects/3df0a209d4c74b6390542f869999787a` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:22.950497 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3f80cca-15fb-4419-90d6-d7b2e222e6d7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3df0a209d4c74b6390542f869999787a)` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:48:23.007327 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3f80cca-15fb-4419-90d6-d7b2e222e6d7 admin admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:48:23.148946 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 902/1809] 10.222.0.13 () {58 vars in 958 bytes} [Wed Aug 31 09:48:22 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 459 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:48:23.285226 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8602d67-d59a-4dd9-a75a-8520b89dca1d None None] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:48:23.287793 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.fernet_utils [None req-e8602d67-d59a-4dd9-a75a-8520b89dca1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:48:23.314995 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=100 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:48:23.377968 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.notifications [None req-e3f80cca-15fb-4419-90d6-d7b2e222e6d7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3df0a209d4c74b6390542f869999787a', 'request_id': 'req-e3f80cca-15fb-4419-90d6-d7b2e222e6d7'} {{(pid=88114) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:48:23.406423 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.notifications [None req-e3f80cca-15fb-4419-90d6-d7b2e222e6d7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3df0a209d4c74b6390542f869999787a', 'request_id': 'req-e3f80cca-15fb-4419-90d6-d7b2e222e6d7'} {{(pid=88114) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:48:23.435786 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.notifications [None req-e3f80cca-15fb-4419-90d6-d7b2e222e6d7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3df0a209d4c74b6390542f869999787a', 'request_id': 'req-e3f80cca-15fb-4419-90d6-d7b2e222e6d7'} {{(pid=88114) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:48:23.483472 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.notifications [None req-e3f80cca-15fb-4419-90d6-d7b2e222e6d7 admin admin] The token cache is being invalidate because project 3df0a209d4c74b6390542f869999787a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88114) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 09:48:23.484155 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.notifications [None req-e3f80cca-15fb-4419-90d6-d7b2e222e6d7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88114) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:48:23.485679 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.notifications [None req-e3f80cca-15fb-4419-90d6-d7b2e222e6d7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88114) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:48:23.491471 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8602d67-d59a-4dd9-a75a-8520b89dca1d None None] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:48:23.495738 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 908/1810] 10.222.0.13 () {58 vars in 1233 bytes} [Wed Aug 31 09:48:22 2022] DELETE /identity/v3/projects/3df0a209d4c74b6390542f869999787a => generated 0 bytes in 1117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:48:23.499571 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7e0e57-d5d0-4ef5-936e-d5b58cf1215f None None] REQUEST_METHOD: `POST` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:23.500216 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7e0e57-d5d0-4ef5-936e-d5b58cf1215f None None] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:23.500876 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7e0e57-d5d0-4ef5-936e-d5b58cf1215f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:23.547354 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.auth.core [None req-4f7e0e57-d5d0-4ef5-936e-d5b58cf1215f None None] MFA Rules not processed for user `fd836cf590a145df9acf19cf3ae2ac56`. Rule list: `[]` (Enabled: `True`). {{(pid=88114) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:48:23.551314 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.fernet_utils [None req-4f7e0e57-d5d0-4ef5-936e-d5b58cf1215f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88114) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:48:23.724482 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-255a2b1e-1d61-4203-90f5-6e1ddc4f5276 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d52b69c8e94441dacfcff944326f169', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '49548eb44d62415582af820ab66decee', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:48:23.725444 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255a2b1e-1d61-4203-90f5-6e1ddc4f5276 service cinder] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:23.725736 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255a2b1e-1d61-4203-90f5-6e1ddc4f5276 service cinder] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:23.726016 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255a2b1e-1d61-4203-90f5-6e1ddc4f5276 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:23.729930 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.fernet_utils [None req-255a2b1e-1d61-4203-90f5-6e1ddc4f5276 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:48:23.871380 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 909/1811] 10.222.0.13 () {58 vars in 958 bytes} [Wed Aug 31 09:48:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 375 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:48:23.875792 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203874f6-6fa0-43e5-be13-06f51cd1d559 None None] REQUEST_METHOD: `POST` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:23.876153 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203874f6-6fa0-43e5-be13-06f51cd1d559 None None] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:23.876431 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203874f6-6fa0-43e5-be13-06f51cd1d559 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:23.966145 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.auth.core [None req-203874f6-6fa0-43e5-be13-06f51cd1d559 None None] MFA Rules not processed for user `fd836cf590a145df9acf19cf3ae2ac56`. Rule list: `[]` (Enabled: `True`). {{(pid=88114) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:48:23.977835 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.fernet_utils [None req-203874f6-6fa0-43e5-be13-06f51cd1d559 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88114) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:48:23.981209 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-255a2b1e-1d61-4203-90f5-6e1ddc4f5276 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:48:23.982842 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-255a2b1e-1d61-4203-90f5-6e1ddc4f5276 service cinder] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:48:23.992679 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 910/1812] 10.222.0.13 () {58 vars in 958 bytes} [Wed Aug 31 09:48:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:48:23.998407 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-203874f6-6fa0-43e5-be13-06f51cd1d559 None None] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:48:24.025628 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 903/1813] 10.222.0.13 () {60 vars in 1365 bytes} [Wed Aug 31 09:48:23 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 741 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 09:48:24.027745 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-203874f6-6fa0-43e5-be13-06f51cd1d559 None None] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:48:24.036167 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63bc9725-c905-4eb3-af09-069afc0a494a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:48:24.037211 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63bc9725-c905-4eb3-af09-069afc0a494a admin admin] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:24.037668 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63bc9725-c905-4eb3-af09-069afc0a494a admin admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:24.038184 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63bc9725-c905-4eb3-af09-069afc0a494a admin admin] PATH_INFO: `/v3/domains` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:24.039604 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63bc9725-c905-4eb3-af09-069afc0a494a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88114) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:48:24.040360 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63bc9725-c905-4eb3-af09-069afc0a494a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:48:24.042880 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63bc9725-c905-4eb3-af09-069afc0a494a admin admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:48:24.066600 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 911/1814] 10.222.0.13 () {58 vars in 1153 bytes} [Wed Aug 31 09:48:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:48:24.072013 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-255a2b1e-1d61-4203-90f5-6e1ddc4f5276 service cinder] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:48:24.084877 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-255a2b1e-1d61-4203-90f5-6e1ddc4f5276 service cinder] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:48:24.090432 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8706cfd-0bcb-47f4-bcf7-8f771b77b0d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:48:24.091079 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8706cfd-0bcb-47f4-bcf7-8f771b77b0d3 admin admin] REQUEST_METHOD: `POST` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:24.091484 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8706cfd-0bcb-47f4-bcf7-8f771b77b0d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:24.091770 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8706cfd-0bcb-47f4-bcf7-8f771b77b0d3 admin admin] PATH_INFO: `/v3/projects` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:24.092443 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8706cfd-0bcb-47f4-bcf7-8f771b77b0d3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:48:24.094658 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8706cfd-0bcb-47f4-bcf7-8f771b77b0d3 admin admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:48:24.155434 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63bc9725-c905-4eb3-af09-069afc0a494a admin admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:48:24.171257 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 904/1815] 10.222.0.13 () {60 vars in 1153 bytes} [Wed Aug 31 09:48:24 2022] POST /identity/v3/projects => generated 363 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 09:48:24.175710 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63bc9725-c905-4eb3-af09-069afc0a494a admin admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:48:24.208800 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8706cfd-0bcb-47f4-bcf7-8f771b77b0d3 admin admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:48:24.211757 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.fernet_utils [None req-b8706cfd-0bcb-47f4-bcf7-8f771b77b0d3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:48:24.323028 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:48:24.363132 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f5fca6-3c46-4acf-a697-1d7fa21945ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:48:24.364018 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f5fca6-3c46-4acf-a697-1d7fa21945ae admin admin] REQUEST_METHOD: `POST` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:24.364419 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f5fca6-3c46-4acf-a697-1d7fa21945ae admin admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:24.364789 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f5fca6-3c46-4acf-a697-1d7fa21945ae admin admin] PATH_INFO: `/v3/users` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:24.367604 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2f5fca6-3c46-4acf-a697-1d7fa21945ae admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:48:24.371565 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2f5fca6-3c46-4acf-a697-1d7fa21945ae admin admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:48:24.398823 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8706cfd-0bcb-47f4-bcf7-8f771b77b0d3 admin admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:48:24.403925 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c600def3-9167-489d-8598-6d4e7125307e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d52b69c8e94441dacfcff944326f169', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '49548eb44d62415582af820ab66decee', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:48:24.404586 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c600def3-9167-489d-8598-6d4e7125307e service cinder] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:24.404864 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c600def3-9167-489d-8598-6d4e7125307e service cinder] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:24.405143 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c600def3-9167-489d-8598-6d4e7125307e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:24.410014 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.fernet_utils [None req-c600def3-9167-489d-8598-6d4e7125307e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:48:24.425530 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 912/1816] 10.222.0.13 () {60 vars in 1144 bytes} [Wed Aug 31 09:48:24 2022] POST /identity/v3/users => generated 282 bytes in 267 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 09:48:24.425732 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:48:24.442987 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f5fca6-3c46-4acf-a697-1d7fa21945ae admin admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:48:24.498250 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f5fca6-3c46-4acf-a697-1d7fa21945ae admin admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:48:24.513113 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f36201a-d161-4b24-8815-b11d95797caa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:48:24.514030 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f36201a-d161-4b24-8815-b11d95797caa admin admin] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:24.514649 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f36201a-d161-4b24-8815-b11d95797caa admin admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:24.517618 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f36201a-d161-4b24-8815-b11d95797caa admin admin] PATH_INFO: `/v3/roles` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:24.518436 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f36201a-d161-4b24-8815-b11d95797caa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:48:24.520669 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f36201a-d161-4b24-8815-b11d95797caa admin admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:48:24.560200 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 913/1817] 10.222.0.13 () {58 vars in 1122 bytes} [Wed Aug 31 09:48:24 2022] GET /identity/v3/roles => generated 1420 bytes in 130 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 09:48:24.566403 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f36201a-d161-4b24-8815-b11d95797caa admin admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:48:24.575786 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c600def3-9167-489d-8598-6d4e7125307e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:48:24.577424 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c600def3-9167-489d-8598-6d4e7125307e service cinder] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:48:24.589670 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f36201a-d161-4b24-8815-b11d95797caa admin admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:48:24.596279 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 905/1818] 10.222.0.13 () {60 vars in 1365 bytes} [Wed Aug 31 09:48:24 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 388 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 09:48:24.599376 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2df1789-edb5-4170-a794-5c965b517b86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:48:24.600010 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2df1789-edb5-4170-a794-5c965b517b86 admin admin] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:24.600286 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2df1789-edb5-4170-a794-5c965b517b86 admin admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:24.600643 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2df1789-edb5-4170-a794-5c965b517b86 admin admin] PATH_INFO: `/v3/roles` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:24.601325 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2df1789-edb5-4170-a794-5c965b517b86 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:48:24.603154 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2df1789-edb5-4170-a794-5c965b517b86 admin admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:48:24.622680 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 914/1819] 10.222.0.13 () {58 vars in 1122 bytes} [Wed Aug 31 09:48:24 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 09:48:24.624029 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c600def3-9167-489d-8598-6d4e7125307e service cinder] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:48:24.640422 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c600def3-9167-489d-8598-6d4e7125307e service cinder] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:48:24.647486 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2df1789-edb5-4170-a794-5c965b517b86 admin admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:48:24.649472 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.fernet_utils [None req-a2df1789-edb5-4170-a794-5c965b517b86 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88114) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:48:24.664847 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a598cb09-62ac-433b-8d19-ce4fcf4e8e8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:48:24.665603 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a598cb09-62ac-433b-8d19-ce4fcf4e8e8a admin admin] REQUEST_METHOD: `PUT` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:24.665894 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a598cb09-62ac-433b-8d19-ce4fcf4e8e8a admin admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:24.666176 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a598cb09-62ac-433b-8d19-ce4fcf4e8e8a admin admin] PATH_INFO: `/v3/projects/0a46666d2494471d8c5a92b498ad2dec/users/a35ce525e06245f0b485c6dfe55a9512/roles/6c99f5d9f1f243eaa134d8540f21095d` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:24.675214 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2df1789-edb5-4170-a794-5c965b517b86 admin admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:48:24.688251 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4701ebe0-9355-4437-997e-d5e4bbcae288 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:48:24.688970 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4701ebe0-9355-4437-997e-d5e4bbcae288 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:24.689211 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4701ebe0-9355-4437-997e-d5e4bbcae288 admin admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:24.689519 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4701ebe0-9355-4437-997e-d5e4bbcae288 admin admin] PATH_INFO: `/v3/users/68b68dcf9382455a926ae2be6c0f5279` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:24.693541 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4701ebe0-9355-4437-997e-d5e4bbcae288 admin admin] RBAC: Authorizing `identity:delete_user(user_id=68b68dcf9382455a926ae2be6c0f5279)` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:48:24.697278 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4701ebe0-9355-4437-997e-d5e4bbcae288 admin admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:48:24.707060 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a598cb09-62ac-433b-8d19-ce4fcf4e8e8a admin admin] RBAC: Authorizing `identity:create_grant(project_id=0a46666d2494471d8c5a92b498ad2dec, user_id=a35ce525e06245f0b485c6dfe55a9512, role_id=6c99f5d9f1f243eaa134d8540f21095d)` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:48:24.712320 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a598cb09-62ac-433b-8d19-ce4fcf4e8e8a admin admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:48:24.753439 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 906/1820] 10.222.0.13 () {60 vars in 1483 bytes} [Wed Aug 31 09:48:24 2022] PUT /identity/v3/projects/0a46666d2494471d8c5a92b498ad2dec/users/a35ce525e06245f0b485c6dfe55a9512/roles/6c99f5d9f1f243eaa134d8540f21095d => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:48:24.773501 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a38fc39-7dab-4ba8-8612-2ba786cddfa2 None None] REQUEST_METHOD: `POST` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:24.776986 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a38fc39-7dab-4ba8-8612-2ba786cddfa2 None None] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:24.779745 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a38fc39-7dab-4ba8-8612-2ba786cddfa2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:24.869093 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.notifications [None req-4701ebe0-9355-4437-997e-d5e4bbcae288 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '68b68dcf9382455a926ae2be6c0f5279'} {{(pid=88114) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:48:24.884765 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.auth.core [None req-7a38fc39-7dab-4ba8-8612-2ba786cddfa2 None None] MFA Rules not processed for user `a35ce525e06245f0b485c6dfe55a9512`. Rule list: `[]` (Enabled: `True`). {{(pid=88113) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:48:24.887097 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.fernet_utils [None req-7a38fc39-7dab-4ba8-8612-2ba786cddfa2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:48:24.924602 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.notifications [None req-4701ebe0-9355-4437-997e-d5e4bbcae288 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '68b68dcf9382455a926ae2be6c0f5279'} {{(pid=88114) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:48:24.964861 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.notifications [None req-4701ebe0-9355-4437-997e-d5e4bbcae288 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '68b68dcf9382455a926ae2be6c0f5279'} {{(pid=88114) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:48:24.970127 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.notifications [None req-4701ebe0-9355-4437-997e-d5e4bbcae288 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '68b68dcf9382455a926ae2be6c0f5279'} {{(pid=88114) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:48:25.066545 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.notifications [None req-4701ebe0-9355-4437-997e-d5e4bbcae288 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '68b68dcf9382455a926ae2be6c0f5279'} {{(pid=88114) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:48:25.084281 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 915/1821] 10.222.0.13 () {58 vars in 1224 bytes} [Wed Aug 31 09:48:24 2022] DELETE /identity/v3/users/68b68dcf9382455a926ae2be6c0f5279 => generated 0 bytes in 437 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:48:25.090609 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f70ced-a34f-4859-a71f-c17ff655e4ba None None] REQUEST_METHOD: `POST` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:25.091337 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f70ced-a34f-4859-a71f-c17ff655e4ba None None] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:25.092336 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f70ced-a34f-4859-a71f-c17ff655e4ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:25.127833 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.auth.core [None req-b4f70ced-a34f-4859-a71f-c17ff655e4ba None None] MFA Rules not processed for user `fd836cf590a145df9acf19cf3ae2ac56`. Rule list: `[]` (Enabled: `True`). {{(pid=88114) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:48:25.131004 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.fernet_utils [None req-b4f70ced-a34f-4859-a71f-c17ff655e4ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88114) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:48:25.281780 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 916/1822] 10.222.0.13 () {58 vars in 958 bytes} [Wed Aug 31 09:48:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 09:48:25.303437 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 907/1823] 10.222.0.13 () {58 vars in 958 bytes} [Wed Aug 31 09:48:24 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 534 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:48:25.318715 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f70ced-a34f-4859-a71f-c17ff655e4ba None None] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:48:25.325656 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.fernet_utils [None req-b4f70ced-a34f-4859-a71f-c17ff655e4ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88114) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:48:25.361108 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a38fc39-7dab-4ba8-8612-2ba786cddfa2 None None] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:48:25.364210 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.fernet_utils [None req-7a38fc39-7dab-4ba8-8612-2ba786cddfa2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:48:25.379639 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f70ced-a34f-4859-a71f-c17ff655e4ba None None] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:48:25.430561 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a18da664-28da-495d-b2bf-9f302ad07582 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:48:25.434356 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18da664-28da-495d-b2bf-9f302ad07582 admin admin] REQUEST_METHOD: `POST` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:25.435869 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18da664-28da-495d-b2bf-9f302ad07582 admin admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:25.437165 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18da664-28da-495d-b2bf-9f302ad07582 admin admin] PATH_INFO: `/v3/projects` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:25.438144 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a18da664-28da-495d-b2bf-9f302ad07582 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:48:25.453660 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a18da664-28da-495d-b2bf-9f302ad07582 admin admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:48:25.554900 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 917/1824] 10.222.0.13 () {60 vars in 1153 bytes} [Wed Aug 31 09:48:25 2022] POST /identity/v3/projects => generated 365 bytes in 254 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 09:48:25.556797 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a18da664-28da-495d-b2bf-9f302ad07582 admin admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:48:25.558976 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.fernet_utils [None req-a18da664-28da-495d-b2bf-9f302ad07582 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88114) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:48:25.565992 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=85,keystone:INSERT=5,keystone:DELETE=10 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:48:25.582891 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a38fc39-7dab-4ba8-8612-2ba786cddfa2 None None] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:48:25.688422 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:48:25.769191 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a18da664-28da-495d-b2bf-9f302ad07582 admin admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:48:25.789957 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f749912a-6b93-43b0-8176-7b8100c24211 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:48:25.791129 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f749912a-6b93-43b0-8176-7b8100c24211 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:25.791625 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f749912a-6b93-43b0-8176-7b8100c24211 admin admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:25.792107 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f749912a-6b93-43b0-8176-7b8100c24211 admin admin] PATH_INFO: `/v3/projects/c68a2bf348bc4da6b00e539140456f3f` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:25.794572 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2332cfea-26fd-47f0-8da6-483ea760a622 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9bdaacd597442c2bcde2c555f2cc140', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '49548eb44d62415582af820ab66decee', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:48:25.795582 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2332cfea-26fd-47f0-8da6-483ea760a622 service neutron] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:25.795952 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2332cfea-26fd-47f0-8da6-483ea760a622 service neutron] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:25.796428 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2332cfea-26fd-47f0-8da6-483ea760a622 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:25.799083 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f749912a-6b93-43b0-8176-7b8100c24211 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c68a2bf348bc4da6b00e539140456f3f)` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:48:25.803221 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f749912a-6b93-43b0-8176-7b8100c24211 admin admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:48:25.823514 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2332cfea-26fd-47f0-8da6-483ea760a622 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:48:25.828195 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2332cfea-26fd-47f0-8da6-483ea760a622 service neutron] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:48:25.940895 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 908/1825] 10.222.0.13 () {60 vars in 1365 bytes} [Wed Aug 31 09:48:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 581 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:48:25.944226 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2332cfea-26fd-47f0-8da6-483ea760a622 service neutron] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:48:25.973914 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2332cfea-26fd-47f0-8da6-483ea760a622 service neutron] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:48:25.999256 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.notifications [None req-f749912a-6b93-43b0-8176-7b8100c24211 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c68a2bf348bc4da6b00e539140456f3f', 'request_id': 'req-f749912a-6b93-43b0-8176-7b8100c24211'} {{(pid=88114) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:48:26.018723 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.notifications [None req-f749912a-6b93-43b0-8176-7b8100c24211 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c68a2bf348bc4da6b00e539140456f3f', 'request_id': 'req-f749912a-6b93-43b0-8176-7b8100c24211'} {{(pid=88114) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:48:26.040832 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.notifications [None req-f749912a-6b93-43b0-8176-7b8100c24211 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c68a2bf348bc4da6b00e539140456f3f', 'request_id': 'req-f749912a-6b93-43b0-8176-7b8100c24211'} {{(pid=88114) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:48:26.107825 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.notifications [None req-f749912a-6b93-43b0-8176-7b8100c24211 admin admin] The token cache is being invalidate because project c68a2bf348bc4da6b00e539140456f3f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88114) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 09:48:26.108553 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.notifications [None req-f749912a-6b93-43b0-8176-7b8100c24211 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88114) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:48:26.112624 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.notifications [None req-f749912a-6b93-43b0-8176-7b8100c24211 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88114) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:48:26.126007 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 918/1826] 10.222.0.13 () {58 vars in 1233 bytes} [Wed Aug 31 09:48:25 2022] DELETE /identity/v3/projects/c68a2bf348bc4da6b00e539140456f3f => generated 0 bytes in 569 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:48:26.133867 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66834aa7-08b6-46c9-87f4-3b5fc5040d6d None None] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:26.134376 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66834aa7-08b6-46c9-87f4-3b5fc5040d6d None None] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:26.143753 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66834aa7-08b6-46c9-87f4-3b5fc5040d6d None None] PATH_INFO: `/` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:26.143945 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a08c4af-bd9b-4f25-97f6-0758bb371c1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:48:26.143945 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a08c4af-bd9b-4f25-97f6-0758bb371c1e admin admin] REQUEST_METHOD: `POST` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:26.144125 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a08c4af-bd9b-4f25-97f6-0758bb371c1e admin admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:26.144488 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a08c4af-bd9b-4f25-97f6-0758bb371c1e admin admin] PATH_INFO: `/v3/users` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:26.148240 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a08c4af-bd9b-4f25-97f6-0758bb371c1e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:48:26.154275 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 919/1827] 10.222.0.13 () {54 vars in 921 bytes} [Wed Aug 31 09:48:26 2022] GET /identity => generated 270 bytes in 29 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:48:26.156323 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a08c4af-bd9b-4f25-97f6-0758bb371c1e admin admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:48:26.156690 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0765a8-37cf-4a29-be49-93b41ccafde1 None None] REQUEST_METHOD: `POST` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:26.167680 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0765a8-37cf-4a29-be49-93b41ccafde1 None None] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:26.168336 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0765a8-37cf-4a29-be49-93b41ccafde1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:26.267906 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 909/1828] 10.222.0.13 () {60 vars in 1144 bytes} [Wed Aug 31 09:48:25 2022] POST /identity/v3/users => generated 289 bytes in 326 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 09:48:26.274981 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a08c4af-bd9b-4f25-97f6-0758bb371c1e admin admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:48:26.290183 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.fernet_utils [None req-4a08c4af-bd9b-4f25-97f6-0758bb371c1e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:48:26.298438 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.auth.core [None req-eb0765a8-37cf-4a29-be49-93b41ccafde1 None None] MFA Rules not processed for user `fd836cf590a145df9acf19cf3ae2ac56`. Rule list: `[]` (Enabled: `True`). {{(pid=88114) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:48:26.300699 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.fernet_utils [None req-eb0765a8-37cf-4a29-be49-93b41ccafde1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88114) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:48:26.356182 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 920/1829] 10.222.0.13 () {58 vars in 958 bytes} [Wed Aug 31 09:48:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:48:26.358454 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a08c4af-bd9b-4f25-97f6-0758bb371c1e admin admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:48:26.371318 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba4e688-0d00-42e7-8b5e-42076a233a12 None None] REQUEST_METHOD: `POST` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:26.372329 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba4e688-0d00-42e7-8b5e-42076a233a12 None None] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:26.373041 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba4e688-0d00-42e7-8b5e-42076a233a12 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:26.383418 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f4a4fd-8bc6-4122-ae56-07883240ccde admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:48:26.384481 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f4a4fd-8bc6-4122-ae56-07883240ccde admin admin] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:26.386396 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f4a4fd-8bc6-4122-ae56-07883240ccde admin admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:26.386882 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f4a4fd-8bc6-4122-ae56-07883240ccde admin admin] PATH_INFO: `/v3/roles` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:26.391115 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9f4a4fd-8bc6-4122-ae56-07883240ccde admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:48:26.394320 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9f4a4fd-8bc6-4122-ae56-07883240ccde admin admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:48:26.437682 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 910/1830] 10.222.0.13 () {58 vars in 1122 bytes} [Wed Aug 31 09:48:26 2022] GET /identity/v3/roles => generated 1420 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 09:48:26.439518 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f4a4fd-8bc6-4122-ae56-07883240ccde admin admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:48:26.461219 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f4a4fd-8bc6-4122-ae56-07883240ccde admin admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:48:26.465642 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.auth.core [None req-2ba4e688-0d00-42e7-8b5e-42076a233a12 None None] MFA Rules not processed for user `fd836cf590a145df9acf19cf3ae2ac56`. Rule list: `[]` (Enabled: `True`). {{(pid=88114) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:48:26.468185 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.fernet_utils [None req-2ba4e688-0d00-42e7-8b5e-42076a233a12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88114) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:48:26.474013 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b18db3-0a88-477e-b263-d5bb995f5d93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:48:26.476710 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b18db3-0a88-477e-b263-d5bb995f5d93 admin admin] REQUEST_METHOD: `PUT` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:26.477551 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b18db3-0a88-477e-b263-d5bb995f5d93 admin admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:26.478045 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b18db3-0a88-477e-b263-d5bb995f5d93 admin admin] PATH_INFO: `/v3/projects/c429b68532d14cdebd9001948104fcb5/users/d8efae832dba45f5b59d608730fd20a1/roles/5d4ee63a3add47fc9a6b8b16d39e0a1f` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:26.486488 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 921/1831] 10.222.0.13 () {58 vars in 958 bytes} [Wed Aug 31 09:48:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:48:26.488581 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23352681-540e-4dfc-8ac3-316a48906274 None None] REQUEST_METHOD: `POST` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:26.489110 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23352681-540e-4dfc-8ac3-316a48906274 None None] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:26.489581 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23352681-540e-4dfc-8ac3-316a48906274 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:26.522146 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53b18db3-0a88-477e-b263-d5bb995f5d93 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c429b68532d14cdebd9001948104fcb5, user_id=d8efae832dba45f5b59d608730fd20a1, role_id=5d4ee63a3add47fc9a6b8b16d39e0a1f)` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:48:26.525986 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.auth.core [None req-23352681-540e-4dfc-8ac3-316a48906274 None None] MFA Rules not processed for user `dc076493239d4ccdbdc37c0369c49529`. Rule list: `[]` (Enabled: `True`). {{(pid=88114) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:48:26.527906 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.fernet_utils [None req-23352681-540e-4dfc-8ac3-316a48906274 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88114) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:48:26.528328 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53b18db3-0a88-477e-b263-d5bb995f5d93 admin admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:48:26.564337 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 911/1832] 10.222.0.13 () {60 vars in 1483 bytes} [Wed Aug 31 09:48:26 2022] PUT /identity/v3/projects/c429b68532d14cdebd9001948104fcb5/users/d8efae832dba45f5b59d608730fd20a1/roles/5d4ee63a3add47fc9a6b8b16d39e0a1f => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:48:26.566120 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b18db3-0a88-477e-b263-d5bb995f5d93 admin admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:48:26.674360 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b18db3-0a88-477e-b263-d5bb995f5d93 admin admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:48:26.686763 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 922/1833] 10.222.0.13 () {58 vars in 1018 bytes} [Wed Aug 31 09:48:26 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:48:26.687771 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23352681-540e-4dfc-8ac3-316a48906274 None None] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:48:26.709456 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b45df52d-fb1f-4250-8784-0d66b0ee8aad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:48:26.712572 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45df52d-fb1f-4250-8784-0d66b0ee8aad admin admin] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:26.712572 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45df52d-fb1f-4250-8784-0d66b0ee8aad admin admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:26.713299 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45df52d-fb1f-4250-8784-0d66b0ee8aad admin admin] PATH_INFO: `/v3/domains` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:26.715977 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b45df52d-fb1f-4250-8784-0d66b0ee8aad admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88113) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:48:26.717695 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b45df52d-fb1f-4250-8784-0d66b0ee8aad admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:48:26.727846 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23352681-540e-4dfc-8ac3-316a48906274 None None] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:48:26.730278 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b45df52d-fb1f-4250-8784-0d66b0ee8aad admin admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:48:26.735697 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-948db3f8-5e4f-4dba-91c8-c896eed1de87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:48:26.736566 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948db3f8-5e4f-4dba-91c8-c896eed1de87 admin admin] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:26.736996 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948db3f8-5e4f-4dba-91c8-c896eed1de87 admin admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:26.737459 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948db3f8-5e4f-4dba-91c8-c896eed1de87 admin admin] PATH_INFO: `/v3/roles` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:26.738406 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-948db3f8-5e4f-4dba-91c8-c896eed1de87 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:48:26.741195 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-948db3f8-5e4f-4dba-91c8-c896eed1de87 admin admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:48:26.750152 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:48:26.786848 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 912/1834] 10.222.0.13 () {58 vars in 1153 bytes} [Wed Aug 31 09:48:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:48:26.792061 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 923/1835] 10.222.0.13 () {58 vars in 1122 bytes} [Wed Aug 31 09:48:26 2022] GET /identity/v3/roles => generated 1420 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 09:48:26.795000 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b45df52d-fb1f-4250-8784-0d66b0ee8aad admin admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:48:26.797362 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-948db3f8-5e4f-4dba-91c8-c896eed1de87 admin admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:48:26.832001 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b45df52d-fb1f-4250-8784-0d66b0ee8aad admin admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:48:26.849343 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-660e8189-840e-49ed-bd7c-92588bc29981 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:48:26.854814 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-948db3f8-5e4f-4dba-91c8-c896eed1de87 admin admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:48:26.857034 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660e8189-840e-49ed-bd7c-92588bc29981 admin admin] REQUEST_METHOD: `PUT` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:26.857578 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660e8189-840e-49ed-bd7c-92588bc29981 admin admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:26.869418 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660e8189-840e-49ed-bd7c-92588bc29981 admin admin] PATH_INFO: `/v3/projects/c429b68532d14cdebd9001948104fcb5/users/d8efae832dba45f5b59d608730fd20a1/roles/5d4ee63a3add47fc9a6b8b16d39e0a1f` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:26.869418 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-660e8189-840e-49ed-bd7c-92588bc29981 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c429b68532d14cdebd9001948104fcb5, user_id=d8efae832dba45f5b59d608730fd20a1, role_id=5d4ee63a3add47fc9a6b8b16d39e0a1f)` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:48:26.876169 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-660e8189-840e-49ed-bd7c-92588bc29981 admin admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:48:26.891650 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c7e6a21-c8a4-4093-9ef9-35243396d581 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:48:26.892641 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7e6a21-c8a4-4093-9ef9-35243396d581 admin admin] REQUEST_METHOD: `POST` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:26.893093 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7e6a21-c8a4-4093-9ef9-35243396d581 admin admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:26.893634 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7e6a21-c8a4-4093-9ef9-35243396d581 admin admin] PATH_INFO: `/v3/projects` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:26.894499 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c7e6a21-c8a4-4093-9ef9-35243396d581 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:48:26.897723 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c7e6a21-c8a4-4093-9ef9-35243396d581 admin admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:48:26.955022 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 913/1836] 10.222.0.13 () {60 vars in 1483 bytes} [Wed Aug 31 09:48:26 2022] PUT /identity/v3/projects/c429b68532d14cdebd9001948104fcb5/users/d8efae832dba45f5b59d608730fd20a1/roles/5d4ee63a3add47fc9a6b8b16d39e0a1f => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:48:26.963766 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1dfc3f-8ac2-440e-88a2-e36ac303f853 None None] REQUEST_METHOD: `POST` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:26.964024 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1dfc3f-8ac2-440e-88a2-e36ac303f853 None None] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:26.964184 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1dfc3f-8ac2-440e-88a2-e36ac303f853 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:26.977522 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:48:26.999300 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 924/1837] 10.222.0.13 () {60 vars in 1153 bytes} [Wed Aug 31 09:48:26 2022] POST /identity/v3/projects => generated 383 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 09:48:27.007222 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c7e6a21-c8a4-4093-9ef9-35243396d581 admin admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:48:27.017984 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.auth.core [None req-2b1dfc3f-8ac2-440e-88a2-e36ac303f853 None None] MFA Rules not processed for user `d8efae832dba45f5b59d608730fd20a1`. Rule list: `[]` (Enabled: `True`). {{(pid=88113) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:48:27.021069 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.fernet_utils [None req-2b1dfc3f-8ac2-440e-88a2-e36ac303f853 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:48:27.155539 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c7e6a21-c8a4-4093-9ef9-35243396d581 admin admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:48:27.156937 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 914/1838] 10.222.0.13 () {58 vars in 958 bytes} [Wed Aug 31 09:48:26 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:48:27.166225 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b1dfc3f-8ac2-440e-88a2-e36ac303f853 None None] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:48:27.168252 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e37f2a-7c2e-4c17-82aa-9f5fd069f3c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:48:27.168498 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.fernet_utils [None req-2b1dfc3f-8ac2-440e-88a2-e36ac303f853 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:48:27.169989 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e37f2a-7c2e-4c17-82aa-9f5fd069f3c4 admin admin] REQUEST_METHOD: `POST` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:27.170483 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e37f2a-7c2e-4c17-82aa-9f5fd069f3c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:27.170966 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e37f2a-7c2e-4c17-82aa-9f5fd069f3c4 admin admin] PATH_INFO: `/v3/users` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:27.172486 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63e37f2a-7c2e-4c17-82aa-9f5fd069f3c4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:48:27.178299 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63e37f2a-7c2e-4c17-82aa-9f5fd069f3c4 admin admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:48:27.247630 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 925/1839] 10.222.0.13 () {60 vars in 1144 bytes} [Wed Aug 31 09:48:27 2022] POST /identity/v3/users => generated 292 bytes in 242 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 09:48:27.249412 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e37f2a-7c2e-4c17-82aa-9f5fd069f3c4 admin admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:48:27.259553 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.fernet_utils [None req-63e37f2a-7c2e-4c17-82aa-9f5fd069f3c4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88114) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:48:27.319609 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b1dfc3f-8ac2-440e-88a2-e36ac303f853 None None] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:48:27.329838 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39db2d78-cbb8-4d0d-95c8-5e9b5070ea87 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d52b69c8e94441dacfcff944326f169', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '49548eb44d62415582af820ab66decee', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:48:27.333312 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39db2d78-cbb8-4d0d-95c8-5e9b5070ea87 service cinder] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:27.334570 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39db2d78-cbb8-4d0d-95c8-5e9b5070ea87 service cinder] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:27.337898 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39db2d78-cbb8-4d0d-95c8-5e9b5070ea87 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:27.366621 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39db2d78-cbb8-4d0d-95c8-5e9b5070ea87 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:48:27.368545 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39db2d78-cbb8-4d0d-95c8-5e9b5070ea87 service cinder] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:48:27.398952 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 915/1840] 10.222.0.13 () {60 vars in 1365 bytes} [Wed Aug 31 09:48:27 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 233 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 09:48:27.400087 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39db2d78-cbb8-4d0d-95c8-5e9b5070ea87 service cinder] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:48:27.422620 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39db2d78-cbb8-4d0d-95c8-5e9b5070ea87 service cinder] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:48:27.433182 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42875cfe-5ad6-4afd-ac11-34c12064dab3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:48:27.433828 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42875cfe-5ad6-4afd-ac11-34c12064dab3 admin admin] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:27.435653 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42875cfe-5ad6-4afd-ac11-34c12064dab3 admin admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:27.435944 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42875cfe-5ad6-4afd-ac11-34c12064dab3 admin admin] PATH_INFO: `/v3/roles` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:27.436643 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42875cfe-5ad6-4afd-ac11-34c12064dab3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:48:27.438328 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42875cfe-5ad6-4afd-ac11-34c12064dab3 admin admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:48:27.483470 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 916/1841] 10.222.0.13 () {58 vars in 1122 bytes} [Wed Aug 31 09:48:27 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 09:48:27.490229 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42875cfe-5ad6-4afd-ac11-34c12064dab3 admin admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:48:27.526247 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42875cfe-5ad6-4afd-ac11-34c12064dab3 admin admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:48:27.548692 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ad6cc1-99d9-4514-ae53-17723b8e0b84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:48:27.553512 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ad6cc1-99d9-4514-ae53-17723b8e0b84 admin admin] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:27.554092 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ad6cc1-99d9-4514-ae53-17723b8e0b84 admin admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:27.554409 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ad6cc1-99d9-4514-ae53-17723b8e0b84 admin admin] PATH_INFO: `/v3/roles` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:27.555538 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41ad6cc1-99d9-4514-ae53-17723b8e0b84 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:48:27.560640 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41ad6cc1-99d9-4514-ae53-17723b8e0b84 admin admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:48:27.615861 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 917/1842] 10.222.0.13 () {58 vars in 1122 bytes} [Wed Aug 31 09:48:27 2022] GET /identity/v3/roles => generated 1420 bytes in 125 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 09:48:27.616925 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ad6cc1-99d9-4514-ae53-17723b8e0b84 admin admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:48:27.625750 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e37f2a-7c2e-4c17-82aa-9f5fd069f3c4 admin admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:48:27.647614 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcafdde4-d22f-4a5d-8ccf-066341d0e9e1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc076493239d4ccdbdc37c0369c49529', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '49548eb44d62415582af820ab66decee', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:48:27.649764 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcafdde4-d22f-4a5d-8ccf-066341d0e9e1 service nova] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:27.650940 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcafdde4-d22f-4a5d-8ccf-066341d0e9e1 service nova] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:27.651231 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcafdde4-d22f-4a5d-8ccf-066341d0e9e1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:27.668420 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ad6cc1-99d9-4514-ae53-17723b8e0b84 admin admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:48:27.680372 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcafdde4-d22f-4a5d-8ccf-066341d0e9e1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:48:27.682501 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcafdde4-d22f-4a5d-8ccf-066341d0e9e1 service nova] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:48:27.687090 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e5d1eb-822a-4de3-abd9-84a9dde99a0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:48:27.688222 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e5d1eb-822a-4de3-abd9-84a9dde99a0b admin admin] REQUEST_METHOD: `PUT` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:27.688686 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e5d1eb-822a-4de3-abd9-84a9dde99a0b admin admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:27.689163 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e5d1eb-822a-4de3-abd9-84a9dde99a0b admin admin] PATH_INFO: `/v3/projects/32cd034c0d60483eb19614049ccefb41/users/297786c3e5db4d2890d09b922bea7dd2/roles/6c99f5d9f1f243eaa134d8540f21095d` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:27.718364 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 926/1843] 10.222.0.13 () {60 vars in 1365 bytes} [Wed Aug 31 09:48:27 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 470 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:48:27.719503 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcafdde4-d22f-4a5d-8ccf-066341d0e9e1 service nova] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:48:27.726415 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.fernet_utils [None req-bcafdde4-d22f-4a5d-8ccf-066341d0e9e1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88114) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:48:27.731368 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92e5d1eb-822a-4de3-abd9-84a9dde99a0b admin admin] RBAC: Authorizing `identity:create_grant(project_id=32cd034c0d60483eb19614049ccefb41, user_id=297786c3e5db4d2890d09b922bea7dd2, role_id=6c99f5d9f1f243eaa134d8540f21095d)` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:48:27.737927 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92e5d1eb-822a-4de3-abd9-84a9dde99a0b admin admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:48:27.759673 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcafdde4-d22f-4a5d-8ccf-066341d0e9e1 service nova] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:48:27.775276 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 918/1844] 10.222.0.13 () {60 vars in 1483 bytes} [Wed Aug 31 09:48:27 2022] PUT /identity/v3/projects/32cd034c0d60483eb19614049ccefb41/users/297786c3e5db4d2890d09b922bea7dd2/roles/6c99f5d9f1f243eaa134d8540f21095d => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:48:27.788304 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4cb5aa-caa7-4cfb-ae8a-84c9ca084c15 None None] REQUEST_METHOD: `POST` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:27.788827 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4cb5aa-caa7-4cfb-ae8a-84c9ca084c15 None None] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:27.789269 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4cb5aa-caa7-4cfb-ae8a-84c9ca084c15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:27.902074 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.auth.core [None req-3d4cb5aa-caa7-4cfb-ae8a-84c9ca084c15 None None] MFA Rules not processed for user `297786c3e5db4d2890d09b922bea7dd2`. Rule list: `[]` (Enabled: `True`). {{(pid=88113) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:48:27.904055 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:48:27.904906 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.fernet_utils [None req-3d4cb5aa-caa7-4cfb-ae8a-84c9ca084c15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:48:27.928225 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f24832-7301-48e3-9e11-61668ceb8758 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:48:27.931378 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f24832-7301-48e3-9e11-61668ceb8758 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:27.932452 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f24832-7301-48e3-9e11-61668ceb8758 admin admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:27.933126 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f24832-7301-48e3-9e11-61668ceb8758 admin admin] PATH_INFO: `/v3/projects/6f2fae641f6c4f8188a8f8828248936f` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:27.952051 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41f24832-7301-48e3-9e11-61668ceb8758 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6f2fae641f6c4f8188a8f8828248936f)` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:48:27.958598 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41f24832-7301-48e3-9e11-61668ceb8758 admin admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:48:28.083671 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:48:28.115747 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.notifications [None req-41f24832-7301-48e3-9e11-61668ceb8758 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6f2fae641f6c4f8188a8f8828248936f', 'request_id': 'req-41f24832-7301-48e3-9e11-61668ceb8758'} {{(pid=88114) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:48:28.133126 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.notifications [None req-41f24832-7301-48e3-9e11-61668ceb8758 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6f2fae641f6c4f8188a8f8828248936f', 'request_id': 'req-41f24832-7301-48e3-9e11-61668ceb8758'} {{(pid=88114) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:48:28.168746 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.notifications [None req-41f24832-7301-48e3-9e11-61668ceb8758 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6f2fae641f6c4f8188a8f8828248936f', 'request_id': 'req-41f24832-7301-48e3-9e11-61668ceb8758'} {{(pid=88114) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:48:28.201428 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.notifications [None req-41f24832-7301-48e3-9e11-61668ceb8758 admin admin] The token cache is being invalidate because project 6f2fae641f6c4f8188a8f8828248936f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88114) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 09:48:28.201428 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.notifications [None req-41f24832-7301-48e3-9e11-61668ceb8758 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88114) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:48:28.203865 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.notifications [None req-41f24832-7301-48e3-9e11-61668ceb8758 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88114) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:48:28.206518 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 927/1845] 10.222.0.13 () {58 vars in 1233 bytes} [Wed Aug 31 09:48:27 2022] DELETE /identity/v3/projects/6f2fae641f6c4f8188a8f8828248936f => generated 0 bytes in 487 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:48:28.211790 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f24832-7301-48e3-9e11-61668ceb8758 admin admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:48:28.213433 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.fernet_utils [None req-41f24832-7301-48e3-9e11-61668ceb8758 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88114) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:48:28.266321 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 919/1846] 10.222.0.13 () {58 vars in 958 bytes} [Wed Aug 31 09:48:27 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 476 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:48:28.267956 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d4cb5aa-caa7-4cfb-ae8a-84c9ca084c15 None None] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:48:28.327456 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d4cb5aa-caa7-4cfb-ae8a-84c9ca084c15 None None] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:48:28.382557 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f24832-7301-48e3-9e11-61668ceb8758 admin admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:48:28.398192 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a40d21-6b4f-4540-aa90-9272a537cdcd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:48:28.398925 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a40d21-6b4f-4540-aa90-9272a537cdcd admin admin] REQUEST_METHOD: `DELETE` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:28.399651 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a40d21-6b4f-4540-aa90-9272a537cdcd admin admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:28.399826 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a40d21-6b4f-4540-aa90-9272a537cdcd admin admin] PATH_INFO: `/v3/users/086fbacfad4a42ec9be55fcb49f80d9d` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:28.403337 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8a40d21-6b4f-4540-aa90-9272a537cdcd admin admin] RBAC: Authorizing `identity:delete_user(user_id=086fbacfad4a42ec9be55fcb49f80d9d)` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:48:28.412516 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8a40d21-6b4f-4540-aa90-9272a537cdcd admin admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:48:28.491584 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba2d44cd-ec25-40c3-a7cc-8996b35f322a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:48:28.494047 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2d44cd-ec25-40c3-a7cc-8996b35f322a admin admin] REQUEST_METHOD: `POST` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:28.495114 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2d44cd-ec25-40c3-a7cc-8996b35f322a admin admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:28.496023 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2d44cd-ec25-40c3-a7cc-8996b35f322a admin admin] PATH_INFO: `/v3/projects` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:28.500371 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba2d44cd-ec25-40c3-a7cc-8996b35f322a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:48:28.503581 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba2d44cd-ec25-40c3-a7cc-8996b35f322a admin admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:48:28.553010 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.notifications [None req-d8a40d21-6b4f-4540-aa90-9272a537cdcd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '086fbacfad4a42ec9be55fcb49f80d9d'} {{(pid=88114) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:48:28.573607 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.notifications [None req-d8a40d21-6b4f-4540-aa90-9272a537cdcd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '086fbacfad4a42ec9be55fcb49f80d9d'} {{(pid=88114) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:48:28.584952 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.notifications [None req-d8a40d21-6b4f-4540-aa90-9272a537cdcd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '086fbacfad4a42ec9be55fcb49f80d9d'} {{(pid=88114) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:48:28.585874 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.notifications [None req-d8a40d21-6b4f-4540-aa90-9272a537cdcd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '086fbacfad4a42ec9be55fcb49f80d9d'} {{(pid=88114) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:48:28.588031 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 920/1847] 10.222.0.13 () {60 vars in 1153 bytes} [Wed Aug 31 09:48:28 2022] POST /identity/v3/projects => generated 383 bytes in 321 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 09:48:28.589735 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba2d44cd-ec25-40c3-a7cc-8996b35f322a admin admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:48:28.593226 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.fernet_utils [None req-ba2d44cd-ec25-40c3-a7cc-8996b35f322a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:48:28.608180 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.notifications [None req-d8a40d21-6b4f-4540-aa90-9272a537cdcd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '086fbacfad4a42ec9be55fcb49f80d9d'} {{(pid=88114) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:48:28.617547 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 928/1848] 10.222.0.13 () {58 vars in 1224 bytes} [Wed Aug 31 09:48:28 2022] DELETE /identity/v3/users/086fbacfad4a42ec9be55fcb49f80d9d => generated 0 bytes in 406 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 09:48:28.618490 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a40d21-6b4f-4540-aa90-9272a537cdcd admin admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:48:28.620680 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.fernet_utils [None req-d8a40d21-6b4f-4540-aa90-9272a537cdcd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88114) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:48:28.627843 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:48:28.779384 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba2d44cd-ec25-40c3-a7cc-8996b35f322a admin admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:48:28.831355 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a40d21-6b4f-4540-aa90-9272a537cdcd admin admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:48:28.847769 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f86569-12ef-415d-82f0-67ad268a16bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:48:28.849468 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f86569-12ef-415d-82f0-67ad268a16bd admin admin] REQUEST_METHOD: `POST` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:28.850266 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f86569-12ef-415d-82f0-67ad268a16bd admin admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:28.851002 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f86569-12ef-415d-82f0-67ad268a16bd admin admin] PATH_INFO: `/v3/users` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:28.852693 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63f86569-12ef-415d-82f0-67ad268a16bd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:48:28.865307 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63f86569-12ef-415d-82f0-67ad268a16bd admin admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:48:28.931130 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 929/1849] 10.222.0.13 () {60 vars in 1144 bytes} [Wed Aug 31 09:48:28 2022] POST /identity/v3/users => generated 298 bytes in 312 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 09:48:28.939856 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f86569-12ef-415d-82f0-67ad268a16bd admin admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:48:28.944819 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d2428b-ccbb-4e76-a09b-601f9049adb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:48:28.945860 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d2428b-ccbb-4e76-a09b-601f9049adb1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:28.946133 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d2428b-ccbb-4e76-a09b-601f9049adb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:28.946394 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d2428b-ccbb-4e76-a09b-601f9049adb1 admin admin] PATH_INFO: `/v3/users/a35ce525e06245f0b485c6dfe55a9512` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:28.948053 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4d2428b-ccbb-4e76-a09b-601f9049adb1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a35ce525e06245f0b485c6dfe55a9512)` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:48:28.951058 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4d2428b-ccbb-4e76-a09b-601f9049adb1 admin admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:48:28.975254 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:48:28.980766 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f86569-12ef-415d-82f0-67ad268a16bd admin admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:48:28.992226 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7455b0ac-c033-41d3-894f-fa0bafd2d161 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:48:28.994944 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7455b0ac-c033-41d3-894f-fa0bafd2d161 admin admin] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:28.994944 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7455b0ac-c033-41d3-894f-fa0bafd2d161 admin admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:28.994944 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7455b0ac-c033-41d3-894f-fa0bafd2d161 admin admin] PATH_INFO: `/v3/roles` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:28.995790 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7455b0ac-c033-41d3-894f-fa0bafd2d161 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:48:29.000846 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7455b0ac-c033-41d3-894f-fa0bafd2d161 admin admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:48:29.053473 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 930/1850] 10.222.0.13 () {58 vars in 1122 bytes} [Wed Aug 31 09:48:28 2022] GET /identity/v3/roles => generated 1420 bytes in 116 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 09:48:29.063980 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7455b0ac-c033-41d3-894f-fa0bafd2d161 admin admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:48:29.084986 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7455b0ac-c033-41d3-894f-fa0bafd2d161 admin admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:48:29.101460 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68148e8f-f54d-4157-8ec9-e4c88b13793a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:48:29.102023 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68148e8f-f54d-4157-8ec9-e4c88b13793a admin admin] REQUEST_METHOD: `PUT` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:29.102620 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68148e8f-f54d-4157-8ec9-e4c88b13793a admin admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:29.103043 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68148e8f-f54d-4157-8ec9-e4c88b13793a admin admin] PATH_INFO: `/v3/projects/5f145ab743f54358a029d0cf18202b8b/users/464bb24610c34dfc8ff3fb7be1920ad9/roles/5d4ee63a3add47fc9a6b8b16d39e0a1f` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:29.115803 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.notifications [None req-b4d2428b-ccbb-4e76-a09b-601f9049adb1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a35ce525e06245f0b485c6dfe55a9512'} {{(pid=88113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:48:29.135675 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.notifications [None req-b4d2428b-ccbb-4e76-a09b-601f9049adb1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a35ce525e06245f0b485c6dfe55a9512'} {{(pid=88113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:48:29.170456 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68148e8f-f54d-4157-8ec9-e4c88b13793a admin admin] RBAC: Authorizing `identity:create_grant(project_id=5f145ab743f54358a029d0cf18202b8b, user_id=464bb24610c34dfc8ff3fb7be1920ad9, role_id=5d4ee63a3add47fc9a6b8b16d39e0a1f)` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:48:29.177418 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68148e8f-f54d-4157-8ec9-e4c88b13793a admin admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:48:29.182675 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.notifications [None req-b4d2428b-ccbb-4e76-a09b-601f9049adb1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a35ce525e06245f0b485c6dfe55a9512'} {{(pid=88113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:48:29.183984 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.notifications [None req-b4d2428b-ccbb-4e76-a09b-601f9049adb1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a35ce525e06245f0b485c6dfe55a9512'} {{(pid=88113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:48:29.200242 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.notifications [None req-b4d2428b-ccbb-4e76-a09b-601f9049adb1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a35ce525e06245f0b485c6dfe55a9512'} {{(pid=88113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:48:29.209160 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 931/1851] 10.222.0.13 () {60 vars in 1483 bytes} [Wed Aug 31 09:48:29 2022] PUT /identity/v3/projects/5f145ab743f54358a029d0cf18202b8b/users/464bb24610c34dfc8ff3fb7be1920ad9/roles/5d4ee63a3add47fc9a6b8b16d39e0a1f => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 09:48:29.211536 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 921/1852] 10.222.0.13 () {58 vars in 1224 bytes} [Wed Aug 31 09:48:28 2022] DELETE /identity/v3/users/a35ce525e06245f0b485c6dfe55a9512 => generated 0 bytes in 623 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:48:29.216748 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68148e8f-f54d-4157-8ec9-e4c88b13793a admin admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:48:29.219453 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b1a013-9dd1-478a-b6d2-c6d89d577d0f None None] REQUEST_METHOD: `POST` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:29.220249 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b1a013-9dd1-478a-b6d2-c6d89d577d0f None None] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:29.220542 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b1a013-9dd1-478a-b6d2-c6d89d577d0f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:29.222440 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.fernet_utils [None req-68148e8f-f54d-4157-8ec9-e4c88b13793a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88114) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:48:29.278593 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.auth.core [None req-b9b1a013-9dd1-478a-b6d2-c6d89d577d0f None None] MFA Rules not processed for user `fd836cf590a145df9acf19cf3ae2ac56`. Rule list: `[]` (Enabled: `True`). {{(pid=88113) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:48:29.280116 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.fernet_utils [None req-b9b1a013-9dd1-478a-b6d2-c6d89d577d0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:48:29.403966 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68148e8f-f54d-4157-8ec9-e4c88b13793a admin admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:48:29.467608 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 922/1853] 10.222.0.13 () {58 vars in 958 bytes} [Wed Aug 31 09:48:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 250 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:48:29.469364 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b1a013-9dd1-478a-b6d2-c6d89d577d0f None None] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:48:29.480295 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.fernet_utils [None req-b9b1a013-9dd1-478a-b6d2-c6d89d577d0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:48:29.500596 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b1a013-9dd1-478a-b6d2-c6d89d577d0f None None] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:48:29.501782 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-103b6ddc-41c3-4fb0-b24a-5152c2bea144 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:48:29.502924 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103b6ddc-41c3-4fb0-b24a-5152c2bea144 admin admin] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:29.503421 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103b6ddc-41c3-4fb0-b24a-5152c2bea144 admin admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:29.503746 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103b6ddc-41c3-4fb0-b24a-5152c2bea144 admin admin] PATH_INFO: `/v3/roles` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:29.505326 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-103b6ddc-41c3-4fb0-b24a-5152c2bea144 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:48:29.507260 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-103b6ddc-41c3-4fb0-b24a-5152c2bea144 admin admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:48:29.509007 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c950e4e-004d-4a41-860d-8ae66760b791 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:48:29.509868 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c950e4e-004d-4a41-860d-8ae66760b791 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:29.510693 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c950e4e-004d-4a41-860d-8ae66760b791 admin admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:29.510960 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c950e4e-004d-4a41-860d-8ae66760b791 admin admin] PATH_INFO: `/v3/projects/96994ecf01df44a398f67e4ada14f7f7` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:29.512694 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c950e4e-004d-4a41-860d-8ae66760b791 admin admin] RBAC: Authorizing `identity:delete_project(project_id=96994ecf01df44a398f67e4ada14f7f7)` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:48:29.515631 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c950e4e-004d-4a41-860d-8ae66760b791 admin admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:48:29.516445 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:48:29.529565 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 932/1854] 10.222.0.13 () {58 vars in 1122 bytes} [Wed Aug 31 09:48:29 2022] GET /identity/v3/roles => generated 1420 bytes in 313 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 09:48:29.529565 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-103b6ddc-41c3-4fb0-b24a-5152c2bea144 admin admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:48:29.531925 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.fernet_utils [None req-103b6ddc-41c3-4fb0-b24a-5152c2bea144 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88114) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:48:29.609966 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.notifications [None req-9c950e4e-004d-4a41-860d-8ae66760b791 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '96994ecf01df44a398f67e4ada14f7f7', 'request_id': 'req-9c950e4e-004d-4a41-860d-8ae66760b791'} {{(pid=88113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:48:29.620924 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.notifications [None req-9c950e4e-004d-4a41-860d-8ae66760b791 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '96994ecf01df44a398f67e4ada14f7f7', 'request_id': 'req-9c950e4e-004d-4a41-860d-8ae66760b791'} {{(pid=88113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:48:29.635957 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.notifications [None req-9c950e4e-004d-4a41-860d-8ae66760b791 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '96994ecf01df44a398f67e4ada14f7f7', 'request_id': 'req-9c950e4e-004d-4a41-860d-8ae66760b791'} {{(pid=88113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:48:29.650515 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-103b6ddc-41c3-4fb0-b24a-5152c2bea144 admin admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:48:29.657611 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.notifications [None req-9c950e4e-004d-4a41-860d-8ae66760b791 admin admin] The token cache is being invalidate because project 96994ecf01df44a398f67e4ada14f7f7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88113) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 09:48:29.657912 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.notifications [None req-9c950e4e-004d-4a41-860d-8ae66760b791 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:48:29.658740 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.notifications [None req-9c950e4e-004d-4a41-860d-8ae66760b791 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:48:29.661896 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 923/1855] 10.222.0.13 () {58 vars in 1233 bytes} [Wed Aug 31 09:48:29 2022] DELETE /identity/v3/projects/96994ecf01df44a398f67e4ada14f7f7 => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:48:29.662858 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c950e4e-004d-4a41-860d-8ae66760b791 admin admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:48:29.664685 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.fernet_utils [None req-9c950e4e-004d-4a41-860d-8ae66760b791 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:48:29.681880 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:DELETE=17,keystone:SELECT=81,keystone:INSERT=2 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:48:29.771670 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-306ee107-376e-45c2-b2f4-1ee40102e7d7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9bdaacd597442c2bcde2c555f2cc140', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '49548eb44d62415582af820ab66decee', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:48:29.771670 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306ee107-376e-45c2-b2f4-1ee40102e7d7 service neutron] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:29.771670 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306ee107-376e-45c2-b2f4-1ee40102e7d7 service neutron] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:29.771670 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306ee107-376e-45c2-b2f4-1ee40102e7d7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:29.781162 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.fernet_utils [None req-306ee107-376e-45c2-b2f4-1ee40102e7d7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88114) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:48:29.860891 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c950e4e-004d-4a41-860d-8ae66760b791 admin admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:48:29.868191 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8fdee6b-6b69-432f-997d-4209b4fd3ade admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:48:29.869353 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8fdee6b-6b69-432f-997d-4209b4fd3ade admin admin] REQUEST_METHOD: `PUT` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:29.869839 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8fdee6b-6b69-432f-997d-4209b4fd3ade admin admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:29.870317 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8fdee6b-6b69-432f-997d-4209b4fd3ade admin admin] PATH_INFO: `/v3/projects/5f145ab743f54358a029d0cf18202b8b/users/464bb24610c34dfc8ff3fb7be1920ad9/roles/5d4ee63a3add47fc9a6b8b16d39e0a1f` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:29.873969 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8fdee6b-6b69-432f-997d-4209b4fd3ade admin admin] RBAC: Authorizing `identity:create_grant(project_id=5f145ab743f54358a029d0cf18202b8b, user_id=464bb24610c34dfc8ff3fb7be1920ad9, role_id=5d4ee63a3add47fc9a6b8b16d39e0a1f)` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:48:29.878004 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8fdee6b-6b69-432f-997d-4209b4fd3ade admin admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:48:29.892210 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-306ee107-376e-45c2-b2f4-1ee40102e7d7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:48:29.893866 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-306ee107-376e-45c2-b2f4-1ee40102e7d7 service neutron] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:48:29.897425 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 924/1856] 10.222.0.13 () {60 vars in 1483 bytes} [Wed Aug 31 09:48:29 2022] PUT /identity/v3/projects/5f145ab743f54358a029d0cf18202b8b/users/464bb24610c34dfc8ff3fb7be1920ad9/roles/5d4ee63a3add47fc9a6b8b16d39e0a1f => generated 0 bytes in 235 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:48:29.900003 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90bf7f12-9a10-4be0-80c5-e39aaae2ab3d None None] REQUEST_METHOD: `POST` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:29.900501 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90bf7f12-9a10-4be0-80c5-e39aaae2ab3d None None] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:29.901000 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90bf7f12-9a10-4be0-80c5-e39aaae2ab3d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:29.913402 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 933/1857] 10.222.0.13 () {60 vars in 1365 bytes} [Wed Aug 31 09:48:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 385 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:48:29.916745 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23af851a-8c91-4256-9f20-5e6deeae56a8 None None] REQUEST_METHOD: `POST` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:29.917368 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23af851a-8c91-4256-9f20-5e6deeae56a8 None None] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:29.917887 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23af851a-8c91-4256-9f20-5e6deeae56a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:29.939071 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.auth.core [None req-90bf7f12-9a10-4be0-80c5-e39aaae2ab3d None None] MFA Rules not processed for user `fd836cf590a145df9acf19cf3ae2ac56`. Rule list: `[]` (Enabled: `True`). {{(pid=88113) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:48:29.942228 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.fernet_utils [None req-90bf7f12-9a10-4be0-80c5-e39aaae2ab3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:48:29.956793 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.auth.core [None req-23af851a-8c91-4256-9f20-5e6deeae56a8 None None] MFA Rules not processed for user `464bb24610c34dfc8ff3fb7be1920ad9`. Rule list: `[]` (Enabled: `True`). {{(pid=88114) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:48:29.959066 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.fernet_utils [None req-23af851a-8c91-4256-9f20-5e6deeae56a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88114) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:48:30.038428 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=100 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:48:30.071633 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 934/1858] 10.222.0.13 () {58 vars in 958 bytes} [Wed Aug 31 09:48:29 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 09:48:30.085825 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23af851a-8c91-4256-9f20-5e6deeae56a8 None None] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:48:30.087681 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.fernet_utils [None req-23af851a-8c91-4256-9f20-5e6deeae56a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88114) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:48:30.100664 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 925/1859] 10.222.0.13 () {58 vars in 958 bytes} [Wed Aug 31 09:48:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:48:30.114039 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf89e1b-5557-4c05-a7bd-bd940bc1478f None None] REQUEST_METHOD: `POST` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:30.114039 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf89e1b-5557-4c05-a7bd-bd940bc1478f None None] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:30.114039 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf89e1b-5557-4c05-a7bd-bd940bc1478f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:30.170435 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.auth.core [None req-0cf89e1b-5557-4c05-a7bd-bd940bc1478f None None] MFA Rules not processed for user `fd836cf590a145df9acf19cf3ae2ac56`. Rule list: `[]` (Enabled: `True`). {{(pid=88113) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:48:30.174507 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.fernet_utils [None req-0cf89e1b-5557-4c05-a7bd-bd940bc1478f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:48:30.198861 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 926/1860] 10.222.0.13 () {58 vars in 958 bytes} [Wed Aug 31 09:48:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:48:30.200234 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf89e1b-5557-4c05-a7bd-bd940bc1478f None None] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:48:30.214419 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf89e1b-5557-4c05-a7bd-bd940bc1478f None None] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:48:30.221828 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a963686d-7d30-4c83-8a33-93880e3c529f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:48:30.223313 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a963686d-7d30-4c83-8a33-93880e3c529f admin admin] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:30.223739 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a963686d-7d30-4c83-8a33-93880e3c529f admin admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:30.224144 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a963686d-7d30-4c83-8a33-93880e3c529f admin admin] PATH_INFO: `/v3/domains` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:30.225005 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a963686d-7d30-4c83-8a33-93880e3c529f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88113) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:48:30.225575 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a963686d-7d30-4c83-8a33-93880e3c529f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:48:30.227724 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a963686d-7d30-4c83-8a33-93880e3c529f admin admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:48:30.236932 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23af851a-8c91-4256-9f20-5e6deeae56a8 None None] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:48:30.258578 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f77b767-20f9-4807-b865-13f3d6d5ed87 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d52b69c8e94441dacfcff944326f169', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '49548eb44d62415582af820ab66decee', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:48:30.263309 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 927/1861] 10.222.0.13 () {58 vars in 1153 bytes} [Wed Aug 31 09:48:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:48:30.268697 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f77b767-20f9-4807-b865-13f3d6d5ed87 service cinder] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:30.268697 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f77b767-20f9-4807-b865-13f3d6d5ed87 service cinder] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:30.268697 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f77b767-20f9-4807-b865-13f3d6d5ed87 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:30.269439 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a963686d-7d30-4c83-8a33-93880e3c529f admin admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:48:30.289545 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a963686d-7d30-4c83-8a33-93880e3c529f admin admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:48:30.299100 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db253ab7-bd92-4ffb-bb3d-f2d51b048a59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:48:30.300021 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db253ab7-bd92-4ffb-bb3d-f2d51b048a59 admin admin] REQUEST_METHOD: `POST` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:30.300021 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db253ab7-bd92-4ffb-bb3d-f2d51b048a59 admin admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:30.300299 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db253ab7-bd92-4ffb-bb3d-f2d51b048a59 admin admin] PATH_INFO: `/v3/projects` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:30.305666 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db253ab7-bd92-4ffb-bb3d-f2d51b048a59 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:48:30.310455 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db253ab7-bd92-4ffb-bb3d-f2d51b048a59 admin admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:48:30.345823 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f77b767-20f9-4807-b865-13f3d6d5ed87 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:48:30.356975 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f77b767-20f9-4807-b865-13f3d6d5ed87 service cinder] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:48:30.392621 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 928/1862] 10.222.0.13 () {60 vars in 1153 bytes} [Wed Aug 31 09:48:30 2022] POST /identity/v3/projects => generated 375 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 09:48:30.398322 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db253ab7-bd92-4ffb-bb3d-f2d51b048a59 admin admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:48:30.409326 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 935/1863] 10.222.0.13 () {60 vars in 1365 bytes} [Wed Aug 31 09:48:30 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 09:48:30.554548 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f77b767-20f9-4807-b865-13f3d6d5ed87 service cinder] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:48:30.556832 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.fernet_utils [None req-0f77b767-20f9-4807-b865-13f3d6d5ed87 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88114) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:48:30.576943 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db253ab7-bd92-4ffb-bb3d-f2d51b048a59 admin admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:48:30.584755 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f77b767-20f9-4807-b865-13f3d6d5ed87 service cinder] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:48:30.590259 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b96279e-19b1-45e5-839b-01278bc48453 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:48:30.590987 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b96279e-19b1-45e5-839b-01278bc48453 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:30.591267 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b96279e-19b1-45e5-839b-01278bc48453 admin admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:30.591548 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b96279e-19b1-45e5-839b-01278bc48453 admin admin] PATH_INFO: `/v3/projects/0a46666d2494471d8c5a92b498ad2dec` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:30.593042 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b96279e-19b1-45e5-839b-01278bc48453 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0a46666d2494471d8c5a92b498ad2dec)` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:48:30.593372 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b0e20b-717b-4104-ab7b-593fde9ab5ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:48:30.595173 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b96279e-19b1-45e5-839b-01278bc48453 admin admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:48:30.595598 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b0e20b-717b-4104-ab7b-593fde9ab5ac admin admin] REQUEST_METHOD: `POST` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:30.596366 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b0e20b-717b-4104-ab7b-593fde9ab5ac admin admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:30.597448 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b0e20b-717b-4104-ab7b-593fde9ab5ac admin admin] PATH_INFO: `/v3/users` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:30.599402 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60b0e20b-717b-4104-ab7b-593fde9ab5ac admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:48:30.604140 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60b0e20b-717b-4104-ab7b-593fde9ab5ac admin admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:48:30.626772 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:48:30.669731 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 929/1864] 10.222.0.13 () {60 vars in 1144 bytes} [Wed Aug 31 09:48:30 2022] POST /identity/v3/users => generated 288 bytes in 272 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 09:48:30.671371 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b0e20b-717b-4104-ab7b-593fde9ab5ac admin admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:48:30.699566 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.notifications [None req-9b96279e-19b1-45e5-839b-01278bc48453 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0a46666d2494471d8c5a92b498ad2dec', 'request_id': 'req-9b96279e-19b1-45e5-839b-01278bc48453'} {{(pid=88114) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:48:30.714106 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.notifications [None req-9b96279e-19b1-45e5-839b-01278bc48453 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0a46666d2494471d8c5a92b498ad2dec', 'request_id': 'req-9b96279e-19b1-45e5-839b-01278bc48453'} {{(pid=88114) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:48:30.726141 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.notifications [None req-9b96279e-19b1-45e5-839b-01278bc48453 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0a46666d2494471d8c5a92b498ad2dec', 'request_id': 'req-9b96279e-19b1-45e5-839b-01278bc48453'} {{(pid=88114) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:48:30.746805 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.notifications [None req-9b96279e-19b1-45e5-839b-01278bc48453 admin admin] The token cache is being invalidate because project 0a46666d2494471d8c5a92b498ad2dec was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88114) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 09:48:30.747198 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.notifications [None req-9b96279e-19b1-45e5-839b-01278bc48453 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88114) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:48:30.748083 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.notifications [None req-9b96279e-19b1-45e5-839b-01278bc48453 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88114) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:48:30.755682 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 936/1865] 10.222.0.13 () {58 vars in 1233 bytes} [Wed Aug 31 09:48:30 2022] DELETE /identity/v3/projects/0a46666d2494471d8c5a92b498ad2dec => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:48:30.755682 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b96279e-19b1-45e5-839b-01278bc48453 admin admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:48:30.755682 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.fernet_utils [None req-9b96279e-19b1-45e5-839b-01278bc48453 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88114) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:48:30.779012 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b0e20b-717b-4104-ab7b-593fde9ab5ac admin admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:48:30.857013 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0b7ecd4-11ba-479a-8ef0-ae001953266c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d52b69c8e94441dacfcff944326f169', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '49548eb44d62415582af820ab66decee', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:48:30.858185 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b7ecd4-11ba-479a-8ef0-ae001953266c service cinder] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:30.858679 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b7ecd4-11ba-479a-8ef0-ae001953266c service cinder] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:30.859214 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b7ecd4-11ba-479a-8ef0-ae001953266c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:30.861740 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b96279e-19b1-45e5-839b-01278bc48453 admin admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:48:30.863787 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.fernet_utils [None req-f0b7ecd4-11ba-479a-8ef0-ae001953266c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:48:30.868191 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a734d304-decf-432e-8b18-26c3a7761617 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:48:30.868919 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a734d304-decf-432e-8b18-26c3a7761617 admin admin] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:30.869269 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a734d304-decf-432e-8b18-26c3a7761617 admin admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:30.869650 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a734d304-decf-432e-8b18-26c3a7761617 admin admin] PATH_INFO: `/v3/roles` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:30.870429 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a734d304-decf-432e-8b18-26c3a7761617 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:48:30.874050 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a734d304-decf-432e-8b18-26c3a7761617 admin admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:48:30.898158 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 937/1866] 10.222.0.13 () {58 vars in 1122 bytes} [Wed Aug 31 09:48:30 2022] GET /identity/v3/roles => generated 1420 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 09:48:30.898158 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a734d304-decf-432e-8b18-26c3a7761617 admin admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:48:30.899928 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.fernet_utils [None req-a734d304-decf-432e-8b18-26c3a7761617 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88114) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:48:30.922528 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a734d304-decf-432e-8b18-26c3a7761617 admin admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:48:30.929853 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b120adc1-619f-4693-9612-92d99eb0e327 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:48:30.930543 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b120adc1-619f-4693-9612-92d99eb0e327 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:30.930856 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b120adc1-619f-4693-9612-92d99eb0e327 admin admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:30.931111 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b120adc1-619f-4693-9612-92d99eb0e327 admin admin] PATH_INFO: `/v3/users/d8efae832dba45f5b59d608730fd20a1` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:30.932491 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b120adc1-619f-4693-9612-92d99eb0e327 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d8efae832dba45f5b59d608730fd20a1)` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:48:30.934504 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b120adc1-619f-4693-9612-92d99eb0e327 admin admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:48:30.942880 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:48:31.018781 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0b7ecd4-11ba-479a-8ef0-ae001953266c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:48:31.022106 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0b7ecd4-11ba-479a-8ef0-ae001953266c service cinder] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:48:31.053293 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 930/1867] 10.222.0.13 () {60 vars in 1365 bytes} [Wed Aug 31 09:48:30 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 383 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:48:31.054855 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0b7ecd4-11ba-479a-8ef0-ae001953266c service cinder] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:48:31.078123 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0b7ecd4-11ba-479a-8ef0-ae001953266c service cinder] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:48:31.093830 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8221488e-687d-4658-8b34-98ec49634394 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:48:31.094930 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.notifications [None req-b120adc1-619f-4693-9612-92d99eb0e327 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd8efae832dba45f5b59d608730fd20a1'} {{(pid=88114) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:48:31.095105 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8221488e-687d-4658-8b34-98ec49634394 admin admin] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:31.095693 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8221488e-687d-4658-8b34-98ec49634394 admin admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:31.096383 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8221488e-687d-4658-8b34-98ec49634394 admin admin] PATH_INFO: `/v3/roles` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:31.097679 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8221488e-687d-4658-8b34-98ec49634394 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:48:31.100639 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8221488e-687d-4658-8b34-98ec49634394 admin admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:48:31.110484 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.notifications [None req-b120adc1-619f-4693-9612-92d99eb0e327 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd8efae832dba45f5b59d608730fd20a1'} {{(pid=88114) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:48:31.123134 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 931/1868] 10.222.0.13 () {58 vars in 1122 bytes} [Wed Aug 31 09:48:31 2022] GET /identity/v3/roles => generated 1420 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 09:48:31.128485 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8221488e-687d-4658-8b34-98ec49634394 admin admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:48:31.132406 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.notifications [None req-b120adc1-619f-4693-9612-92d99eb0e327 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd8efae832dba45f5b59d608730fd20a1'} {{(pid=88114) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:48:31.133085 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.notifications [None req-b120adc1-619f-4693-9612-92d99eb0e327 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd8efae832dba45f5b59d608730fd20a1'} {{(pid=88114) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:48:31.159371 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8221488e-687d-4658-8b34-98ec49634394 admin admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:48:31.170755 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.notifications [None req-b120adc1-619f-4693-9612-92d99eb0e327 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd8efae832dba45f5b59d608730fd20a1'} {{(pid=88114) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:48:31.172430 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e862628-728a-465d-991f-3be01e339f44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:48:31.173863 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e862628-728a-465d-991f-3be01e339f44 admin admin] REQUEST_METHOD: `PUT` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:31.175072 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e862628-728a-465d-991f-3be01e339f44 admin admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:31.175621 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e862628-728a-465d-991f-3be01e339f44 admin admin] PATH_INFO: `/v3/projects/4c58f80628584a6283d262a9f9738004/users/114cb253416e48d0bd048b9d0f6fc550/roles/6c99f5d9f1f243eaa134d8540f21095d` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:31.187347 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 938/1869] 10.222.0.13 () {58 vars in 1224 bytes} [Wed Aug 31 09:48:30 2022] DELETE /identity/v3/users/d8efae832dba45f5b59d608730fd20a1 => generated 0 bytes in 295 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:48:31.260187 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e862628-728a-465d-991f-3be01e339f44 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4c58f80628584a6283d262a9f9738004, user_id=114cb253416e48d0bd048b9d0f6fc550, role_id=6c99f5d9f1f243eaa134d8540f21095d)` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:48:31.264042 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e862628-728a-465d-991f-3be01e339f44 admin admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:48:31.295204 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 932/1870] 10.222.0.13 () {60 vars in 1483 bytes} [Wed Aug 31 09:48:31 2022] PUT /identity/v3/projects/4c58f80628584a6283d262a9f9738004/users/114cb253416e48d0bd048b9d0f6fc550/roles/6c99f5d9f1f243eaa134d8540f21095d => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:48:31.315103 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5af18fc-b22e-43ad-9973-ae5839a5e549 None None] REQUEST_METHOD: `POST` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:31.317541 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5af18fc-b22e-43ad-9973-ae5839a5e549 None None] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:31.317541 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5af18fc-b22e-43ad-9973-ae5839a5e549 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:31.388529 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.auth.core [None req-b5af18fc-b22e-43ad-9973-ae5839a5e549 None None] MFA Rules not processed for user `114cb253416e48d0bd048b9d0f6fc550`. Rule list: `[]` (Enabled: `True`). {{(pid=88114) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:48:31.395112 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.fernet_utils [None req-b5af18fc-b22e-43ad-9973-ae5839a5e549 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88114) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:48:31.548865 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 939/1871] 10.222.0.13 () {58 vars in 958 bytes} [Wed Aug 31 09:48:31 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 235 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:48:31.557699 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e862628-728a-465d-991f-3be01e339f44 admin admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:48:31.559585 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.fernet_utils [None req-2e862628-728a-465d-991f-3be01e339f44 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:48:31.657592 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:48:31.772710 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e862628-728a-465d-991f-3be01e339f44 admin admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:48:31.783718 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0757b111-2ea9-495a-9b84-1541ee184278 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:48:31.785628 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0757b111-2ea9-495a-9b84-1541ee184278 admin admin] REQUEST_METHOD: `POST` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:31.786201 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0757b111-2ea9-495a-9b84-1541ee184278 admin admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:31.786744 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0757b111-2ea9-495a-9b84-1541ee184278 admin admin] PATH_INFO: `/v3/projects` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:31.809530 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0757b111-2ea9-495a-9b84-1541ee184278 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:48:31.820544 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0757b111-2ea9-495a-9b84-1541ee184278 admin admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:48:31.911410 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 933/1872] 10.222.0.13 () {60 vars in 1153 bytes} [Wed Aug 31 09:48:31 2022] POST /identity/v3/projects => generated 373 bytes in 356 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 09:48:31.917435 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5af18fc-b22e-43ad-9973-ae5839a5e549 None None] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:48:31.928137 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0757b111-2ea9-495a-9b84-1541ee184278 admin admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:48:31.933015 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.fernet_utils [None req-0757b111-2ea9-495a-9b84-1541ee184278 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:48:31.943664 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5af18fc-b22e-43ad-9973-ae5839a5e549 None None] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:48:32.130705 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a4a5ba8-c164-4880-8315-f527a962e3f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:48:32.133226 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4a5ba8-c164-4880-8315-f527a962e3f1 admin admin] REQUEST_METHOD: `POST` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:32.133765 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4a5ba8-c164-4880-8315-f527a962e3f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:32.134271 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4a5ba8-c164-4880-8315-f527a962e3f1 admin admin] PATH_INFO: `/v3/users` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:32.135660 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a4a5ba8-c164-4880-8315-f527a962e3f1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:48:32.141002 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a4a5ba8-c164-4880-8315-f527a962e3f1 admin admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:48:32.173109 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0757b111-2ea9-495a-9b84-1541ee184278 admin admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:48:32.179821 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1197be09-b576-4a22-aba6-27f525b9eb64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:48:32.180729 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1197be09-b576-4a22-aba6-27f525b9eb64 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:32.181160 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1197be09-b576-4a22-aba6-27f525b9eb64 admin admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:32.181606 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1197be09-b576-4a22-aba6-27f525b9eb64 admin admin] PATH_INFO: `/v3/projects/c429b68532d14cdebd9001948104fcb5` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:32.183034 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1197be09-b576-4a22-aba6-27f525b9eb64 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c429b68532d14cdebd9001948104fcb5)` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:48:32.185496 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1197be09-b576-4a22-aba6-27f525b9eb64 admin admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:48:32.192701 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 940/1873] 10.222.0.13 () {60 vars in 1144 bytes} [Wed Aug 31 09:48:31 2022] POST /identity/v3/users => generated 293 bytes in 277 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 09:48:32.199656 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a4a5ba8-c164-4880-8315-f527a962e3f1 admin admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:48:32.214525 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:48:32.225377 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a4a5ba8-c164-4880-8315-f527a962e3f1 admin admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:48:32.239866 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fa62276-b81e-4f45-833f-27862936b232 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:48:32.240949 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa62276-b81e-4f45-833f-27862936b232 admin admin] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:32.241997 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa62276-b81e-4f45-833f-27862936b232 admin admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:32.242889 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa62276-b81e-4f45-833f-27862936b232 admin admin] PATH_INFO: `/v3/roles` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:32.244193 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fa62276-b81e-4f45-833f-27862936b232 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:48:32.247565 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fa62276-b81e-4f45-833f-27862936b232 admin admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:48:32.270914 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 941/1874] 10.222.0.13 () {58 vars in 1122 bytes} [Wed Aug 31 09:48:32 2022] GET /identity/v3/roles => generated 1420 bytes in 72 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 09:48:32.274826 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.notifications [None req-1197be09-b576-4a22-aba6-27f525b9eb64 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c429b68532d14cdebd9001948104fcb5', 'request_id': 'req-1197be09-b576-4a22-aba6-27f525b9eb64'} {{(pid=88113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:48:32.277674 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fa62276-b81e-4f45-833f-27862936b232 admin admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:48:32.285037 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.notifications [None req-1197be09-b576-4a22-aba6-27f525b9eb64 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c429b68532d14cdebd9001948104fcb5', 'request_id': 'req-1197be09-b576-4a22-aba6-27f525b9eb64'} {{(pid=88113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:48:32.304852 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.notifications [None req-1197be09-b576-4a22-aba6-27f525b9eb64 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c429b68532d14cdebd9001948104fcb5', 'request_id': 'req-1197be09-b576-4a22-aba6-27f525b9eb64'} {{(pid=88113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:48:32.308246 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fa62276-b81e-4f45-833f-27862936b232 admin admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:48:32.321866 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.notifications [None req-1197be09-b576-4a22-aba6-27f525b9eb64 admin admin] The token cache is being invalidate because project c429b68532d14cdebd9001948104fcb5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88113) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 09:48:32.322168 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.notifications [None req-1197be09-b576-4a22-aba6-27f525b9eb64 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:48:32.324292 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.notifications [None req-1197be09-b576-4a22-aba6-27f525b9eb64 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:48:32.326311 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 934/1875] 10.222.0.13 () {58 vars in 1233 bytes} [Wed Aug 31 09:48:31 2022] DELETE /identity/v3/projects/c429b68532d14cdebd9001948104fcb5 => generated 0 bytes in 400 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 09:48:32.331391 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8f0302-b4e1-4d7c-86a5-b6007bfcc903 None None] REQUEST_METHOD: `POST` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:32.333891 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8f0302-b4e1-4d7c-86a5-b6007bfcc903 None None] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:32.334237 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8f0302-b4e1-4d7c-86a5-b6007bfcc903 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:32.363829 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.auth.core [None req-7d8f0302-b4e1-4d7c-86a5-b6007bfcc903 None None] MFA Rules not processed for user `fd836cf590a145df9acf19cf3ae2ac56`. Rule list: `[]` (Enabled: `True`). {{(pid=88113) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:48:32.366180 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.fernet_utils [None req-7d8f0302-b4e1-4d7c-86a5-b6007bfcc903 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:48:32.428292 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ef9302-a56f-4bba-bc82-034f78b73c57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:48:32.429517 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ef9302-a56f-4bba-bc82-034f78b73c57 admin admin] REQUEST_METHOD: `PUT` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:32.430047 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ef9302-a56f-4bba-bc82-034f78b73c57 admin admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:32.430510 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ef9302-a56f-4bba-bc82-034f78b73c57 admin admin] PATH_INFO: `/v3/projects/93a0948106c440bc8172dd99b955420c/users/af000563e032411fa5098260260b7962/roles/5d4ee63a3add47fc9a6b8b16d39e0a1f` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:32.467096 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:48:32.482793 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02ef9302-a56f-4bba-bc82-034f78b73c57 admin admin] RBAC: Authorizing `identity:create_grant(project_id=93a0948106c440bc8172dd99b955420c, user_id=af000563e032411fa5098260260b7962, role_id=5d4ee63a3add47fc9a6b8b16d39e0a1f)` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:48:32.488772 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02ef9302-a56f-4bba-bc82-034f78b73c57 admin admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:48:32.526744 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 935/1876] 10.222.0.13 () {58 vars in 958 bytes} [Wed Aug 31 09:48:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:48:32.529642 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b959d736-b373-4514-a1f0-d55804ffaaaf None None] REQUEST_METHOD: `POST` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:32.530105 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b959d736-b373-4514-a1f0-d55804ffaaaf None None] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:32.530563 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b959d736-b373-4514-a1f0-d55804ffaaaf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:32.539720 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 942/1877] 10.222.0.13 () {60 vars in 1483 bytes} [Wed Aug 31 09:48:32 2022] PUT /identity/v3/projects/93a0948106c440bc8172dd99b955420c/users/af000563e032411fa5098260260b7962/roles/5d4ee63a3add47fc9a6b8b16d39e0a1f => generated 0 bytes in 260 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:48:32.541311 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ef9302-a56f-4bba-bc82-034f78b73c57 admin admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:48:32.543525 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.fernet_utils [None req-02ef9302-a56f-4bba-bc82-034f78b73c57 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88114) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:48:32.567432 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.auth.core [None req-b959d736-b373-4514-a1f0-d55804ffaaaf None None] MFA Rules not processed for user `fd836cf590a145df9acf19cf3ae2ac56`. Rule list: `[]` (Enabled: `True`). {{(pid=88113) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:48:32.569191 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.fernet_utils [None req-b959d736-b373-4514-a1f0-d55804ffaaaf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:48:32.658238 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ef9302-a56f-4bba-bc82-034f78b73c57 admin admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:48:32.666415 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9871840-c917-4f3c-8c4c-e159a587919b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:48:32.667458 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9871840-c917-4f3c-8c4c-e159a587919b admin admin] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:32.667944 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9871840-c917-4f3c-8c4c-e159a587919b admin admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:32.668431 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9871840-c917-4f3c-8c4c-e159a587919b admin admin] PATH_INFO: `/v3/roles` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:32.669687 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9871840-c917-4f3c-8c4c-e159a587919b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:48:32.672900 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9871840-c917-4f3c-8c4c-e159a587919b admin admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:48:32.689202 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 936/1878] 10.222.0.13 () {58 vars in 958 bytes} [Wed Aug 31 09:48:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:48:32.695365 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 943/1879] 10.222.0.13 () {58 vars in 1122 bytes} [Wed Aug 31 09:48:32 2022] GET /identity/v3/roles => generated 1420 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 09:48:32.697047 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b959d736-b373-4514-a1f0-d55804ffaaaf None None] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:48:32.702085 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9871840-c917-4f3c-8c4c-e159a587919b admin admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:48:32.709548 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b959d736-b373-4514-a1f0-d55804ffaaaf None None] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:48:32.716368 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d22759a-1d0e-48ce-a3c4-56c11a61bc44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:48:32.717332 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d22759a-1d0e-48ce-a3c4-56c11a61bc44 admin admin] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:32.718054 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d22759a-1d0e-48ce-a3c4-56c11a61bc44 admin admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:32.718452 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d22759a-1d0e-48ce-a3c4-56c11a61bc44 admin admin] PATH_INFO: `/v3/domains` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:32.720179 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d22759a-1d0e-48ce-a3c4-56c11a61bc44 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88113) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:48:32.720179 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d22759a-1d0e-48ce-a3c4-56c11a61bc44 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:48:32.720867 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9871840-c917-4f3c-8c4c-e159a587919b admin admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:48:32.722907 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d22759a-1d0e-48ce-a3c4-56c11a61bc44 admin admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:48:32.730774 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4503aa1f-eae1-4124-9b72-588af38adb2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:48:32.731944 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4503aa1f-eae1-4124-9b72-588af38adb2b admin admin] REQUEST_METHOD: `PUT` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:32.732433 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4503aa1f-eae1-4124-9b72-588af38adb2b admin admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:32.732919 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4503aa1f-eae1-4124-9b72-588af38adb2b admin admin] PATH_INFO: `/v3/projects/93a0948106c440bc8172dd99b955420c/users/af000563e032411fa5098260260b7962/roles/5d4ee63a3add47fc9a6b8b16d39e0a1f` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:32.738244 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4503aa1f-eae1-4124-9b72-588af38adb2b admin admin] RBAC: Authorizing `identity:create_grant(project_id=93a0948106c440bc8172dd99b955420c, user_id=af000563e032411fa5098260260b7962, role_id=5d4ee63a3add47fc9a6b8b16d39e0a1f)` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:48:32.741364 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 937/1880] 10.222.0.13 () {58 vars in 1153 bytes} [Wed Aug 31 09:48:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 09:48:32.742977 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4503aa1f-eae1-4124-9b72-588af38adb2b admin admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:48:32.747799 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d22759a-1d0e-48ce-a3c4-56c11a61bc44 admin admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:48:32.761008 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 944/1881] 10.222.0.13 () {60 vars in 1483 bytes} [Wed Aug 31 09:48:32 2022] PUT /identity/v3/projects/93a0948106c440bc8172dd99b955420c/users/af000563e032411fa5098260260b7962/roles/5d4ee63a3add47fc9a6b8b16d39e0a1f => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:48:32.764306 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d22759a-1d0e-48ce-a3c4-56c11a61bc44 admin admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:48:32.781309 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22144c61-1886-4707-824b-ac77c3439f7a None None] REQUEST_METHOD: `POST` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:32.781873 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22144c61-1886-4707-824b-ac77c3439f7a None None] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:32.782324 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22144c61-1886-4707-824b-ac77c3439f7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:32.845204 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.auth.core [None req-22144c61-1886-4707-824b-ac77c3439f7a None None] MFA Rules not processed for user `af000563e032411fa5098260260b7962`. Rule list: `[]` (Enabled: `True`). {{(pid=88114) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:48:32.847612 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.fernet_utils [None req-22144c61-1886-4707-824b-ac77c3439f7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88114) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:48:32.873325 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a72c9d13-bb23-4fc1-9d91-c6319512a1d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:48:32.874257 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72c9d13-bb23-4fc1-9d91-c6319512a1d1 admin admin] REQUEST_METHOD: `POST` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:32.874936 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72c9d13-bb23-4fc1-9d91-c6319512a1d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:32.875424 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72c9d13-bb23-4fc1-9d91-c6319512a1d1 admin admin] PATH_INFO: `/v3/projects` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:32.876515 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a72c9d13-bb23-4fc1-9d91-c6319512a1d1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:48:32.879966 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a72c9d13-bb23-4fc1-9d91-c6319512a1d1 admin admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:48:32.929465 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 938/1882] 10.222.0.13 () {60 vars in 1153 bytes} [Wed Aug 31 09:48:32 2022] POST /identity/v3/projects => generated 355 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 09:48:32.934894 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a72c9d13-bb23-4fc1-9d91-c6319512a1d1 admin admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:48:32.940324 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:48:32.994336 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 945/1883] 10.222.0.13 () {58 vars in 958 bytes} [Wed Aug 31 09:48:32 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 215 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:48:33.008561 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22144c61-1886-4707-824b-ac77c3439f7a None None] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:48:33.010081 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.fernet_utils [None req-22144c61-1886-4707-824b-ac77c3439f7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88114) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:48:33.050089 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:48:33.055430 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a72c9d13-bb23-4fc1-9d91-c6319512a1d1 admin admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:48:33.065415 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea7f075-db41-4244-9a3f-4620dd289a5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:48:33.068990 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea7f075-db41-4244-9a3f-4620dd289a5f admin admin] REQUEST_METHOD: `POST` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:33.069758 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea7f075-db41-4244-9a3f-4620dd289a5f admin admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:33.070271 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea7f075-db41-4244-9a3f-4620dd289a5f admin admin] PATH_INFO: `/v3/users` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:33.072024 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ea7f075-db41-4244-9a3f-4620dd289a5f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:48:33.078454 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ea7f075-db41-4244-9a3f-4620dd289a5f admin admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:48:33.106320 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 939/1884] 10.222.0.13 () {60 vars in 1144 bytes} [Wed Aug 31 09:48:32 2022] POST /identity/v3/users => generated 278 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 09:48:33.111634 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22144c61-1886-4707-824b-ac77c3439f7a None None] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:48:33.114856 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea7f075-db41-4244-9a3f-4620dd289a5f admin admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:48:33.131419 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea7f075-db41-4244-9a3f-4620dd289a5f admin admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:48:33.138736 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a2dfa3-3d4c-43f8-83d8-8759a6b5cd1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:48:33.139715 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a2dfa3-3d4c-43f8-83d8-8759a6b5cd1d admin admin] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:33.140177 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a2dfa3-3d4c-43f8-83d8-8759a6b5cd1d admin admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:33.141096 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a2dfa3-3d4c-43f8-83d8-8759a6b5cd1d admin admin] PATH_INFO: `/v3/roles` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:33.141755 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3a2dfa3-3d4c-43f8-83d8-8759a6b5cd1d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:48:33.144551 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3a2dfa3-3d4c-43f8-83d8-8759a6b5cd1d admin admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:48:33.158057 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 940/1885] 10.222.0.13 () {58 vars in 1122 bytes} [Wed Aug 31 09:48:33 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 09:48:33.163580 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a2dfa3-3d4c-43f8-83d8-8759a6b5cd1d admin admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:48:33.180792 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a2dfa3-3d4c-43f8-83d8-8759a6b5cd1d admin admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:48:33.187609 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd27b8a-668b-4b23-9f9d-98d05feb0465 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:48:33.188584 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd27b8a-668b-4b23-9f9d-98d05feb0465 admin admin] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:33.188960 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd27b8a-668b-4b23-9f9d-98d05feb0465 admin admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:33.189446 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd27b8a-668b-4b23-9f9d-98d05feb0465 admin admin] PATH_INFO: `/v3/roles` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:33.190516 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddd27b8a-668b-4b23-9f9d-98d05feb0465 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:48:33.194539 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddd27b8a-668b-4b23-9f9d-98d05feb0465 admin admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:48:33.197324 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d491c1e-52a4-4a33-aa37-4b927d1a7a0e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d52b69c8e94441dacfcff944326f169', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '49548eb44d62415582af820ab66decee', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:48:33.198357 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d491c1e-52a4-4a33-aa37-4b927d1a7a0e service cinder] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:33.198820 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d491c1e-52a4-4a33-aa37-4b927d1a7a0e service cinder] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:33.199270 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d491c1e-52a4-4a33-aa37-4b927d1a7a0e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:33.202341 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.fernet_utils [None req-2d491c1e-52a4-4a33-aa37-4b927d1a7a0e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88114) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:48:33.214454 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 941/1886] 10.222.0.13 () {58 vars in 1122 bytes} [Wed Aug 31 09:48:33 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 09:48:33.219333 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd27b8a-668b-4b23-9f9d-98d05feb0465 admin admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:48:33.236118 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd27b8a-668b-4b23-9f9d-98d05feb0465 admin admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:48:33.244647 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c323cb-55cd-4ec6-a336-173cd231d127 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:48:33.245603 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c323cb-55cd-4ec6-a336-173cd231d127 admin admin] REQUEST_METHOD: `PUT` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:33.246051 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c323cb-55cd-4ec6-a336-173cd231d127 admin admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:33.246496 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c323cb-55cd-4ec6-a336-173cd231d127 admin admin] PATH_INFO: `/v3/projects/f132219bdfb14b5395c5cf3811b1b57c/users/a79193bb8fb04786aeb2acfb9d1a77b6/roles/6c99f5d9f1f243eaa134d8540f21095d` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:33.271833 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79c323cb-55cd-4ec6-a336-173cd231d127 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f132219bdfb14b5395c5cf3811b1b57c, user_id=a79193bb8fb04786aeb2acfb9d1a77b6, role_id=6c99f5d9f1f243eaa134d8540f21095d)` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:48:33.275840 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79c323cb-55cd-4ec6-a336-173cd231d127 admin admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:48:33.299300 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 942/1887] 10.222.0.13 () {60 vars in 1483 bytes} [Wed Aug 31 09:48:33 2022] PUT /identity/v3/projects/f132219bdfb14b5395c5cf3811b1b57c/users/a79193bb8fb04786aeb2acfb9d1a77b6/roles/6c99f5d9f1f243eaa134d8540f21095d => generated 0 bytes in 81 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 09:48:33.302080 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d491c1e-52a4-4a33-aa37-4b927d1a7a0e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:48:33.303641 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d491c1e-52a4-4a33-aa37-4b927d1a7a0e service cinder] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:48:33.314327 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 946/1888] 10.222.0.13 () {60 vars in 1365 bytes} [Wed Aug 31 09:48:33 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 09:48:33.321025 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855b6299-ab5e-4c57-8049-9a1447440be8 None None] REQUEST_METHOD: `POST` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:33.321569 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855b6299-ab5e-4c57-8049-9a1447440be8 None None] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:33.322112 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855b6299-ab5e-4c57-8049-9a1447440be8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:33.377941 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.auth.core [None req-855b6299-ab5e-4c57-8049-9a1447440be8 None None] MFA Rules not processed for user `a79193bb8fb04786aeb2acfb9d1a77b6`. Rule list: `[]` (Enabled: `True`). {{(pid=88113) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:48:33.380172 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.fernet_utils [None req-855b6299-ab5e-4c57-8049-9a1447440be8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:48:33.494169 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 943/1889] 10.222.0.13 () {58 vars in 958 bytes} [Wed Aug 31 09:48:33 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:48:33.503602 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d491c1e-52a4-4a33-aa37-4b927d1a7a0e service cinder] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:48:33.528546 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d491c1e-52a4-4a33-aa37-4b927d1a7a0e service cinder] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:48:33.681564 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=100 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:48:33.714136 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62013eb4-d498-4d15-9d01-48fd52b323e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:48:33.718130 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62013eb4-d498-4d15-9d01-48fd52b323e3 admin admin] REQUEST_METHOD: `POST` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:33.719704 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62013eb4-d498-4d15-9d01-48fd52b323e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:33.720261 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62013eb4-d498-4d15-9d01-48fd52b323e3 admin admin] PATH_INFO: `/v3/projects` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:33.721533 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62013eb4-d498-4d15-9d01-48fd52b323e3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:48:33.725590 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62013eb4-d498-4d15-9d01-48fd52b323e3 admin admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:48:33.788671 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 947/1890] 10.222.0.13 () {60 vars in 1153 bytes} [Wed Aug 31 09:48:33 2022] POST /identity/v3/projects => generated 357 bytes in 289 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 09:48:33.795125 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-855b6299-ab5e-4c57-8049-9a1447440be8 None None] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:48:33.812160 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-855b6299-ab5e-4c57-8049-9a1447440be8 None None] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:48:33.951257 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e40f151b-7c4e-445c-994f-dacf51d8ade8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:48:33.951970 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40f151b-7c4e-445c-994f-dacf51d8ade8 admin admin] REQUEST_METHOD: `POST` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:33.952495 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40f151b-7c4e-445c-994f-dacf51d8ade8 admin admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:33.952828 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40f151b-7c4e-445c-994f-dacf51d8ade8 admin admin] PATH_INFO: `/v3/users` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:33.953571 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e40f151b-7c4e-445c-994f-dacf51d8ade8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:48:33.955806 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e40f151b-7c4e-445c-994f-dacf51d8ade8 admin admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:48:34.009305 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 944/1891] 10.222.0.13 () {60 vars in 1144 bytes} [Wed Aug 31 09:48:33 2022] POST /identity/v3/users => generated 285 bytes in 213 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 09:48:34.012536 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62013eb4-d498-4d15-9d01-48fd52b323e3 admin admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:48:34.027354 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62013eb4-d498-4d15-9d01-48fd52b323e3 admin admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:48:34.033297 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a588f17-24aa-462a-a259-8febe44c1e45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:48:34.034262 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a588f17-24aa-462a-a259-8febe44c1e45 admin admin] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:34.034741 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a588f17-24aa-462a-a259-8febe44c1e45 admin admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:34.035990 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a588f17-24aa-462a-a259-8febe44c1e45 admin admin] PATH_INFO: `/v3/roles` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:34.036940 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a588f17-24aa-462a-a259-8febe44c1e45 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:48:34.038769 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a588f17-24aa-462a-a259-8febe44c1e45 admin admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:48:34.052538 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 948/1892] 10.222.0.13 () {58 vars in 1122 bytes} [Wed Aug 31 09:48:34 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 09:48:34.058637 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e40f151b-7c4e-445c-994f-dacf51d8ade8 admin admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:48:34.078362 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e40f151b-7c4e-445c-994f-dacf51d8ade8 admin admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:48:34.088227 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2476108-9dc4-4cb0-acbc-b2f0fb6282fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:48:34.090076 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2476108-9dc4-4cb0-acbc-b2f0fb6282fa admin admin] REQUEST_METHOD: `PUT` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:34.090626 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2476108-9dc4-4cb0-acbc-b2f0fb6282fa admin admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:34.091332 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2476108-9dc4-4cb0-acbc-b2f0fb6282fa admin admin] PATH_INFO: `/v3/projects/e2d1e8d7858849eaaf92de0406282ac8/users/1d7b408a9b804ccca57fce60fc527e7c/roles/5d4ee63a3add47fc9a6b8b16d39e0a1f` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:34.125316 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2476108-9dc4-4cb0-acbc-b2f0fb6282fa admin admin] RBAC: Authorizing `identity:create_grant(project_id=e2d1e8d7858849eaaf92de0406282ac8, user_id=1d7b408a9b804ccca57fce60fc527e7c, role_id=5d4ee63a3add47fc9a6b8b16d39e0a1f)` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:48:34.130467 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2476108-9dc4-4cb0-acbc-b2f0fb6282fa admin admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:48:34.135228 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:48:34.154746 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 945/1893] 10.222.0.13 () {60 vars in 1483 bytes} [Wed Aug 31 09:48:34 2022] PUT /identity/v3/projects/e2d1e8d7858849eaaf92de0406282ac8/users/1d7b408a9b804ccca57fce60fc527e7c/roles/5d4ee63a3add47fc9a6b8b16d39e0a1f => generated 0 bytes in 97 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:48:34.161415 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a588f17-24aa-462a-a259-8febe44c1e45 admin admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:48:34.174550 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a588f17-24aa-462a-a259-8febe44c1e45 admin admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:48:34.284252 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb64188f-8dd5-4c63-b183-fdcdae9bdd65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:48:34.285834 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb64188f-8dd5-4c63-b183-fdcdae9bdd65 admin admin] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:34.286291 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb64188f-8dd5-4c63-b183-fdcdae9bdd65 admin admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:34.286747 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb64188f-8dd5-4c63-b183-fdcdae9bdd65 admin admin] PATH_INFO: `/v3/roles` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:34.287906 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb64188f-8dd5-4c63-b183-fdcdae9bdd65 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:48:34.291115 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb64188f-8dd5-4c63-b183-fdcdae9bdd65 admin admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:48:34.312113 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 949/1894] 10.222.0.13 () {58 vars in 1122 bytes} [Wed Aug 31 09:48:34 2022] GET /identity/v3/roles => generated 1420 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 09:48:34.318559 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2476108-9dc4-4cb0-acbc-b2f0fb6282fa admin admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:48:34.336440 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2476108-9dc4-4cb0-acbc-b2f0fb6282fa admin admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:48:34.344563 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af75a843-5437-46fe-ad54-f67a495122db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:48:34.345712 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af75a843-5437-46fe-ad54-f67a495122db admin admin] REQUEST_METHOD: `PUT` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:34.346355 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af75a843-5437-46fe-ad54-f67a495122db admin admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:34.346953 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af75a843-5437-46fe-ad54-f67a495122db admin admin] PATH_INFO: `/v3/projects/e2d1e8d7858849eaaf92de0406282ac8/users/1d7b408a9b804ccca57fce60fc527e7c/roles/5d4ee63a3add47fc9a6b8b16d39e0a1f` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:34.351739 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af75a843-5437-46fe-ad54-f67a495122db admin admin] RBAC: Authorizing `identity:create_grant(project_id=e2d1e8d7858849eaaf92de0406282ac8, user_id=1d7b408a9b804ccca57fce60fc527e7c, role_id=5d4ee63a3add47fc9a6b8b16d39e0a1f)` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:48:34.357780 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af75a843-5437-46fe-ad54-f67a495122db admin admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:48:34.374123 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 946/1895] 10.222.0.13 () {60 vars in 1483 bytes} [Wed Aug 31 09:48:34 2022] PUT /identity/v3/projects/e2d1e8d7858849eaaf92de0406282ac8/users/1d7b408a9b804ccca57fce60fc527e7c/roles/5d4ee63a3add47fc9a6b8b16d39e0a1f => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:48:34.385737 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e5bf34-2b6d-4e3a-8e6c-e2b87f55828d None None] REQUEST_METHOD: `POST` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:34.386025 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e5bf34-2b6d-4e3a-8e6c-e2b87f55828d None None] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:34.386380 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e5bf34-2b6d-4e3a-8e6c-e2b87f55828d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:34.422870 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.auth.core [None req-f7e5bf34-2b6d-4e3a-8e6c-e2b87f55828d None None] MFA Rules not processed for user `1d7b408a9b804ccca57fce60fc527e7c`. Rule list: `[]` (Enabled: `True`). {{(pid=88114) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:48:34.424551 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.fernet_utils [None req-f7e5bf34-2b6d-4e3a-8e6c-e2b87f55828d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88114) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:48:34.544370 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 950/1896] 10.222.0.13 () {58 vars in 958 bytes} [Wed Aug 31 09:48:34 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 09:48:34.558497 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af75a843-5437-46fe-ad54-f67a495122db admin admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:48:34.560893 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.fernet_utils [None req-af75a843-5437-46fe-ad54-f67a495122db admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:48:34.653736 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af75a843-5437-46fe-ad54-f67a495122db admin admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:48:34.660273 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-378daf15-6855-4310-a9e5-9996a67466f5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d52b69c8e94441dacfcff944326f169', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '49548eb44d62415582af820ab66decee', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:48:34.661089 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378daf15-6855-4310-a9e5-9996a67466f5 service cinder] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:34.661551 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378daf15-6855-4310-a9e5-9996a67466f5 service cinder] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:34.661935 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378daf15-6855-4310-a9e5-9996a67466f5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:34.677171 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-378daf15-6855-4310-a9e5-9996a67466f5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:48:34.679290 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-378daf15-6855-4310-a9e5-9996a67466f5 service cinder] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:48:34.693317 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 947/1897] 10.222.0.13 () {60 vars in 1365 bytes} [Wed Aug 31 09:48:34 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 09:48:34.859100 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e5bf34-2b6d-4e3a-8e6c-e2b87f55828d None None] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:48:34.888218 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e5bf34-2b6d-4e3a-8e6c-e2b87f55828d None None] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:48:34.895514 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-501610cc-a635-4ae5-8289-b2728454593c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d52b69c8e94441dacfcff944326f169', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '49548eb44d62415582af820ab66decee', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:48:34.897034 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501610cc-a635-4ae5-8289-b2728454593c service cinder] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:34.897564 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501610cc-a635-4ae5-8289-b2728454593c service cinder] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:34.898045 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501610cc-a635-4ae5-8289-b2728454593c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:34.985653 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:48:34.998221 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-501610cc-a635-4ae5-8289-b2728454593c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:48:35.002244 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-501610cc-a635-4ae5-8289-b2728454593c service cinder] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:48:35.023916 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 951/1898] 10.222.0.13 () {60 vars in 1365 bytes} [Wed Aug 31 09:48:34 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 165 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:48:36.762607 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-378daf15-6855-4310-a9e5-9996a67466f5 service cinder] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:48:36.765024 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.fernet_utils [None req-378daf15-6855-4310-a9e5-9996a67466f5 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:48:36.891923 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-378daf15-6855-4310-a9e5-9996a67466f5 service cinder] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:48:36.900385 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3162b96c-57c1-4355-a586-7ddf52fa62ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:48:36.901043 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3162b96c-57c1-4355-a586-7ddf52fa62ae admin admin] REQUEST_METHOD: `DELETE` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:36.901558 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3162b96c-57c1-4355-a586-7ddf52fa62ae admin admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:36.902089 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3162b96c-57c1-4355-a586-7ddf52fa62ae admin admin] PATH_INFO: `/v3/users/297786c3e5db4d2890d09b922bea7dd2` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:36.903349 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3162b96c-57c1-4355-a586-7ddf52fa62ae admin admin] RBAC: Authorizing `identity:delete_user(user_id=297786c3e5db4d2890d09b922bea7dd2)` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:48:36.905775 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3162b96c-57c1-4355-a586-7ddf52fa62ae admin admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:48:37.012345 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.notifications [None req-3162b96c-57c1-4355-a586-7ddf52fa62ae admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '297786c3e5db4d2890d09b922bea7dd2'} {{(pid=88113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:48:37.031570 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.notifications [None req-3162b96c-57c1-4355-a586-7ddf52fa62ae admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '297786c3e5db4d2890d09b922bea7dd2'} {{(pid=88113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:48:37.062849 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.notifications [None req-3162b96c-57c1-4355-a586-7ddf52fa62ae admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '297786c3e5db4d2890d09b922bea7dd2'} {{(pid=88113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:48:37.070906 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.notifications [None req-3162b96c-57c1-4355-a586-7ddf52fa62ae admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '297786c3e5db4d2890d09b922bea7dd2'} {{(pid=88113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:48:37.083045 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.notifications [None req-3162b96c-57c1-4355-a586-7ddf52fa62ae admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '297786c3e5db4d2890d09b922bea7dd2'} {{(pid=88113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:48:37.096300 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 948/1899] 10.222.0.13 () {58 vars in 1224 bytes} [Wed Aug 31 09:48:36 2022] DELETE /identity/v3/users/297786c3e5db4d2890d09b922bea7dd2 => generated 0 bytes in 335 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:48:37.104496 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54b32de-113c-42c4-9794-f4deaa086d40 None None] REQUEST_METHOD: `POST` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:37.105313 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54b32de-113c-42c4-9794-f4deaa086d40 None None] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:37.105498 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54b32de-113c-42c4-9794-f4deaa086d40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:37.141083 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.auth.core [None req-e54b32de-113c-42c4-9794-f4deaa086d40 None None] MFA Rules not processed for user `fd836cf590a145df9acf19cf3ae2ac56`. Rule list: `[]` (Enabled: `True`). {{(pid=88114) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:48:37.143632 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.fernet_utils [None req-e54b32de-113c-42c4-9794-f4deaa086d40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88114) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:48:37.281886 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 952/1900] 10.222.0.13 () {58 vars in 958 bytes} [Wed Aug 31 09:48:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:48:37.299140 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3162b96c-57c1-4355-a586-7ddf52fa62ae admin admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:48:37.302037 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.fernet_utils [None req-3162b96c-57c1-4355-a586-7ddf52fa62ae admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:48:37.313235 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:48:37.408742 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e54b32de-113c-42c4-9794-f4deaa086d40 None None] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:48:37.411939 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.fernet_utils [None req-e54b32de-113c-42c4-9794-f4deaa086d40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88114) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:48:37.423032 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3162b96c-57c1-4355-a586-7ddf52fa62ae admin admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:48:37.435024 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a9131a-fd49-4d8d-bca5-28433e9230a0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9bdaacd597442c2bcde2c555f2cc140', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '49548eb44d62415582af820ab66decee', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:48:37.436672 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a9131a-fd49-4d8d-bca5-28433e9230a0 service neutron] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:37.437613 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a9131a-fd49-4d8d-bca5-28433e9230a0 service neutron] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:37.438895 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e54b32de-113c-42c4-9794-f4deaa086d40 None None] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:48:37.439669 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a9131a-fd49-4d8d-bca5-28433e9230a0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:37.445653 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f93207ca-f90b-4bcf-820b-4cb8bfeb0e57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:48:37.446701 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93207ca-f90b-4bcf-820b-4cb8bfeb0e57 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:37.447182 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93207ca-f90b-4bcf-820b-4cb8bfeb0e57 admin admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:37.447657 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93207ca-f90b-4bcf-820b-4cb8bfeb0e57 admin admin] PATH_INFO: `/v3/users/114cb253416e48d0bd048b9d0f6fc550` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:37.449136 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f93207ca-f90b-4bcf-820b-4cb8bfeb0e57 admin admin] RBAC: Authorizing `identity:delete_user(user_id=114cb253416e48d0bd048b9d0f6fc550)` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:48:37.451815 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f93207ca-f90b-4bcf-820b-4cb8bfeb0e57 admin admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:48:37.461101 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99a9131a-fd49-4d8d-bca5-28433e9230a0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:48:37.463772 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99a9131a-fd49-4d8d-bca5-28433e9230a0 service neutron] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:48:37.487024 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 949/1901] 10.222.0.13 () {60 vars in 1365 bytes} [Wed Aug 31 09:48:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 189 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:48:37.564594 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.notifications [None req-f93207ca-f90b-4bcf-820b-4cb8bfeb0e57 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '114cb253416e48d0bd048b9d0f6fc550'} {{(pid=88114) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:48:37.618161 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.notifications [None req-f93207ca-f90b-4bcf-820b-4cb8bfeb0e57 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '114cb253416e48d0bd048b9d0f6fc550'} {{(pid=88114) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:48:37.655706 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.notifications [None req-f93207ca-f90b-4bcf-820b-4cb8bfeb0e57 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '114cb253416e48d0bd048b9d0f6fc550'} {{(pid=88114) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:48:37.656978 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.notifications [None req-f93207ca-f90b-4bcf-820b-4cb8bfeb0e57 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '114cb253416e48d0bd048b9d0f6fc550'} {{(pid=88114) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:48:37.706585 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.notifications [None req-f93207ca-f90b-4bcf-820b-4cb8bfeb0e57 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '114cb253416e48d0bd048b9d0f6fc550'} {{(pid=88114) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:48:37.752773 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 953/1902] 10.222.0.13 () {58 vars in 1224 bytes} [Wed Aug 31 09:48:37 2022] DELETE /identity/v3/users/114cb253416e48d0bd048b9d0f6fc550 => generated 0 bytes in 345 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:48:37.764658 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233db8d3-5dbd-42a7-942f-d45951a8fd79 None None] REQUEST_METHOD: `POST` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:37.765417 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233db8d3-5dbd-42a7-942f-d45951a8fd79 None None] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:37.765718 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233db8d3-5dbd-42a7-942f-d45951a8fd79 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:37.804568 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.auth.core [None req-233db8d3-5dbd-42a7-942f-d45951a8fd79 None None] MFA Rules not processed for user `fd836cf590a145df9acf19cf3ae2ac56`. Rule list: `[]` (Enabled: `True`). {{(pid=88113) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:48:37.806363 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.fernet_utils [None req-233db8d3-5dbd-42a7-942f-d45951a8fd79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:48:37.882782 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 950/1903] 10.222.0.13 () {58 vars in 958 bytes} [Wed Aug 31 09:48:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:48:37.906957 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f93207ca-f90b-4bcf-820b-4cb8bfeb0e57 admin admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:48:37.908617 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.fernet_utils [None req-f93207ca-f90b-4bcf-820b-4cb8bfeb0e57 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88114) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:48:37.990264 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-233db8d3-5dbd-42a7-942f-d45951a8fd79 None None] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:48:37.992515 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.fernet_utils [None req-233db8d3-5dbd-42a7-942f-d45951a8fd79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:48:38.053858 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-233db8d3-5dbd-42a7-942f-d45951a8fd79 None None] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:48:38.070211 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acdf13b1-33e3-4550-a84b-8a1e5ad8de87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:48:38.072189 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acdf13b1-33e3-4550-a84b-8a1e5ad8de87 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:38.073265 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acdf13b1-33e3-4550-a84b-8a1e5ad8de87 admin admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:38.074128 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acdf13b1-33e3-4550-a84b-8a1e5ad8de87 admin admin] PATH_INFO: `/v3/users/41c204e476a1402ba8b056065184dd15` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:38.077200 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acdf13b1-33e3-4550-a84b-8a1e5ad8de87 admin admin] RBAC: Authorizing `identity:delete_user(user_id=41c204e476a1402ba8b056065184dd15)` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:48:38.096747 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acdf13b1-33e3-4550-a84b-8a1e5ad8de87 admin admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:48:38.131420 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f93207ca-f90b-4bcf-820b-4cb8bfeb0e57 admin admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:48:38.154723 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-253e77f0-7da3-4ad9-8604-32e3ab47da52 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9bdaacd597442c2bcde2c555f2cc140', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '49548eb44d62415582af820ab66decee', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:48:38.156299 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253e77f0-7da3-4ad9-8604-32e3ab47da52 service neutron] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:38.156935 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253e77f0-7da3-4ad9-8604-32e3ab47da52 service neutron] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:38.157650 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253e77f0-7da3-4ad9-8604-32e3ab47da52 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:38.232571 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-253e77f0-7da3-4ad9-8604-32e3ab47da52 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:48:38.234983 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-253e77f0-7da3-4ad9-8604-32e3ab47da52 service neutron] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:48:38.290035 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 954/1904] 10.222.0.13 () {60 vars in 1365 bytes} [Wed Aug 31 09:48:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 383 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:48:38.292129 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-253e77f0-7da3-4ad9-8604-32e3ab47da52 service neutron] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:48:38.295169 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.fernet_utils [None req-253e77f0-7da3-4ad9-8604-32e3ab47da52 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88114) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:48:38.336989 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:48:38.358678 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-253e77f0-7da3-4ad9-8604-32e3ab47da52 service neutron] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:48:38.389932 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-803f9c2c-0bd0-4b1e-b577-9bde07074a1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:48:38.391145 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803f9c2c-0bd0-4b1e-b577-9bde07074a1a admin admin] REQUEST_METHOD: `DELETE` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:38.391552 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803f9c2c-0bd0-4b1e-b577-9bde07074a1a admin admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:38.391990 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803f9c2c-0bd0-4b1e-b577-9bde07074a1a admin admin] PATH_INFO: `/v3/projects/32cd034c0d60483eb19614049ccefb41` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:38.395425 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.notifications [None req-acdf13b1-33e3-4550-a84b-8a1e5ad8de87 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '41c204e476a1402ba8b056065184dd15'} {{(pid=88113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:48:38.398986 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-803f9c2c-0bd0-4b1e-b577-9bde07074a1a admin admin] RBAC: Authorizing `identity:delete_project(project_id=32cd034c0d60483eb19614049ccefb41)` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:48:38.412481 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-803f9c2c-0bd0-4b1e-b577-9bde07074a1a admin admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:48:38.433208 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.notifications [None req-acdf13b1-33e3-4550-a84b-8a1e5ad8de87 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '41c204e476a1402ba8b056065184dd15'} {{(pid=88113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:48:38.514925 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.notifications [None req-acdf13b1-33e3-4550-a84b-8a1e5ad8de87 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '41c204e476a1402ba8b056065184dd15'} {{(pid=88113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:48:38.517107 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.notifications [None req-acdf13b1-33e3-4550-a84b-8a1e5ad8de87 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '41c204e476a1402ba8b056065184dd15'} {{(pid=88113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:48:38.550276 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.notifications [None req-acdf13b1-33e3-4550-a84b-8a1e5ad8de87 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '41c204e476a1402ba8b056065184dd15'} {{(pid=88113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:48:38.598022 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 951/1905] 10.222.0.13 () {58 vars in 1224 bytes} [Wed Aug 31 09:48:37 2022] DELETE /identity/v3/users/41c204e476a1402ba8b056065184dd15 => generated 0 bytes in 608 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:48:38.605061 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23234050-d597-4370-bb20-c985e83c524b None None] REQUEST_METHOD: `POST` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:38.608824 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23234050-d597-4370-bb20-c985e83c524b None None] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:38.609424 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23234050-d597-4370-bb20-c985e83c524b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:38.664649 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.notifications [None req-803f9c2c-0bd0-4b1e-b577-9bde07074a1a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '32cd034c0d60483eb19614049ccefb41', 'request_id': 'req-803f9c2c-0bd0-4b1e-b577-9bde07074a1a'} {{(pid=88114) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:48:38.673230 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.auth.core [None req-23234050-d597-4370-bb20-c985e83c524b None None] MFA Rules not processed for user `fd836cf590a145df9acf19cf3ae2ac56`. Rule list: `[]` (Enabled: `True`). {{(pid=88113) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:48:38.674099 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.fernet_utils [None req-23234050-d597-4370-bb20-c985e83c524b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:48:38.688069 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.notifications [None req-803f9c2c-0bd0-4b1e-b577-9bde07074a1a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '32cd034c0d60483eb19614049ccefb41', 'request_id': 'req-803f9c2c-0bd0-4b1e-b577-9bde07074a1a'} {{(pid=88114) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:48:38.709368 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:48:38.727377 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.notifications [None req-803f9c2c-0bd0-4b1e-b577-9bde07074a1a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '32cd034c0d60483eb19614049ccefb41', 'request_id': 'req-803f9c2c-0bd0-4b1e-b577-9bde07074a1a'} {{(pid=88114) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:48:38.777915 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.notifications [None req-803f9c2c-0bd0-4b1e-b577-9bde07074a1a admin admin] The token cache is being invalidate because project 32cd034c0d60483eb19614049ccefb41 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88114) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 09:48:38.781117 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.notifications [None req-803f9c2c-0bd0-4b1e-b577-9bde07074a1a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88114) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:48:38.782919 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.notifications [None req-803f9c2c-0bd0-4b1e-b577-9bde07074a1a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88114) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:48:38.794270 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 955/1906] 10.222.0.13 () {58 vars in 1233 bytes} [Wed Aug 31 09:48:38 2022] DELETE /identity/v3/projects/32cd034c0d60483eb19614049ccefb41 => generated 0 bytes in 504 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:48:38.805615 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-803f9c2c-0bd0-4b1e-b577-9bde07074a1a admin admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:48:38.812049 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.fernet_utils [None req-803f9c2c-0bd0-4b1e-b577-9bde07074a1a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88114) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:48:38.820932 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 952/1907] 10.222.0.13 () {58 vars in 958 bytes} [Wed Aug 31 09:48:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 219 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:48:38.877168 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23234050-d597-4370-bb20-c985e83c524b None None] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:48:38.880367 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.fernet_utils [None req-23234050-d597-4370-bb20-c985e83c524b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:48:39.125424 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23234050-d597-4370-bb20-c985e83c524b None None] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:48:39.150398 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dfe999d-1c09-4c95-a181-f8d00b8985e7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9bdaacd597442c2bcde2c555f2cc140', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '49548eb44d62415582af820ab66decee', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:48:39.153618 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dfe999d-1c09-4c95-a181-f8d00b8985e7 service neutron] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:39.154465 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dfe999d-1c09-4c95-a181-f8d00b8985e7 service neutron] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:39.156628 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dfe999d-1c09-4c95-a181-f8d00b8985e7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:39.197731 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-803f9c2c-0bd0-4b1e-b577-9bde07074a1a admin admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:48:39.205281 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dfe999d-1c09-4c95-a181-f8d00b8985e7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:48:39.216393 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dfe999d-1c09-4c95-a181-f8d00b8985e7 service neutron] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:48:39.224160 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c613d6e9-ebd6-40ea-901d-f4816787c96e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:48:39.230258 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c613d6e9-ebd6-40ea-901d-f4816787c96e admin admin] REQUEST_METHOD: `DELETE` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:39.230983 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c613d6e9-ebd6-40ea-901d-f4816787c96e admin admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:39.231675 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c613d6e9-ebd6-40ea-901d-f4816787c96e admin admin] PATH_INFO: `/v3/users/464bb24610c34dfc8ff3fb7be1920ad9` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:39.237702 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c613d6e9-ebd6-40ea-901d-f4816787c96e admin admin] RBAC: Authorizing `identity:delete_user(user_id=464bb24610c34dfc8ff3fb7be1920ad9)` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:48:39.244194 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c613d6e9-ebd6-40ea-901d-f4816787c96e admin admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:48:39.271306 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 953/1908] 10.222.0.13 () {60 vars in 1365 bytes} [Wed Aug 31 09:48:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 396 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:48:39.441871 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.notifications [None req-c613d6e9-ebd6-40ea-901d-f4816787c96e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '464bb24610c34dfc8ff3fb7be1920ad9'} {{(pid=88114) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:48:39.458543 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.notifications [None req-c613d6e9-ebd6-40ea-901d-f4816787c96e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '464bb24610c34dfc8ff3fb7be1920ad9'} {{(pid=88114) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:48:39.472104 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.notifications [None req-c613d6e9-ebd6-40ea-901d-f4816787c96e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '464bb24610c34dfc8ff3fb7be1920ad9'} {{(pid=88114) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:48:39.473922 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.notifications [None req-c613d6e9-ebd6-40ea-901d-f4816787c96e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '464bb24610c34dfc8ff3fb7be1920ad9'} {{(pid=88114) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:48:39.506989 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.notifications [None req-c613d6e9-ebd6-40ea-901d-f4816787c96e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '464bb24610c34dfc8ff3fb7be1920ad9'} {{(pid=88114) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:48:39.523417 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 956/1909] 10.222.0.13 () {58 vars in 1224 bytes} [Wed Aug 31 09:48:38 2022] DELETE /identity/v3/users/464bb24610c34dfc8ff3fb7be1920ad9 => generated 0 bytes in 723 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:48:39.900094 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9538d8a-27d7-45f2-9378-c524af57f334 None None] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:39.900766 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9538d8a-27d7-45f2-9378-c524af57f334 None None] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:39.900984 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9538d8a-27d7-45f2-9378-c524af57f334 None None] PATH_INFO: `/` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:39.903228 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 954/1910] 10.222.0.13 () {54 vars in 921 bytes} [Wed Aug 31 09:48:39 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:48:39.932455 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c376748-051b-4de1-b95b-b8be1e1d7459 None None] REQUEST_METHOD: `POST` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:39.933715 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c376748-051b-4de1-b95b-b8be1e1d7459 None None] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:39.934574 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c376748-051b-4de1-b95b-b8be1e1d7459 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:39.991950 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:48:40.020525 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.auth.core [None req-3c376748-051b-4de1-b95b-b8be1e1d7459 None None] MFA Rules not processed for user `dc076493239d4ccdbdc37c0369c49529`. Rule list: `[]` (Enabled: `True`). {{(pid=88114) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:48:40.024670 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.fernet_utils [None req-3c376748-051b-4de1-b95b-b8be1e1d7459 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88114) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:48:40.057520 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9538d8a-27d7-45f2-9378-c524af57f334 None None] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:48:40.059592 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.fernet_utils [None req-e9538d8a-27d7-45f2-9378-c524af57f334 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:48:40.291481 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9538d8a-27d7-45f2-9378-c524af57f334 None None] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:48:40.297362 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0baf6261-0796-4331-be48-482d5561e57c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:48:40.298102 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0baf6261-0796-4331-be48-482d5561e57c admin admin] REQUEST_METHOD: `DELETE` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:40.298372 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0baf6261-0796-4331-be48-482d5561e57c admin admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:40.298629 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0baf6261-0796-4331-be48-482d5561e57c admin admin] PATH_INFO: `/v3/projects/4c58f80628584a6283d262a9f9738004` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:40.301824 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0baf6261-0796-4331-be48-482d5561e57c admin admin] RBAC: Authorizing `identity:delete_project(project_id=4c58f80628584a6283d262a9f9738004)` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:48:40.303825 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0baf6261-0796-4331-be48-482d5561e57c admin admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:48:40.312534 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 957/1911] 10.222.0.13 () {58 vars in 1018 bytes} [Wed Aug 31 09:48:39 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 383 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:48:40.320695 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c376748-051b-4de1-b95b-b8be1e1d7459 None None] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:48:40.326080 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.fernet_utils [None req-3c376748-051b-4de1-b95b-b8be1e1d7459 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88114) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:48:40.358867 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c376748-051b-4de1-b95b-b8be1e1d7459 None None] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:48:40.371495 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-decad85d-06b4-4c9e-8dd0-31aae5d10693 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc076493239d4ccdbdc37c0369c49529', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '49548eb44d62415582af820ab66decee', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:48:40.372934 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-decad85d-06b4-4c9e-8dd0-31aae5d10693 service nova] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:40.373532 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-decad85d-06b4-4c9e-8dd0-31aae5d10693 service nova] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:40.374568 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-decad85d-06b4-4c9e-8dd0-31aae5d10693 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:40.443305 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-decad85d-06b4-4c9e-8dd0-31aae5d10693 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:48:40.445616 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-decad85d-06b4-4c9e-8dd0-31aae5d10693 service nova] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:48:40.475063 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 958/1912] 10.222.0.13 () {60 vars in 1365 bytes} [Wed Aug 31 09:48:40 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 156 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 09:48:40.490157 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.notifications [None req-0baf6261-0796-4331-be48-482d5561e57c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4c58f80628584a6283d262a9f9738004', 'request_id': 'req-0baf6261-0796-4331-be48-482d5561e57c'} {{(pid=88113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:48:40.500109 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.notifications [None req-0baf6261-0796-4331-be48-482d5561e57c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4c58f80628584a6283d262a9f9738004', 'request_id': 'req-0baf6261-0796-4331-be48-482d5561e57c'} {{(pid=88113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:48:40.515203 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.notifications [None req-0baf6261-0796-4331-be48-482d5561e57c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4c58f80628584a6283d262a9f9738004', 'request_id': 'req-0baf6261-0796-4331-be48-482d5561e57c'} {{(pid=88113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:48:40.516989 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:48:40.554854 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.notifications [None req-0baf6261-0796-4331-be48-482d5561e57c admin admin] The token cache is being invalidate because project 4c58f80628584a6283d262a9f9738004 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88113) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 09:48:40.555487 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.notifications [None req-0baf6261-0796-4331-be48-482d5561e57c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:48:40.558639 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.notifications [None req-0baf6261-0796-4331-be48-482d5561e57c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:48:40.567831 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 955/1913] 10.222.0.13 () {58 vars in 1233 bytes} [Wed Aug 31 09:48:40 2022] DELETE /identity/v3/projects/4c58f80628584a6283d262a9f9738004 => generated 0 bytes in 507 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:48:40.570990 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-decad85d-06b4-4c9e-8dd0-31aae5d10693 service nova] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:48:40.589665 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-decad85d-06b4-4c9e-8dd0-31aae5d10693 service nova] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:48:40.620723 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0baf6261-0796-4331-be48-482d5561e57c admin admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:48:40.623492 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.fernet_utils [None req-0baf6261-0796-4331-be48-482d5561e57c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:48:40.799422 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0baf6261-0796-4331-be48-482d5561e57c admin admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:48:40.807948 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d08744-affb-450a-82e8-3e47b8edf05e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:48:40.808621 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d08744-affb-450a-82e8-3e47b8edf05e admin admin] REQUEST_METHOD: `DELETE` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:40.808949 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d08744-affb-450a-82e8-3e47b8edf05e admin admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:40.809465 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d08744-affb-450a-82e8-3e47b8edf05e admin admin] PATH_INFO: `/v3/projects/3611fa772bfc4e1daadff496ca269d7f` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:40.811723 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8d08744-affb-450a-82e8-3e47b8edf05e admin admin] RBAC: Authorizing `identity:delete_project(project_id=3611fa772bfc4e1daadff496ca269d7f)` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:48:40.819743 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d79dca4-2c69-4ce6-8f6a-215c84ad026c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:48:40.820259 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8d08744-affb-450a-82e8-3e47b8edf05e admin admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:48:40.824371 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d79dca4-2c69-4ce6-8f6a-215c84ad026c admin admin] REQUEST_METHOD: `DELETE` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:40.824984 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d79dca4-2c69-4ce6-8f6a-215c84ad026c admin admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:40.825717 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d79dca4-2c69-4ce6-8f6a-215c84ad026c admin admin] PATH_INFO: `/v3/users/af000563e032411fa5098260260b7962` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:40.834600 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d79dca4-2c69-4ce6-8f6a-215c84ad026c admin admin] RBAC: Authorizing `identity:delete_user(user_id=af000563e032411fa5098260260b7962)` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:48:40.849763 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d79dca4-2c69-4ce6-8f6a-215c84ad026c admin admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:48:40.943126 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.notifications [None req-f8d08744-affb-450a-82e8-3e47b8edf05e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3611fa772bfc4e1daadff496ca269d7f', 'request_id': 'req-f8d08744-affb-450a-82e8-3e47b8edf05e'} {{(pid=88113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:48:40.953089 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.notifications [None req-f8d08744-affb-450a-82e8-3e47b8edf05e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3611fa772bfc4e1daadff496ca269d7f', 'request_id': 'req-f8d08744-affb-450a-82e8-3e47b8edf05e'} {{(pid=88113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:48:40.977030 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.notifications [None req-f8d08744-affb-450a-82e8-3e47b8edf05e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3611fa772bfc4e1daadff496ca269d7f', 'request_id': 'req-f8d08744-affb-450a-82e8-3e47b8edf05e'} {{(pid=88113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:48:40.992513 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.notifications [None req-f8d08744-affb-450a-82e8-3e47b8edf05e admin admin] The token cache is being invalidate because project 3611fa772bfc4e1daadff496ca269d7f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88113) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 09:48:40.993055 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.notifications [None req-f8d08744-affb-450a-82e8-3e47b8edf05e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:48:40.993974 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.notifications [None req-f8d08744-affb-450a-82e8-3e47b8edf05e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:48:40.996862 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 956/1914] 10.222.0.13 () {58 vars in 1233 bytes} [Wed Aug 31 09:48:40 2022] DELETE /identity/v3/projects/3611fa772bfc4e1daadff496ca269d7f => generated 0 bytes in 378 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 09:48:40.998046 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d08744-affb-450a-82e8-3e47b8edf05e admin admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:48:40.999850 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.fernet_utils [None req-f8d08744-affb-450a-82e8-3e47b8edf05e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:48:41.005526 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.notifications [None req-9d79dca4-2c69-4ce6-8f6a-215c84ad026c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'af000563e032411fa5098260260b7962'} {{(pid=88114) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:48:41.038489 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.notifications [None req-9d79dca4-2c69-4ce6-8f6a-215c84ad026c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'af000563e032411fa5098260260b7962'} {{(pid=88114) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:48:41.058538 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.notifications [None req-9d79dca4-2c69-4ce6-8f6a-215c84ad026c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'af000563e032411fa5098260260b7962'} {{(pid=88114) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:48:41.059835 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.notifications [None req-9d79dca4-2c69-4ce6-8f6a-215c84ad026c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'af000563e032411fa5098260260b7962'} {{(pid=88114) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:48:41.115397 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.notifications [None req-9d79dca4-2c69-4ce6-8f6a-215c84ad026c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'af000563e032411fa5098260260b7962'} {{(pid=88114) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:48:41.127395 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d08744-affb-450a-82e8-3e47b8edf05e admin admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:48:41.133907 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-910a660b-283a-45f1-b3e0-69ba50b28ed8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:48:41.134584 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910a660b-283a-45f1-b3e0-69ba50b28ed8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:41.134888 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910a660b-283a-45f1-b3e0-69ba50b28ed8 admin admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:41.135137 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910a660b-283a-45f1-b3e0-69ba50b28ed8 admin admin] PATH_INFO: `/v3/projects/5f145ab743f54358a029d0cf18202b8b` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:41.136861 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-910a660b-283a-45f1-b3e0-69ba50b28ed8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5f145ab743f54358a029d0cf18202b8b)` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:48:41.139208 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-910a660b-283a-45f1-b3e0-69ba50b28ed8 admin admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:48:41.146639 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 959/1915] 10.222.0.13 () {58 vars in 1224 bytes} [Wed Aug 31 09:48:40 2022] DELETE /identity/v3/users/af000563e032411fa5098260260b7962 => generated 0 bytes in 573 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:48:41.146639 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d79dca4-2c69-4ce6-8f6a-215c84ad026c admin admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:48:41.156081 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.fernet_utils [None req-9d79dca4-2c69-4ce6-8f6a-215c84ad026c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88114) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:48:41.173834 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:48:41.291379 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:48:41.361824 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.notifications [None req-910a660b-283a-45f1-b3e0-69ba50b28ed8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5f145ab743f54358a029d0cf18202b8b', 'request_id': 'req-910a660b-283a-45f1-b3e0-69ba50b28ed8'} {{(pid=88113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:48:41.380493 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.notifications [None req-910a660b-283a-45f1-b3e0-69ba50b28ed8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5f145ab743f54358a029d0cf18202b8b', 'request_id': 'req-910a660b-283a-45f1-b3e0-69ba50b28ed8'} {{(pid=88113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:48:41.390194 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d79dca4-2c69-4ce6-8f6a-215c84ad026c admin admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:48:41.397279 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.notifications [None req-910a660b-283a-45f1-b3e0-69ba50b28ed8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5f145ab743f54358a029d0cf18202b8b', 'request_id': 'req-910a660b-283a-45f1-b3e0-69ba50b28ed8'} {{(pid=88113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:48:41.419482 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.notifications [None req-910a660b-283a-45f1-b3e0-69ba50b28ed8 admin admin] The token cache is being invalidate because project 5f145ab743f54358a029d0cf18202b8b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88113) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 09:48:41.419810 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.notifications [None req-910a660b-283a-45f1-b3e0-69ba50b28ed8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:48:41.420941 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.notifications [None req-910a660b-283a-45f1-b3e0-69ba50b28ed8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:48:41.431516 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 957/1916] 10.222.0.13 () {58 vars in 1233 bytes} [Wed Aug 31 09:48:40 2022] DELETE /identity/v3/projects/5f145ab743f54358a029d0cf18202b8b => generated 0 bytes in 429 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:48:41.433120 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75451cc8-9c46-4135-b955-ad9bce852a20 None None] REQUEST_METHOD: `POST` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:41.433543 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75451cc8-9c46-4135-b955-ad9bce852a20 None None] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:41.433824 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75451cc8-9c46-4135-b955-ad9bce852a20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:41.491675 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.auth.core [None req-75451cc8-9c46-4135-b955-ad9bce852a20 None None] MFA Rules not processed for user `fd836cf590a145df9acf19cf3ae2ac56`. Rule list: `[]` (Enabled: `True`). {{(pid=88113) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:48:41.495862 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.fernet_utils [None req-75451cc8-9c46-4135-b955-ad9bce852a20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:48:41.548018 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06fb2c96-ad84-4b12-9360-dc216fd13222 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:48:41.549680 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06fb2c96-ad84-4b12-9360-dc216fd13222 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:41.550317 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06fb2c96-ad84-4b12-9360-dc216fd13222 admin admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:41.550906 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06fb2c96-ad84-4b12-9360-dc216fd13222 admin admin] PATH_INFO: `/v3/users/6794803d8b2d470cb5d924011d565670` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:41.554407 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06fb2c96-ad84-4b12-9360-dc216fd13222 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6794803d8b2d470cb5d924011d565670)` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:48:41.558847 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06fb2c96-ad84-4b12-9360-dc216fd13222 admin admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:48:41.661619 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.notifications [None req-06fb2c96-ad84-4b12-9360-dc216fd13222 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6794803d8b2d470cb5d924011d565670'} {{(pid=88114) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:48:41.669760 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 958/1917] 10.222.0.13 () {58 vars in 958 bytes} [Wed Aug 31 09:48:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 237 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:48:41.672197 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75451cc8-9c46-4135-b955-ad9bce852a20 None None] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:48:41.672497 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.notifications [None req-06fb2c96-ad84-4b12-9360-dc216fd13222 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6794803d8b2d470cb5d924011d565670'} {{(pid=88114) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:48:41.677094 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.fernet_utils [None req-75451cc8-9c46-4135-b955-ad9bce852a20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:48:41.684484 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.notifications [None req-06fb2c96-ad84-4b12-9360-dc216fd13222 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6794803d8b2d470cb5d924011d565670'} {{(pid=88114) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:48:41.685718 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.notifications [None req-06fb2c96-ad84-4b12-9360-dc216fd13222 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6794803d8b2d470cb5d924011d565670'} {{(pid=88114) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:48:41.706176 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.notifications [None req-06fb2c96-ad84-4b12-9360-dc216fd13222 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6794803d8b2d470cb5d924011d565670'} {{(pid=88114) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:48:41.715113 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75451cc8-9c46-4135-b955-ad9bce852a20 None None] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:48:41.721668 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 960/1918] 10.222.0.13 () {58 vars in 1224 bytes} [Wed Aug 31 09:48:41 2022] DELETE /identity/v3/users/6794803d8b2d470cb5d924011d565670 => generated 0 bytes in 572 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:48:41.721668 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8eca595-c8d6-421a-82d8-31a26fc91c7a None None] REQUEST_METHOD: `POST` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:41.721668 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8eca595-c8d6-421a-82d8-31a26fc91c7a None None] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:41.721668 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8eca595-c8d6-421a-82d8-31a26fc91c7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:41.763849 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.auth.core [None req-b8eca595-c8d6-421a-82d8-31a26fc91c7a None None] MFA Rules not processed for user `fd836cf590a145df9acf19cf3ae2ac56`. Rule list: `[]` (Enabled: `True`). {{(pid=88114) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:48:41.766819 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.fernet_utils [None req-b8eca595-c8d6-421a-82d8-31a26fc91c7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88114) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:48:41.846619 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:48:42.038095 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 961/1919] 10.222.0.13 () {58 vars in 958 bytes} [Wed Aug 31 09:48:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 321 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:48:42.039956 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8eca595-c8d6-421a-82d8-31a26fc91c7a None None] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:48:42.043246 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.fernet_utils [None req-b8eca595-c8d6-421a-82d8-31a26fc91c7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88114) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:48:42.072875 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8eca595-c8d6-421a-82d8-31a26fc91c7a None None] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:48:42.087188 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fb4aa6d-4140-48cd-bb41-1bea9322a9d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:48:42.088045 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb4aa6d-4140-48cd-bb41-1bea9322a9d8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:42.088457 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb4aa6d-4140-48cd-bb41-1bea9322a9d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:42.088816 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb4aa6d-4140-48cd-bb41-1bea9322a9d8 admin admin] PATH_INFO: `/v3/projects/93a0948106c440bc8172dd99b955420c` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:42.090254 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fb4aa6d-4140-48cd-bb41-1bea9322a9d8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=93a0948106c440bc8172dd99b955420c)` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:48:42.092474 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fb4aa6d-4140-48cd-bb41-1bea9322a9d8 admin admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:48:42.116301 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02aaea96-2742-4957-91e5-6cb6d9dafb53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:48:42.119598 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02aaea96-2742-4957-91e5-6cb6d9dafb53 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:42.120288 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02aaea96-2742-4957-91e5-6cb6d9dafb53 admin admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:42.121045 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02aaea96-2742-4957-91e5-6cb6d9dafb53 admin admin] PATH_INFO: `/v3/users/f1bb1e059d26454a88ec518cda057bfc` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:42.124008 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02aaea96-2742-4957-91e5-6cb6d9dafb53 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f1bb1e059d26454a88ec518cda057bfc)` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:48:42.130870 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02aaea96-2742-4957-91e5-6cb6d9dafb53 admin admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:48:42.200604 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.notifications [None req-8fb4aa6d-4140-48cd-bb41-1bea9322a9d8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '93a0948106c440bc8172dd99b955420c', 'request_id': 'req-8fb4aa6d-4140-48cd-bb41-1bea9322a9d8'} {{(pid=88114) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:48:42.208958 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.notifications [None req-8fb4aa6d-4140-48cd-bb41-1bea9322a9d8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '93a0948106c440bc8172dd99b955420c', 'request_id': 'req-8fb4aa6d-4140-48cd-bb41-1bea9322a9d8'} {{(pid=88114) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:48:42.218070 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.notifications [None req-8fb4aa6d-4140-48cd-bb41-1bea9322a9d8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '93a0948106c440bc8172dd99b955420c', 'request_id': 'req-8fb4aa6d-4140-48cd-bb41-1bea9322a9d8'} {{(pid=88114) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:48:42.248551 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.notifications [None req-8fb4aa6d-4140-48cd-bb41-1bea9322a9d8 admin admin] The token cache is being invalidate because project 93a0948106c440bc8172dd99b955420c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88114) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 09:48:42.249080 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.notifications [None req-8fb4aa6d-4140-48cd-bb41-1bea9322a9d8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88114) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:48:42.250268 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.notifications [None req-8fb4aa6d-4140-48cd-bb41-1bea9322a9d8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88114) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:48:42.255264 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 962/1920] 10.222.0.13 () {58 vars in 1233 bytes} [Wed Aug 31 09:48:42 2022] DELETE /identity/v3/projects/93a0948106c440bc8172dd99b955420c => generated 0 bytes in 216 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:48:42.256159 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fb4aa6d-4140-48cd-bb41-1bea9322a9d8 admin admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:48:42.259400 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.fernet_utils [None req-8fb4aa6d-4140-48cd-bb41-1bea9322a9d8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88114) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:48:42.271743 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.notifications [None req-02aaea96-2742-4957-91e5-6cb6d9dafb53 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f1bb1e059d26454a88ec518cda057bfc'} {{(pid=88113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:48:42.282766 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.notifications [None req-02aaea96-2742-4957-91e5-6cb6d9dafb53 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f1bb1e059d26454a88ec518cda057bfc'} {{(pid=88113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:48:42.285516 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:DELETE=13,keystone:SELECT=86,keystone:INSERT=1 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:48:42.309648 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.notifications [None req-02aaea96-2742-4957-91e5-6cb6d9dafb53 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f1bb1e059d26454a88ec518cda057bfc'} {{(pid=88113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:48:42.311150 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.notifications [None req-02aaea96-2742-4957-91e5-6cb6d9dafb53 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f1bb1e059d26454a88ec518cda057bfc'} {{(pid=88113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:48:42.324964 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.notifications [None req-02aaea96-2742-4957-91e5-6cb6d9dafb53 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f1bb1e059d26454a88ec518cda057bfc'} {{(pid=88113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:48:42.336910 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 959/1921] 10.222.0.13 () {58 vars in 1224 bytes} [Wed Aug 31 09:48:41 2022] DELETE /identity/v3/users/f1bb1e059d26454a88ec518cda057bfc => generated 0 bytes in 666 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:48:42.338829 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943934e6-da0c-4a41-9dcb-214caf645d4b None None] REQUEST_METHOD: `POST` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:42.339969 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943934e6-da0c-4a41-9dcb-214caf645d4b None None] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:42.340417 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943934e6-da0c-4a41-9dcb-214caf645d4b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:42.385220 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.auth.core [None req-943934e6-da0c-4a41-9dcb-214caf645d4b None None] MFA Rules not processed for user `fd836cf590a145df9acf19cf3ae2ac56`. Rule list: `[]` (Enabled: `True`). {{(pid=88113) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:48:42.387173 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.fernet_utils [None req-943934e6-da0c-4a41-9dcb-214caf645d4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:48:42.451974 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fb4aa6d-4140-48cd-bb41-1bea9322a9d8 admin admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:48:42.471223 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 960/1922] 10.222.0.13 () {58 vars in 958 bytes} [Wed Aug 31 09:48:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:48:42.473761 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53f9da5-ccbb-49cf-b44b-f2845649fafc None None] REQUEST_METHOD: `POST` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:42.474091 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53f9da5-ccbb-49cf-b44b-f2845649fafc None None] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:42.474388 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53f9da5-ccbb-49cf-b44b-f2845649fafc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:42.511120 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.auth.core [None req-a53f9da5-ccbb-49cf-b44b-f2845649fafc None None] MFA Rules not processed for user `fd836cf590a145df9acf19cf3ae2ac56`. Rule list: `[]` (Enabled: `True`). {{(pid=88113) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:48:42.513321 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.fernet_utils [None req-a53f9da5-ccbb-49cf-b44b-f2845649fafc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:48:42.541816 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 961/1923] 10.222.0.13 () {58 vars in 958 bytes} [Wed Aug 31 09:48:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 68 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:48:42.542556 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12907aa3-0774-4604-b4d9-c47b80f33aeb None None] REQUEST_METHOD: `POST` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:42.543030 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12907aa3-0774-4604-b4d9-c47b80f33aeb None None] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:42.543496 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12907aa3-0774-4604-b4d9-c47b80f33aeb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:42.588572 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.auth.core [None req-12907aa3-0774-4604-b4d9-c47b80f33aeb None None] MFA Rules not processed for user `fd836cf590a145df9acf19cf3ae2ac56`. Rule list: `[]` (Enabled: `True`). {{(pid=88113) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:48:42.591417 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.fernet_utils [None req-12907aa3-0774-4604-b4d9-c47b80f33aeb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:48:42.617941 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 962/1924] 10.222.0.13 () {58 vars in 958 bytes} [Wed Aug 31 09:48:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 74 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:48:42.617941 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12907aa3-0774-4604-b4d9-c47b80f33aeb None None] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:48:42.621194 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.fernet_utils [None req-12907aa3-0774-4604-b4d9-c47b80f33aeb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:48:42.642719 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12907aa3-0774-4604-b4d9-c47b80f33aeb None None] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:48:42.646536 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ffef07d-8a75-480e-a7de-e8c664de2ef5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:48:42.648556 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ffef07d-8a75-480e-a7de-e8c664de2ef5 admin admin] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:42.649272 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ffef07d-8a75-480e-a7de-e8c664de2ef5 admin admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:42.649918 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ffef07d-8a75-480e-a7de-e8c664de2ef5 admin admin] PATH_INFO: `/v3/domains` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:42.650967 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a816bb-443b-4003-97e2-31ce51f39c71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:48:42.651719 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a816bb-443b-4003-97e2-31ce51f39c71 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:42.652078 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a816bb-443b-4003-97e2-31ce51f39c71 admin admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:42.652224 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ffef07d-8a75-480e-a7de-e8c664de2ef5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88114) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:48:42.652486 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a816bb-443b-4003-97e2-31ce51f39c71 admin admin] PATH_INFO: `/v3/projects/d8376c848fa04ebca5b4e9e8351bd363` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:42.652954 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ffef07d-8a75-480e-a7de-e8c664de2ef5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:48:42.654004 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2a816bb-443b-4003-97e2-31ce51f39c71 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d8376c848fa04ebca5b4e9e8351bd363)` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:48:42.656740 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2a816bb-443b-4003-97e2-31ce51f39c71 admin admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:48:42.657128 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ffef07d-8a75-480e-a7de-e8c664de2ef5 admin admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:48:42.663861 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:48:42.692138 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 963/1925] 10.222.0.13 () {58 vars in 1153 bytes} [Wed Aug 31 09:48:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 436 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:48:42.693518 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ffef07d-8a75-480e-a7de-e8c664de2ef5 admin admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:48:42.726104 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ffef07d-8a75-480e-a7de-e8c664de2ef5 admin admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:48:42.735769 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.notifications [None req-a2a816bb-443b-4003-97e2-31ce51f39c71 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd8376c848fa04ebca5b4e9e8351bd363', 'request_id': 'req-a2a816bb-443b-4003-97e2-31ce51f39c71'} {{(pid=88113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:48:42.745287 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.notifications [None req-a2a816bb-443b-4003-97e2-31ce51f39c71 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd8376c848fa04ebca5b4e9e8351bd363', 'request_id': 'req-a2a816bb-443b-4003-97e2-31ce51f39c71'} {{(pid=88113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:48:42.760046 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.notifications [None req-a2a816bb-443b-4003-97e2-31ce51f39c71 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd8376c848fa04ebca5b4e9e8351bd363', 'request_id': 'req-a2a816bb-443b-4003-97e2-31ce51f39c71'} {{(pid=88113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:48:42.779397 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.notifications [None req-a2a816bb-443b-4003-97e2-31ce51f39c71 admin admin] The token cache is being invalidate because project d8376c848fa04ebca5b4e9e8351bd363 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88113) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 09:48:42.780175 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.notifications [None req-a2a816bb-443b-4003-97e2-31ce51f39c71 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:48:42.781636 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.notifications [None req-a2a816bb-443b-4003-97e2-31ce51f39c71 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:48:42.783705 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 963/1926] 10.222.0.13 () {58 vars in 1233 bytes} [Wed Aug 31 09:48:42 2022] DELETE /identity/v3/projects/d8376c848fa04ebca5b4e9e8351bd363 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:48:42.786884 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a816bb-443b-4003-97e2-31ce51f39c71 admin admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:48:42.791801 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.fernet_utils [None req-a2a816bb-443b-4003-97e2-31ce51f39c71 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:48:42.820097 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3647f8b0-6c24-4bf6-8304-b55cdcdf1143 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:48:42.821007 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3647f8b0-6c24-4bf6-8304-b55cdcdf1143 admin admin] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:42.822061 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3647f8b0-6c24-4bf6-8304-b55cdcdf1143 admin admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:42.822388 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3647f8b0-6c24-4bf6-8304-b55cdcdf1143 admin admin] PATH_INFO: `/v3/domains` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:42.823072 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3647f8b0-6c24-4bf6-8304-b55cdcdf1143 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88114) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:48:42.823379 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3647f8b0-6c24-4bf6-8304-b55cdcdf1143 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:48:42.825764 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3647f8b0-6c24-4bf6-8304-b55cdcdf1143 admin admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:48:42.852991 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 964/1927] 10.222.0.13 () {58 vars in 1153 bytes} [Wed Aug 31 09:48:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:48:42.854432 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3647f8b0-6c24-4bf6-8304-b55cdcdf1143 admin admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:48:42.864641 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.fernet_utils [None req-3647f8b0-6c24-4bf6-8304-b55cdcdf1143 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88114) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:48:42.884830 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:48:42.893478 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3647f8b0-6c24-4bf6-8304-b55cdcdf1143 admin admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:48:42.907051 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-239d9cd0-ed59-47fb-be41-cad4c8e7c33f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:48:42.907723 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a816bb-443b-4003-97e2-31ce51f39c71 admin admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:48:42.908763 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239d9cd0-ed59-47fb-be41-cad4c8e7c33f admin admin] REQUEST_METHOD: `POST` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:42.909322 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239d9cd0-ed59-47fb-be41-cad4c8e7c33f admin admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:42.911012 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239d9cd0-ed59-47fb-be41-cad4c8e7c33f admin admin] PATH_INFO: `/v3/projects` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:42.912581 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-239d9cd0-ed59-47fb-be41-cad4c8e7c33f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:48:42.916261 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9bd7035-b60a-4a2b-9460-613d73ec1ebb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9bdaacd597442c2bcde2c555f2cc140', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '49548eb44d62415582af820ab66decee', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:48:42.916458 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-239d9cd0-ed59-47fb-be41-cad4c8e7c33f admin admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:48:42.917369 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9bd7035-b60a-4a2b-9460-613d73ec1ebb service neutron] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:42.917858 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9bd7035-b60a-4a2b-9460-613d73ec1ebb service neutron] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:42.918307 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9bd7035-b60a-4a2b-9460-613d73ec1ebb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:42.920751 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.fernet_utils [None req-e9bd7035-b60a-4a2b-9460-613d73ec1ebb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:48:42.940051 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9bd7035-b60a-4a2b-9460-613d73ec1ebb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:48:42.946318 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9bd7035-b60a-4a2b-9460-613d73ec1ebb service neutron] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:48:42.967347 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 964/1928] 10.222.0.13 () {60 vars in 1365 bytes} [Wed Aug 31 09:48:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 183 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:48:42.973427 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c4669d-b53d-46fd-ae0a-0350a33ac84b None None] REQUEST_METHOD: `POST` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:42.973427 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c4669d-b53d-46fd-ae0a-0350a33ac84b None None] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:42.973427 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c4669d-b53d-46fd-ae0a-0350a33ac84b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:42.973741 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 965/1929] 10.222.0.13 () {60 vars in 1153 bytes} [Wed Aug 31 09:48:42 2022] POST /identity/v3/projects => generated 373 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 09:48:42.974118 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-239d9cd0-ed59-47fb-be41-cad4c8e7c33f admin admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:48:42.977782 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.fernet_utils [None req-239d9cd0-ed59-47fb-be41-cad4c8e7c33f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88114) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:48:43.012233 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.auth.core [None req-77c4669d-b53d-46fd-ae0a-0350a33ac84b None None] MFA Rules not processed for user `fd836cf590a145df9acf19cf3ae2ac56`. Rule list: `[]` (Enabled: `True`). {{(pid=88113) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:48:43.015078 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.fernet_utils [None req-77c4669d-b53d-46fd-ae0a-0350a33ac84b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:48:43.138472 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 965/1930] 10.222.0.13 () {58 vars in 958 bytes} [Wed Aug 31 09:48:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:48:43.143489 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c4669d-b53d-46fd-ae0a-0350a33ac84b None None] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:48:43.169774 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c4669d-b53d-46fd-ae0a-0350a33ac84b None None] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:48:43.186176 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b47c2945-1aff-4079-a716-f0e58bdc78f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:48:43.186958 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47c2945-1aff-4079-a716-f0e58bdc78f1 admin admin] REQUEST_METHOD: `POST` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:43.187367 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47c2945-1aff-4079-a716-f0e58bdc78f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:43.187766 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47c2945-1aff-4079-a716-f0e58bdc78f1 admin admin] PATH_INFO: `/v3/users` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:43.188551 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b47c2945-1aff-4079-a716-f0e58bdc78f1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:48:43.191752 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b47c2945-1aff-4079-a716-f0e58bdc78f1 admin admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:48:43.246242 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-239d9cd0-ed59-47fb-be41-cad4c8e7c33f admin admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:48:43.261969 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 966/1931] 10.222.0.13 () {60 vars in 1144 bytes} [Wed Aug 31 09:48:43 2022] POST /identity/v3/users => generated 287 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 09:48:43.264509 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4836de-073a-4f86-af35-307784c1fb84 None None] REQUEST_METHOD: `POST` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:43.264992 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4836de-073a-4f86-af35-307784c1fb84 None None] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:43.265524 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4836de-073a-4f86-af35-307784c1fb84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:43.267287 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd578d97-63c5-48bf-b124-541bcf392377 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:48:43.268421 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd578d97-63c5-48bf-b124-541bcf392377 admin admin] REQUEST_METHOD: `POST` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:43.269723 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd578d97-63c5-48bf-b124-541bcf392377 admin admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:43.270236 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd578d97-63c5-48bf-b124-541bcf392377 admin admin] PATH_INFO: `/v3/projects` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:43.271460 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd578d97-63c5-48bf-b124-541bcf392377 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:48:43.278128 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd578d97-63c5-48bf-b124-541bcf392377 admin admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:48:43.307280 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.auth.core [None req-8e4836de-073a-4f86-af35-307784c1fb84 None None] MFA Rules not processed for user `fd836cf590a145df9acf19cf3ae2ac56`. Rule list: `[]` (Enabled: `True`). {{(pid=88113) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:48:43.310069 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.fernet_utils [None req-8e4836de-073a-4f86-af35-307784c1fb84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:48:43.337473 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 967/1932] 10.222.0.13 () {58 vars in 958 bytes} [Wed Aug 31 09:48:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 72 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:48:43.337473 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e4836de-073a-4f86-af35-307784c1fb84 None None] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:48:43.358673 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e4836de-073a-4f86-af35-307784c1fb84 None None] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:48:43.370584 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eef51de-3617-4644-89cb-79d5408e67db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:48:43.371880 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eef51de-3617-4644-89cb-79d5408e67db admin admin] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:43.372502 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eef51de-3617-4644-89cb-79d5408e67db admin admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:43.372999 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eef51de-3617-4644-89cb-79d5408e67db admin admin] PATH_INFO: `/v3/roles` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:43.374435 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eef51de-3617-4644-89cb-79d5408e67db admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:48:43.379806 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eef51de-3617-4644-89cb-79d5408e67db admin admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:48:43.398045 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 966/1933] 10.222.0.13 () {60 vars in 1153 bytes} [Wed Aug 31 09:48:42 2022] POST /identity/v3/projects => generated 377 bytes in 426 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 09:48:43.399949 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd578d97-63c5-48bf-b124-541bcf392377 admin admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:48:43.408482 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 968/1934] 10.222.0.13 () {58 vars in 1122 bytes} [Wed Aug 31 09:48:43 2022] GET /identity/v3/roles => generated 1420 bytes in 74 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 09:48:43.410887 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eef51de-3617-4644-89cb-79d5408e67db admin admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:48:43.453836 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eef51de-3617-4644-89cb-79d5408e67db admin admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:48:43.628854 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:48:43.667027 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c986e2ef-b2b2-4259-923f-7877a3c26905 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:48:43.667826 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c986e2ef-b2b2-4259-923f-7877a3c26905 admin admin] REQUEST_METHOD: `POST` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:43.668308 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c986e2ef-b2b2-4259-923f-7877a3c26905 admin admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:43.668804 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c986e2ef-b2b2-4259-923f-7877a3c26905 admin admin] PATH_INFO: `/v3/users` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:43.670225 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c986e2ef-b2b2-4259-923f-7877a3c26905 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:48:43.677934 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd578d97-63c5-48bf-b124-541bcf392377 admin admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:48:43.683143 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c986e2ef-b2b2-4259-923f-7877a3c26905 admin admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:48:43.698684 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed004bc-c7db-49c0-80af-e5ff5db46224 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:48:43.699733 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed004bc-c7db-49c0-80af-e5ff5db46224 admin admin] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:43.700185 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed004bc-c7db-49c0-80af-e5ff5db46224 admin admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:43.700660 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed004bc-c7db-49c0-80af-e5ff5db46224 admin admin] PATH_INFO: `/v3/domains` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:43.701923 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ed004bc-c7db-49c0-80af-e5ff5db46224 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88114) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:48:43.702583 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ed004bc-c7db-49c0-80af-e5ff5db46224 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:48:43.716703 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ed004bc-c7db-49c0-80af-e5ff5db46224 admin admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:48:43.756209 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 969/1935] 10.222.0.13 () {60 vars in 1144 bytes} [Wed Aug 31 09:48:43 2022] POST /identity/v3/users => generated 289 bytes in 346 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 09:48:43.760763 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c986e2ef-b2b2-4259-923f-7877a3c26905 admin admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:48:43.774227 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 967/1936] 10.222.0.13 () {58 vars in 1153 bytes} [Wed Aug 31 09:48:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 369 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:48:43.774227 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed004bc-c7db-49c0-80af-e5ff5db46224 admin admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:48:43.779597 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c986e2ef-b2b2-4259-923f-7877a3c26905 admin admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:48:43.787764 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be9e8c0c-5b80-4376-9e71-1a01af65550a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:48:43.789020 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9e8c0c-5b80-4376-9e71-1a01af65550a admin admin] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:43.789420 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9e8c0c-5b80-4376-9e71-1a01af65550a admin admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:43.789863 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9e8c0c-5b80-4376-9e71-1a01af65550a admin admin] PATH_INFO: `/v3/roles` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:43.790785 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be9e8c0c-5b80-4376-9e71-1a01af65550a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:48:43.794061 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be9e8c0c-5b80-4376-9e71-1a01af65550a admin admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:48:43.808660 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed004bc-c7db-49c0-80af-e5ff5db46224 admin admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:48:43.826123 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 970/1937] 10.222.0.13 () {58 vars in 1122 bytes} [Wed Aug 31 09:48:43 2022] GET /identity/v3/roles => generated 1420 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 09:48:43.830094 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be9e8c0c-5b80-4376-9e71-1a01af65550a admin admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:48:43.859618 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f7fad05-1c70-46b1-9144-471cf404814e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:48:43.861102 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7fad05-1c70-46b1-9144-471cf404814e admin admin] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:43.864215 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7fad05-1c70-46b1-9144-471cf404814e admin admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:43.864980 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7fad05-1c70-46b1-9144-471cf404814e admin admin] PATH_INFO: `/v3/roles` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:43.868321 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f7fad05-1c70-46b1-9144-471cf404814e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:48:43.875675 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f7fad05-1c70-46b1-9144-471cf404814e admin admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:48:43.884795 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be9e8c0c-5b80-4376-9e71-1a01af65550a admin admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:48:43.903500 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 968/1938] 10.222.0.13 () {58 vars in 1122 bytes} [Wed Aug 31 09:48:43 2022] GET /identity/v3/roles => generated 1420 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 09:48:43.904596 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f7fad05-1c70-46b1-9144-471cf404814e admin admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:48:43.904804 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9448a17d-3dcf-4c3f-81d9-f9b1354bf569 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:48:43.905902 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9448a17d-3dcf-4c3f-81d9-f9b1354bf569 admin admin] REQUEST_METHOD: `POST` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:43.906233 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9448a17d-3dcf-4c3f-81d9-f9b1354bf569 admin admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:43.906514 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9448a17d-3dcf-4c3f-81d9-f9b1354bf569 admin admin] PATH_INFO: `/v3/projects` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:43.907267 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9448a17d-3dcf-4c3f-81d9-f9b1354bf569 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:48:43.913613 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9448a17d-3dcf-4c3f-81d9-f9b1354bf569 admin admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:48:43.929662 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f7fad05-1c70-46b1-9144-471cf404814e admin admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:48:43.948193 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da6392c3-ffc9-49b9-af31-a7ba809277e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:48:43.949665 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6392c3-ffc9-49b9-af31-a7ba809277e2 admin admin] REQUEST_METHOD: `PUT` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:43.950789 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6392c3-ffc9-49b9-af31-a7ba809277e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:43.952868 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6392c3-ffc9-49b9-af31-a7ba809277e2 admin admin] PATH_INFO: `/v3/projects/0d7d52f95e5e452ca2ac11900fc0d931/users/c475984de9e84ff18fb96c588a07c80b/roles/6c99f5d9f1f243eaa134d8540f21095d` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:44.004239 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da6392c3-ffc9-49b9-af31-a7ba809277e2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0d7d52f95e5e452ca2ac11900fc0d931, user_id=c475984de9e84ff18fb96c588a07c80b, role_id=6c99f5d9f1f243eaa134d8540f21095d)` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:48:44.007253 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da6392c3-ffc9-49b9-af31-a7ba809277e2 admin admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:48:44.012816 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 971/1939] 10.222.0.13 () {60 vars in 1153 bytes} [Wed Aug 31 09:48:43 2022] POST /identity/v3/projects => generated 371 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 09:48:44.020563 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9448a17d-3dcf-4c3f-81d9-f9b1354bf569 admin admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:48:44.023755 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.fernet_utils [None req-9448a17d-3dcf-4c3f-81d9-f9b1354bf569 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:48:44.044144 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 969/1940] 10.222.0.13 () {60 vars in 1483 bytes} [Wed Aug 31 09:48:43 2022] PUT /identity/v3/projects/0d7d52f95e5e452ca2ac11900fc0d931/users/c475984de9e84ff18fb96c588a07c80b/roles/6c99f5d9f1f243eaa134d8540f21095d => generated 0 bytes in 139 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:48:44.046046 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da6392c3-ffc9-49b9-af31-a7ba809277e2 admin admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:48:44.068637 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:48:44.231663 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da6392c3-ffc9-49b9-af31-a7ba809277e2 admin admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:48:44.236381 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9448a17d-3dcf-4c3f-81d9-f9b1354bf569 admin admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:48:44.241561 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bff0e70c-7c7d-459f-be0a-0efaf7637673 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:48:44.243116 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe9ee52d-d4cf-4526-8c83-de5ddeaef6aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:48:44.243743 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff0e70c-7c7d-459f-be0a-0efaf7637673 admin admin] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:44.243937 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9ee52d-d4cf-4526-8c83-de5ddeaef6aa admin admin] REQUEST_METHOD: `DELETE` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:44.244074 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9ee52d-d4cf-4526-8c83-de5ddeaef6aa admin admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:44.244500 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9ee52d-d4cf-4526-8c83-de5ddeaef6aa admin admin] PATH_INFO: `/v3/projects/854d843d800944a7859e4927293eb5cb` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:44.245329 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff0e70c-7c7d-459f-be0a-0efaf7637673 admin admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:44.246118 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff0e70c-7c7d-459f-be0a-0efaf7637673 admin admin] PATH_INFO: `/v3/roles` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:44.246361 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe9ee52d-d4cf-4526-8c83-de5ddeaef6aa admin admin] RBAC: Authorizing `identity:delete_project(project_id=854d843d800944a7859e4927293eb5cb)` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:48:44.248067 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bff0e70c-7c7d-459f-be0a-0efaf7637673 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:48:44.248526 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe9ee52d-d4cf-4526-8c83-de5ddeaef6aa admin admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:48:44.252724 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bff0e70c-7c7d-459f-be0a-0efaf7637673 admin admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:48:44.285600 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 970/1941] 10.222.0.13 () {58 vars in 1122 bytes} [Wed Aug 31 09:48:44 2022] GET /identity/v3/roles => generated 1420 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 09:48:44.285600 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bff0e70c-7c7d-459f-be0a-0efaf7637673 admin admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:48:44.314625 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bff0e70c-7c7d-459f-be0a-0efaf7637673 admin admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:48:44.331199 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-644f1a1a-6693-4224-a2f9-1ac614751f63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:48:44.334091 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644f1a1a-6693-4224-a2f9-1ac614751f63 admin admin] REQUEST_METHOD: `POST` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:44.334484 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644f1a1a-6693-4224-a2f9-1ac614751f63 admin admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:44.334919 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644f1a1a-6693-4224-a2f9-1ac614751f63 admin admin] PATH_INFO: `/v3/users` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:44.336176 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-644f1a1a-6693-4224-a2f9-1ac614751f63 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:48:44.340771 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-644f1a1a-6693-4224-a2f9-1ac614751f63 admin admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:48:44.404754 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.notifications [None req-fe9ee52d-d4cf-4526-8c83-de5ddeaef6aa admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '854d843d800944a7859e4927293eb5cb', 'request_id': 'req-fe9ee52d-d4cf-4526-8c83-de5ddeaef6aa'} {{(pid=88113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:48:44.414937 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.notifications [None req-fe9ee52d-d4cf-4526-8c83-de5ddeaef6aa admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '854d843d800944a7859e4927293eb5cb', 'request_id': 'req-fe9ee52d-d4cf-4526-8c83-de5ddeaef6aa'} {{(pid=88113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:48:44.435528 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.notifications [None req-fe9ee52d-d4cf-4526-8c83-de5ddeaef6aa admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '854d843d800944a7859e4927293eb5cb', 'request_id': 'req-fe9ee52d-d4cf-4526-8c83-de5ddeaef6aa'} {{(pid=88113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:48:44.437856 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 971/1942] 10.222.0.13 () {60 vars in 1144 bytes} [Wed Aug 31 09:48:44 2022] POST /identity/v3/users => generated 286 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 09:48:44.444819 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6b123c-64ef-419c-a789-d0055010f220 None None] REQUEST_METHOD: `POST` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:44.445111 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6b123c-64ef-419c-a789-d0055010f220 None None] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:44.445409 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6b123c-64ef-419c-a789-d0055010f220 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:44.452729 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.notifications [None req-fe9ee52d-d4cf-4526-8c83-de5ddeaef6aa admin admin] The token cache is being invalidate because project 854d843d800944a7859e4927293eb5cb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88113) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 09:48:44.453572 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.notifications [None req-fe9ee52d-d4cf-4526-8c83-de5ddeaef6aa admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:48:44.454717 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.notifications [None req-fe9ee52d-d4cf-4526-8c83-de5ddeaef6aa admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:48:44.457295 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 972/1943] 10.222.0.13 () {58 vars in 1233 bytes} [Wed Aug 31 09:48:44 2022] DELETE /identity/v3/projects/854d843d800944a7859e4927293eb5cb => generated 0 bytes in 444 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:48:44.458767 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe9ee52d-d4cf-4526-8c83-de5ddeaef6aa admin admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:48:44.461330 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.fernet_utils [None req-fe9ee52d-d4cf-4526-8c83-de5ddeaef6aa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:48:44.525096 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.auth.core [None req-2f6b123c-64ef-419c-a789-d0055010f220 None None] MFA Rules not processed for user `fd836cf590a145df9acf19cf3ae2ac56`. Rule list: `[]` (Enabled: `True`). {{(pid=88114) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:48:44.526445 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.fernet_utils [None req-2f6b123c-64ef-419c-a789-d0055010f220 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88114) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:48:44.555673 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe9ee52d-d4cf-4526-8c83-de5ddeaef6aa admin admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:48:44.571278 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c113de2-e4c6-47ed-98ab-e5f8233f8cf7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:48:44.572395 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c113de2-e4c6-47ed-98ab-e5f8233f8cf7 admin admin] REQUEST_METHOD: `PUT` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:44.572673 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c113de2-e4c6-47ed-98ab-e5f8233f8cf7 admin admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:44.575111 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c113de2-e4c6-47ed-98ab-e5f8233f8cf7 admin admin] PATH_INFO: `/v3/projects/1c9972e700f4459dba1afc3060ca9a39/users/9ca662ced8784e2185764bcbdf921ec4/roles/6c99f5d9f1f243eaa134d8540f21095d` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:44.610856 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c113de2-e4c6-47ed-98ab-e5f8233f8cf7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1c9972e700f4459dba1afc3060ca9a39, user_id=9ca662ced8784e2185764bcbdf921ec4, role_id=6c99f5d9f1f243eaa134d8540f21095d)` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:48:44.615383 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c113de2-e4c6-47ed-98ab-e5f8233f8cf7 admin admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:48:44.652146 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 973/1944] 10.222.0.13 () {60 vars in 1483 bytes} [Wed Aug 31 09:48:44 2022] PUT /identity/v3/projects/1c9972e700f4459dba1afc3060ca9a39/users/9ca662ced8784e2185764bcbdf921ec4/roles/6c99f5d9f1f243eaa134d8540f21095d => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:48:44.653238 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c113de2-e4c6-47ed-98ab-e5f8233f8cf7 admin admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:48:44.655049 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.fernet_utils [None req-5c113de2-e4c6-47ed-98ab-e5f8233f8cf7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:48:44.723443 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 972/1945] 10.222.0.13 () {58 vars in 958 bytes} [Wed Aug 31 09:48:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 285 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:48:44.731255 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa0a655-ef41-4be6-a03e-078e4c0def9d None None] REQUEST_METHOD: `POST` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:44.731797 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa0a655-ef41-4be6-a03e-078e4c0def9d None None] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:44.732521 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa0a655-ef41-4be6-a03e-078e4c0def9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:44.772313 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.auth.core [None req-4fa0a655-ef41-4be6-a03e-078e4c0def9d None None] MFA Rules not processed for user `fd836cf590a145df9acf19cf3ae2ac56`. Rule list: `[]` (Enabled: `True`). {{(pid=88114) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:48:44.774712 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.fernet_utils [None req-4fa0a655-ef41-4be6-a03e-078e4c0def9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88114) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:48:44.780438 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:48:44.835407 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:48:44.904533 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 973/1946] 10.222.0.13 () {58 vars in 958 bytes} [Wed Aug 31 09:48:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:48:44.906648 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e707b1b5-a84d-4d10-b04f-3df628c1d1ae None None] REQUEST_METHOD: `POST` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:44.906930 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e707b1b5-a84d-4d10-b04f-3df628c1d1ae None None] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:44.907200 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e707b1b5-a84d-4d10-b04f-3df628c1d1ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:44.917655 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c113de2-e4c6-47ed-98ab-e5f8233f8cf7 admin admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:48:44.923900 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a863bd20-f55d-4832-bbc6-8267f0a45939 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:48:44.925062 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a863bd20-f55d-4832-bbc6-8267f0a45939 admin admin] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:44.925062 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a863bd20-f55d-4832-bbc6-8267f0a45939 admin admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:44.925836 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a863bd20-f55d-4832-bbc6-8267f0a45939 admin admin] PATH_INFO: `/v3/roles` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:44.929647 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a863bd20-f55d-4832-bbc6-8267f0a45939 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:48:44.931610 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a863bd20-f55d-4832-bbc6-8267f0a45939 admin admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:48:44.961896 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 974/1947] 10.222.0.13 () {58 vars in 1122 bytes} [Wed Aug 31 09:48:44 2022] GET /identity/v3/roles => generated 1420 bytes in 309 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 09:48:44.963587 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a863bd20-f55d-4832-bbc6-8267f0a45939 admin admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:48:44.965962 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.fernet_utils [None req-a863bd20-f55d-4832-bbc6-8267f0a45939 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:48:44.973863 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.auth.core [None req-e707b1b5-a84d-4d10-b04f-3df628c1d1ae None None] MFA Rules not processed for user `9ca662ced8784e2185764bcbdf921ec4`. Rule list: `[]` (Enabled: `True`). {{(pid=88114) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:48:44.975972 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.fernet_utils [None req-e707b1b5-a84d-4d10-b04f-3df628c1d1ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88114) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:48:45.063625 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a863bd20-f55d-4832-bbc6-8267f0a45939 admin admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:48:45.069555 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa5b821-fb31-4eda-8804-77e335771c21 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9bdaacd597442c2bcde2c555f2cc140', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '49548eb44d62415582af820ab66decee', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:48:45.071034 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa5b821-fb31-4eda-8804-77e335771c21 service neutron] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:45.071658 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa5b821-fb31-4eda-8804-77e335771c21 service neutron] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:45.072117 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa5b821-fb31-4eda-8804-77e335771c21 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:45.105886 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aa5b821-fb31-4eda-8804-77e335771c21 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:48:45.108669 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aa5b821-fb31-4eda-8804-77e335771c21 service neutron] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:48:45.125644 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 975/1948] 10.222.0.13 () {60 vars in 1365 bytes} [Wed Aug 31 09:48:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:48:45.127556 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b22ae6-5241-4fcf-8069-52865f2a5c95 None None] REQUEST_METHOD: `POST` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:45.127832 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b22ae6-5241-4fcf-8069-52865f2a5c95 None None] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:45.128113 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b22ae6-5241-4fcf-8069-52865f2a5c95 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:45.153246 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 974/1949] 10.222.0.13 () {58 vars in 958 bytes} [Wed Aug 31 09:48:44 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 244 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:48:45.153246 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e707b1b5-a84d-4d10-b04f-3df628c1d1ae None None] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:48:45.162372 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.auth.core [None req-02b22ae6-5241-4fcf-8069-52865f2a5c95 None None] MFA Rules not processed for user `fd836cf590a145df9acf19cf3ae2ac56`. Rule list: `[]` (Enabled: `True`). {{(pid=88113) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:48:45.166666 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.fernet_utils [None req-02b22ae6-5241-4fcf-8069-52865f2a5c95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:48:45.168456 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e707b1b5-a84d-4d10-b04f-3df628c1d1ae None None] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:48:45.175611 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a619ac8f-17e0-41a3-8f58-ae80197d663b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:48:45.176532 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a619ac8f-17e0-41a3-8f58-ae80197d663b admin admin] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:45.176818 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a619ac8f-17e0-41a3-8f58-ae80197d663b admin admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:45.177089 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a619ac8f-17e0-41a3-8f58-ae80197d663b admin admin] PATH_INFO: `/v3/roles` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:45.177813 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a619ac8f-17e0-41a3-8f58-ae80197d663b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:48:45.179702 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a619ac8f-17e0-41a3-8f58-ae80197d663b admin admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:48:45.186274 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 976/1950] 10.222.0.13 () {58 vars in 958 bytes} [Wed Aug 31 09:48:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 60 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:48:45.188453 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b22ae6-5241-4fcf-8069-52865f2a5c95 None None] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:48:45.202278 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 975/1951] 10.222.0.13 () {58 vars in 1122 bytes} [Wed Aug 31 09:48:45 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 09:48:45.202278 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a619ac8f-17e0-41a3-8f58-ae80197d663b admin admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:48:45.231303 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a619ac8f-17e0-41a3-8f58-ae80197d663b admin admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:48:45.234175 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b22ae6-5241-4fcf-8069-52865f2a5c95 None None] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:48:45.245031 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ccf613-4165-4b78-97eb-c5e750f80660 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:48:45.246089 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ccf613-4165-4b78-97eb-c5e750f80660 admin admin] REQUEST_METHOD: `POST` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:45.246089 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ccf613-4165-4b78-97eb-c5e750f80660 admin admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:45.246331 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ccf613-4165-4b78-97eb-c5e750f80660 admin admin] PATH_INFO: `/v3/projects` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:45.246910 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24ccf613-4165-4b78-97eb-c5e750f80660 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:48:45.249036 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24ccf613-4165-4b78-97eb-c5e750f80660 admin admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:48:45.250916 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e8bac9-7f1b-4e62-a866-06bf7f71adf2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:48:45.252992 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e8bac9-7f1b-4e62-a866-06bf7f71adf2 admin admin] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:45.254420 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e8bac9-7f1b-4e62-a866-06bf7f71adf2 admin admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:45.255902 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e8bac9-7f1b-4e62-a866-06bf7f71adf2 admin admin] PATH_INFO: `/v3/domains` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:45.258974 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4e8bac9-7f1b-4e62-a866-06bf7f71adf2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88114) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:48:45.259920 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4e8bac9-7f1b-4e62-a866-06bf7f71adf2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:48:45.264458 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4e8bac9-7f1b-4e62-a866-06bf7f71adf2 admin admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:48:45.296864 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 976/1952] 10.222.0.13 () {58 vars in 1153 bytes} [Wed Aug 31 09:48:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:48:45.296864 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e8bac9-7f1b-4e62-a866-06bf7f71adf2 admin admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:48:45.305357 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 977/1953] 10.222.0.13 () {60 vars in 1153 bytes} [Wed Aug 31 09:48:45 2022] POST /identity/v3/projects => generated 377 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 09:48:45.307000 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ccf613-4165-4b78-97eb-c5e750f80660 admin admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:48:45.324179 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e8bac9-7f1b-4e62-a866-06bf7f71adf2 admin admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:48:45.484345 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e7090a-1baa-4286-a522-663fa1423480 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:48:45.485762 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e7090a-1baa-4286-a522-663fa1423480 admin admin] REQUEST_METHOD: `PUT` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:45.486797 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e7090a-1baa-4286-a522-663fa1423480 admin admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:45.486797 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e7090a-1baa-4286-a522-663fa1423480 admin admin] PATH_INFO: `/v3/projects/8437124192c34160bc1fba7bdd86e90c/users/5e935eba26644bffb17aa00255ef99ea/roles/6c99f5d9f1f243eaa134d8540f21095d` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:45.510106 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ccf613-4165-4b78-97eb-c5e750f80660 admin admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:48:45.513665 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9e7090a-1baa-4286-a522-663fa1423480 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8437124192c34160bc1fba7bdd86e90c, user_id=5e935eba26644bffb17aa00255ef99ea, role_id=6c99f5d9f1f243eaa134d8540f21095d)` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:48:45.518269 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9e7090a-1baa-4286-a522-663fa1423480 admin admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:48:45.520213 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c1ff28-50fb-49de-9070-d4f04cf32dd2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:48:45.521447 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c1ff28-50fb-49de-9070-d4f04cf32dd2 admin admin] REQUEST_METHOD: `POST` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:45.521996 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c1ff28-50fb-49de-9070-d4f04cf32dd2 admin admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:45.522635 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c1ff28-50fb-49de-9070-d4f04cf32dd2 admin admin] PATH_INFO: `/v3/projects` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:45.523969 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4c1ff28-50fb-49de-9070-d4f04cf32dd2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:48:45.528325 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4c1ff28-50fb-49de-9070-d4f04cf32dd2 admin admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:48:45.543681 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 977/1954] 10.222.0.13 () {60 vars in 1483 bytes} [Wed Aug 31 09:48:45 2022] PUT /identity/v3/projects/8437124192c34160bc1fba7bdd86e90c/users/5e935eba26644bffb17aa00255ef99ea/roles/6c99f5d9f1f243eaa134d8540f21095d => generated 0 bytes in 249 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:48:45.544684 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e7090a-1baa-4286-a522-663fa1423480 admin admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:48:45.584347 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:48:45.598539 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 978/1955] 10.222.0.13 () {60 vars in 1153 bytes} [Wed Aug 31 09:48:45 2022] POST /identity/v3/projects => generated 369 bytes in 292 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 09:48:45.602151 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb4562a-98b0-44a0-92ef-fdb9ff60be35 None None] REQUEST_METHOD: `POST` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:45.602901 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb4562a-98b0-44a0-92ef-fdb9ff60be35 None None] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:45.603510 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb4562a-98b0-44a0-92ef-fdb9ff60be35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:45.656009 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.auth.core [None req-3cb4562a-98b0-44a0-92ef-fdb9ff60be35 None None] MFA Rules not processed for user `5e935eba26644bffb17aa00255ef99ea`. Rule list: `[]` (Enabled: `True`). {{(pid=88113) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:48:45.656288 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.fernet_utils [None req-3cb4562a-98b0-44a0-92ef-fdb9ff60be35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:48:45.663469 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e7090a-1baa-4286-a522-663fa1423480 admin admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:48:45.667967 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:48:45.774059 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 979/1956] 10.222.0.13 () {58 vars in 958 bytes} [Wed Aug 31 09:48:45 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:48:45.776026 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb4562a-98b0-44a0-92ef-fdb9ff60be35 None None] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:48:45.788363 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0d9e30d-b1f1-4d90-bf4f-d5d41b582080 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:48:45.789040 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d9e30d-b1f1-4d90-bf4f-d5d41b582080 admin admin] REQUEST_METHOD: `POST` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:45.789320 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d9e30d-b1f1-4d90-bf4f-d5d41b582080 admin admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:45.789646 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d9e30d-b1f1-4d90-bf4f-d5d41b582080 admin admin] PATH_INFO: `/v3/users` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:45.790313 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0d9e30d-b1f1-4d90-bf4f-d5d41b582080 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:48:45.792722 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0d9e30d-b1f1-4d90-bf4f-d5d41b582080 admin admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:48:45.835130 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 978/1957] 10.222.0.13 () {60 vars in 1144 bytes} [Wed Aug 31 09:48:45 2022] POST /identity/v3/users => generated 289 bytes in 287 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 09:48:45.835130 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0d9e30d-b1f1-4d90-bf4f-d5d41b582080 admin admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:48:45.847999 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0d9e30d-b1f1-4d90-bf4f-d5d41b582080 admin admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:48:45.856244 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-290062ea-e62b-42dc-9ec8-f1b915ed2388 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:48:45.857368 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290062ea-e62b-42dc-9ec8-f1b915ed2388 admin admin] REQUEST_METHOD: `POST` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:45.857788 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290062ea-e62b-42dc-9ec8-f1b915ed2388 admin admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:45.858070 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290062ea-e62b-42dc-9ec8-f1b915ed2388 admin admin] PATH_INFO: `/v3/projects` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:45.858767 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-290062ea-e62b-42dc-9ec8-f1b915ed2388 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:48:45.863959 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-290062ea-e62b-42dc-9ec8-f1b915ed2388 admin admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:48:45.931281 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 979/1958] 10.222.0.13 () {60 vars in 1153 bytes} [Wed Aug 31 09:48:45 2022] POST /identity/v3/projects => generated 373 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 09:48:45.931281 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-290062ea-e62b-42dc-9ec8-f1b915ed2388 admin admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:48:45.960720 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb4562a-98b0-44a0-92ef-fdb9ff60be35 None None] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:48:46.094959 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-177fd189-9cb3-461d-80c8-0746881b522f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:48:46.096393 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177fd189-9cb3-461d-80c8-0746881b522f admin admin] REQUEST_METHOD: `POST` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:46.097579 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177fd189-9cb3-461d-80c8-0746881b522f admin admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:46.098207 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177fd189-9cb3-461d-80c8-0746881b522f admin admin] PATH_INFO: `/v3/users` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:46.099593 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-177fd189-9cb3-461d-80c8-0746881b522f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:48:46.103665 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-177fd189-9cb3-461d-80c8-0746881b522f admin admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:48:46.162452 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 980/1959] 10.222.0.13 () {60 vars in 1144 bytes} [Wed Aug 31 09:48:45 2022] POST /identity/v3/users => generated 285 bytes in 385 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 09:48:46.162780 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-177fd189-9cb3-461d-80c8-0746881b522f admin admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:48:46.182871 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-177fd189-9cb3-461d-80c8-0746881b522f admin admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:48:46.200598 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd6bc3fb-adc3-478a-bd28-247db2ba0e6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:48:46.201554 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6bc3fb-adc3-478a-bd28-247db2ba0e6d admin admin] REQUEST_METHOD: `POST` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:46.202070 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6bc3fb-adc3-478a-bd28-247db2ba0e6d admin admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:46.202444 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6bc3fb-adc3-478a-bd28-247db2ba0e6d admin admin] PATH_INFO: `/v3/users` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:46.203221 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd6bc3fb-adc3-478a-bd28-247db2ba0e6d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:48:46.208599 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-290062ea-e62b-42dc-9ec8-f1b915ed2388 admin admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:48:46.209312 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd6bc3fb-adc3-478a-bd28-247db2ba0e6d admin admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:48:46.218532 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2073a598-3289-4d57-9120-033c1bbf4c9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:48:46.219781 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2073a598-3289-4d57-9120-033c1bbf4c9e admin admin] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:46.220410 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2073a598-3289-4d57-9120-033c1bbf4c9e admin admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:46.221204 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2073a598-3289-4d57-9120-033c1bbf4c9e admin admin] PATH_INFO: `/v3/roles` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:46.222904 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2073a598-3289-4d57-9120-033c1bbf4c9e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:48:46.225595 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2073a598-3289-4d57-9120-033c1bbf4c9e admin admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:48:46.260736 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 981/1960] 10.222.0.13 () {60 vars in 1144 bytes} [Wed Aug 31 09:48:46 2022] POST /identity/v3/users => generated 293 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 09:48:46.262733 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd6bc3fb-adc3-478a-bd28-247db2ba0e6d admin admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:48:46.266249 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 980/1961] 10.222.0.13 () {58 vars in 1122 bytes} [Wed Aug 31 09:48:45 2022] GET /identity/v3/roles => generated 1420 bytes in 339 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 09:48:46.270951 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2073a598-3289-4d57-9120-033c1bbf4c9e admin admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:48:46.286367 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd6bc3fb-adc3-478a-bd28-247db2ba0e6d admin admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:48:46.295134 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2073a598-3289-4d57-9120-033c1bbf4c9e admin admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:48:46.305242 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c3c0c2-3638-40aa-b72e-a7c0ec39c79b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:48:46.305579 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db357788-0c1a-4991-bceb-775bb5fc12c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:48:46.306794 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c3c0c2-3638-40aa-b72e-a7c0ec39c79b admin admin] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:46.307535 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c3c0c2-3638-40aa-b72e-a7c0ec39c79b admin admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:46.308359 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c3c0c2-3638-40aa-b72e-a7c0ec39c79b admin admin] PATH_INFO: `/v3/roles` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:46.309612 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db357788-0c1a-4991-bceb-775bb5fc12c1 admin admin] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:46.310010 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80c3c0c2-3638-40aa-b72e-a7c0ec39c79b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:48:46.311137 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db357788-0c1a-4991-bceb-775bb5fc12c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:46.311137 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db357788-0c1a-4991-bceb-775bb5fc12c1 admin admin] PATH_INFO: `/v3/roles` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:46.312105 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db357788-0c1a-4991-bceb-775bb5fc12c1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:48:46.315330 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db357788-0c1a-4991-bceb-775bb5fc12c1 admin admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:48:46.321889 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80c3c0c2-3638-40aa-b72e-a7c0ec39c79b admin admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:48:46.346960 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 981/1962] 10.222.0.13 () {58 vars in 1122 bytes} [Wed Aug 31 09:48:46 2022] GET /identity/v3/roles => generated 1420 bytes in 74 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 09:48:46.346960 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db357788-0c1a-4991-bceb-775bb5fc12c1 admin admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:48:46.367152 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db357788-0c1a-4991-bceb-775bb5fc12c1 admin admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:48:46.367687 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 982/1963] 10.222.0.13 () {58 vars in 1122 bytes} [Wed Aug 31 09:48:46 2022] GET /identity/v3/roles => generated 1420 bytes in 106 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 09:48:46.371555 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e9d565-5fbf-443a-adb5-8fc08356f68e None None] REQUEST_METHOD: `POST` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:46.372332 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e9d565-5fbf-443a-adb5-8fc08356f68e None None] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:46.372814 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e9d565-5fbf-443a-adb5-8fc08356f68e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:46.387133 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2792e19d-c0db-4440-9ebc-5a966cc6fe14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:48:46.388176 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2792e19d-c0db-4440-9ebc-5a966cc6fe14 admin admin] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:46.388674 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2792e19d-c0db-4440-9ebc-5a966cc6fe14 admin admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:46.389546 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2792e19d-c0db-4440-9ebc-5a966cc6fe14 admin admin] PATH_INFO: `/v3/roles` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:46.390990 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2792e19d-c0db-4440-9ebc-5a966cc6fe14 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:48:46.394319 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2792e19d-c0db-4440-9ebc-5a966cc6fe14 admin admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:48:46.421413 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 982/1964] 10.222.0.13 () {58 vars in 1122 bytes} [Wed Aug 31 09:48:46 2022] GET /identity/v3/roles => generated 1420 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 09:48:46.422911 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2792e19d-c0db-4440-9ebc-5a966cc6fe14 admin admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:48:46.424180 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.auth.core [None req-66e9d565-5fbf-443a-adb5-8fc08356f68e None None] MFA Rules not processed for user `c475984de9e84ff18fb96c588a07c80b`. Rule list: `[]` (Enabled: `True`). {{(pid=88113) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:48:46.426213 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.fernet_utils [None req-66e9d565-5fbf-443a-adb5-8fc08356f68e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:48:46.433657 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:48:46.457263 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2792e19d-c0db-4440-9ebc-5a966cc6fe14 admin admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:48:46.471935 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e92c9513-481f-4905-a897-59d218e6caa0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:48:46.471935 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92c9513-481f-4905-a897-59d218e6caa0 admin admin] REQUEST_METHOD: `PUT` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:46.471935 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92c9513-481f-4905-a897-59d218e6caa0 admin admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:46.471935 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92c9513-481f-4905-a897-59d218e6caa0 admin admin] PATH_INFO: `/v3/projects/79bb1d5c92494216a2805a642f4e8084/users/d95b5de8d1f4471ab8bd563150447b95/roles/6c99f5d9f1f243eaa134d8540f21095d` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:46.472634 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:48:46.523308 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e92c9513-481f-4905-a897-59d218e6caa0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=79bb1d5c92494216a2805a642f4e8084, user_id=d95b5de8d1f4471ab8bd563150447b95, role_id=6c99f5d9f1f243eaa134d8540f21095d)` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:48:46.531811 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e92c9513-481f-4905-a897-59d218e6caa0 admin admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:48:46.572384 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 983/1965] 10.222.0.13 () {60 vars in 1483 bytes} [Wed Aug 31 09:48:46 2022] PUT /identity/v3/projects/79bb1d5c92494216a2805a642f4e8084/users/d95b5de8d1f4471ab8bd563150447b95/roles/6c99f5d9f1f243eaa134d8540f21095d => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:48:46.573009 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e92c9513-481f-4905-a897-59d218e6caa0 admin admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:48:46.586643 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 983/1966] 10.222.0.13 () {58 vars in 958 bytes} [Wed Aug 31 09:48:46 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 218 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:48:46.587684 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e9d565-5fbf-443a-adb5-8fc08356f68e None None] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:48:46.614380 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e9d565-5fbf-443a-adb5-8fc08356f68e None None] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:48:46.701622 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95cea4ef-e245-41d2-93e4-fe86334c1f14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:48:46.702802 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95cea4ef-e245-41d2-93e4-fe86334c1f14 admin admin] REQUEST_METHOD: `PUT` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:46.703186 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95cea4ef-e245-41d2-93e4-fe86334c1f14 admin admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:46.703716 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95cea4ef-e245-41d2-93e4-fe86334c1f14 admin admin] PATH_INFO: `/v3/projects/8f9b66eb8c024cbf8ec51f00ffed6787/users/e5d116b0755440da976b1f782f7c504b/roles/5d4ee63a3add47fc9a6b8b16d39e0a1f` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:46.714530 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e92c9513-481f-4905-a897-59d218e6caa0 admin admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:48:46.720796 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb10d62-7978-4d10-90a6-d22382cb322f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:48:46.721491 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb10d62-7978-4d10-90a6-d22382cb322f admin admin] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:46.721779 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb10d62-7978-4d10-90a6-d22382cb322f admin admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:46.727285 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb10d62-7978-4d10-90a6-d22382cb322f admin admin] PATH_INFO: `/v3/roles` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:46.727285 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddb10d62-7978-4d10-90a6-d22382cb322f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:48:46.727285 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddb10d62-7978-4d10-90a6-d22382cb322f admin admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:48:46.740432 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95cea4ef-e245-41d2-93e4-fe86334c1f14 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8f9b66eb8c024cbf8ec51f00ffed6787, user_id=e5d116b0755440da976b1f782f7c504b, role_id=5d4ee63a3add47fc9a6b8b16d39e0a1f)` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:48:46.741804 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 984/1967] 10.222.0.13 () {58 vars in 1122 bytes} [Wed Aug 31 09:48:46 2022] GET /identity/v3/roles => generated 1420 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 09:48:46.742786 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb10d62-7978-4d10-90a6-d22382cb322f admin admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:48:46.744397 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95cea4ef-e245-41d2-93e4-fe86334c1f14 admin admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:48:46.744654 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.fernet_utils [None req-ddb10d62-7978-4d10-90a6-d22382cb322f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88114) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:48:46.773918 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 984/1968] 10.222.0.13 () {60 vars in 1483 bytes} [Wed Aug 31 09:48:46 2022] PUT /identity/v3/projects/8f9b66eb8c024cbf8ec51f00ffed6787/users/e5d116b0755440da976b1f782f7c504b/roles/5d4ee63a3add47fc9a6b8b16d39e0a1f => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:48:46.775552 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc633e4-d58f-4d45-a590-7e6794a2ae69 None None] REQUEST_METHOD: `POST` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:46.776163 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc633e4-d58f-4d45-a590-7e6794a2ae69 None None] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:46.776467 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc633e4-d58f-4d45-a590-7e6794a2ae69 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:46.817706 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.auth.core [None req-acc633e4-d58f-4d45-a590-7e6794a2ae69 None None] MFA Rules not processed for user `d95b5de8d1f4471ab8bd563150447b95`. Rule list: `[]` (Enabled: `True`). {{(pid=88113) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:48:46.819914 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.fernet_utils [None req-acc633e4-d58f-4d45-a590-7e6794a2ae69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:48:46.849080 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb10d62-7978-4d10-90a6-d22382cb322f admin admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:48:46.927547 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 985/1969] 10.222.0.13 () {58 vars in 958 bytes} [Wed Aug 31 09:48:46 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:48:46.928578 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc633e4-d58f-4d45-a590-7e6794a2ae69 None None] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:48:46.960146 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:48:46.970418 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00502997-852e-40b0-ae3b-a65e836cd2fb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d52b69c8e94441dacfcff944326f169', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '49548eb44d62415582af820ab66decee', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:48:46.971195 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00502997-852e-40b0-ae3b-a65e836cd2fb service cinder] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:46.971560 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00502997-852e-40b0-ae3b-a65e836cd2fb service cinder] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:46.971896 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00502997-852e-40b0-ae3b-a65e836cd2fb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:47.062344 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc633e4-d58f-4d45-a590-7e6794a2ae69 None None] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:48:47.063676 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:48:47.068568 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-147f1e61-8d91-4533-aea0-7ab4a21bea6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:48:47.069508 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147f1e61-8d91-4533-aea0-7ab4a21bea6a admin admin] REQUEST_METHOD: `PUT` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:47.069802 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147f1e61-8d91-4533-aea0-7ab4a21bea6a admin admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:47.070075 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147f1e61-8d91-4533-aea0-7ab4a21bea6a admin admin] PATH_INFO: `/v3/projects/737dc8a01dfe4ba1ba9239e4d183eccd/users/bf8707b404674f8f999976d2160f92ea/roles/6c99f5d9f1f243eaa134d8540f21095d` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:47.095473 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00502997-852e-40b0-ae3b-a65e836cd2fb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:48:47.097529 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-147f1e61-8d91-4533-aea0-7ab4a21bea6a admin admin] RBAC: Authorizing `identity:create_grant(project_id=737dc8a01dfe4ba1ba9239e4d183eccd, user_id=bf8707b404674f8f999976d2160f92ea, role_id=6c99f5d9f1f243eaa134d8540f21095d)` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:48:47.099499 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00502997-852e-40b0-ae3b-a65e836cd2fb service cinder] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:48:47.102316 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-147f1e61-8d91-4533-aea0-7ab4a21bea6a admin admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:48:47.125773 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 985/1970] 10.222.0.13 () {60 vars in 1365 bytes} [Wed Aug 31 09:48:46 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 383 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:48:47.127974 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00502997-852e-40b0-ae3b-a65e836cd2fb service cinder] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:48:47.141054 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 986/1971] 10.222.0.13 () {60 vars in 1483 bytes} [Wed Aug 31 09:48:46 2022] PUT /identity/v3/projects/737dc8a01dfe4ba1ba9239e4d183eccd/users/bf8707b404674f8f999976d2160f92ea/roles/6c99f5d9f1f243eaa134d8540f21095d => generated 0 bytes in 213 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:48:47.143409 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-147f1e61-8d91-4533-aea0-7ab4a21bea6a admin admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:48:47.172819 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00502997-852e-40b0-ae3b-a65e836cd2fb service cinder] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:48:47.303060 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-147f1e61-8d91-4533-aea0-7ab4a21bea6a admin admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:48:47.306171 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-923cd6c0-b344-4045-8bd4-086db13b113a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:48:47.307204 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923cd6c0-b344-4045-8bd4-086db13b113a admin admin] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:47.307746 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923cd6c0-b344-4045-8bd4-086db13b113a admin admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:47.308269 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923cd6c0-b344-4045-8bd4-086db13b113a admin admin] PATH_INFO: `/v3/roles` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:47.309491 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-923cd6c0-b344-4045-8bd4-086db13b113a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:48:47.310736 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e3ddbb-da25-4c1e-8c60-0379f193988e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:48:47.311921 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e3ddbb-da25-4c1e-8c60-0379f193988e admin admin] REQUEST_METHOD: `POST` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:47.312391 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-923cd6c0-b344-4045-8bd4-086db13b113a admin admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:48:47.312491 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e3ddbb-da25-4c1e-8c60-0379f193988e admin admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:47.313141 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e3ddbb-da25-4c1e-8c60-0379f193988e admin admin] PATH_INFO: `/v3/projects` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:47.315417 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02e3ddbb-da25-4c1e-8c60-0379f193988e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:48:47.318976 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02e3ddbb-da25-4c1e-8c60-0379f193988e admin admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:48:47.337656 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 986/1972] 10.222.0.13 () {58 vars in 1122 bytes} [Wed Aug 31 09:48:47 2022] GET /identity/v3/roles => generated 1420 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 09:48:47.340731 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2ee1d5-ca02-4105-8e1f-b83348685c61 None None] REQUEST_METHOD: `POST` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:47.341623 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2ee1d5-ca02-4105-8e1f-b83348685c61 None None] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:47.342307 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2ee1d5-ca02-4105-8e1f-b83348685c61 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:47.373613 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 987/1973] 10.222.0.13 () {60 vars in 1153 bytes} [Wed Aug 31 09:48:47 2022] POST /identity/v3/projects => generated 375 bytes in 232 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 09:48:47.375273 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e3ddbb-da25-4c1e-8c60-0379f193988e admin admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:48:47.389918 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.auth.core [None req-dc2ee1d5-ca02-4105-8e1f-b83348685c61 None None] MFA Rules not processed for user `bf8707b404674f8f999976d2160f92ea`. Rule list: `[]` (Enabled: `True`). {{(pid=88114) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:48:47.391748 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.fernet_utils [None req-dc2ee1d5-ca02-4105-8e1f-b83348685c61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88114) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:48:47.584030 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e3ddbb-da25-4c1e-8c60-0379f193988e admin admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:48:47.598028 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 987/1974] 10.222.0.13 () {58 vars in 958 bytes} [Wed Aug 31 09:48:47 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 259 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:48:47.601264 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc2ee1d5-ca02-4105-8e1f-b83348685c61 None None] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:48:47.607227 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e888e5ef-0f67-44fb-a6b7-1344160d9939 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:48:47.609030 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e888e5ef-0f67-44fb-a6b7-1344160d9939 admin admin] REQUEST_METHOD: `PUT` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:47.610344 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e888e5ef-0f67-44fb-a6b7-1344160d9939 admin admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:47.610536 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e888e5ef-0f67-44fb-a6b7-1344160d9939 admin admin] PATH_INFO: `/v3/projects/8f9b66eb8c024cbf8ec51f00ffed6787/users/e5d116b0755440da976b1f782f7c504b/roles/5d4ee63a3add47fc9a6b8b16d39e0a1f` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:47.615619 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e888e5ef-0f67-44fb-a6b7-1344160d9939 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8f9b66eb8c024cbf8ec51f00ffed6787, user_id=e5d116b0755440da976b1f782f7c504b, role_id=5d4ee63a3add47fc9a6b8b16d39e0a1f)` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:48:47.622468 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e888e5ef-0f67-44fb-a6b7-1344160d9939 admin admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:48:47.635399 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc2ee1d5-ca02-4105-8e1f-b83348685c61 None None] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:48:47.659018 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 988/1975] 10.222.0.13 () {60 vars in 1483 bytes} [Wed Aug 31 09:48:47 2022] PUT /identity/v3/projects/8f9b66eb8c024cbf8ec51f00ffed6787/users/e5d116b0755440da976b1f782f7c504b/roles/5d4ee63a3add47fc9a6b8b16d39e0a1f => generated 0 bytes in 284 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:48:47.663350 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e888e5ef-0f67-44fb-a6b7-1344160d9939 admin admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:48:47.726116 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:48:47.742916 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bcf8714-d5d4-4cdd-9071-3a18aab650f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:48:47.743570 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bcf8714-d5d4-4cdd-9071-3a18aab650f6 admin admin] REQUEST_METHOD: `POST` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:47.743863 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bcf8714-d5d4-4cdd-9071-3a18aab650f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:47.744147 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bcf8714-d5d4-4cdd-9071-3a18aab650f6 admin admin] PATH_INFO: `/v3/users` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:47.744940 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bcf8714-d5d4-4cdd-9071-3a18aab650f6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:48:47.747335 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bcf8714-d5d4-4cdd-9071-3a18aab650f6 admin admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:48:47.787251 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 988/1976] 10.222.0.13 () {60 vars in 1144 bytes} [Wed Aug 31 09:48:47 2022] POST /identity/v3/users => generated 294 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 09:48:47.787251 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d01f8f8-8efb-4613-b3b5-585dccb123bb None None] REQUEST_METHOD: `POST` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:47.787251 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d01f8f8-8efb-4613-b3b5-585dccb123bb None None] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:47.787251 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d01f8f8-8efb-4613-b3b5-585dccb123bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:47.819615 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:48:47.849111 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.auth.core [None req-8d01f8f8-8efb-4613-b3b5-585dccb123bb None None] MFA Rules not processed for user `e5d116b0755440da976b1f782f7c504b`. Rule list: `[]` (Enabled: `True`). {{(pid=88114) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:48:47.856676 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.fernet_utils [None req-8d01f8f8-8efb-4613-b3b5-585dccb123bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88114) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:48:47.864056 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e888e5ef-0f67-44fb-a6b7-1344160d9939 admin admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:48:47.883519 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc7f1fe-c15d-44cd-8c17-670093da57c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:48:47.883519 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc7f1fe-c15d-44cd-8c17-670093da57c2 admin admin] REQUEST_METHOD: `POST` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:47.883519 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc7f1fe-c15d-44cd-8c17-670093da57c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:47.883519 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc7f1fe-c15d-44cd-8c17-670093da57c2 admin admin] PATH_INFO: `/v3/projects` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:47.888551 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbc7f1fe-c15d-44cd-8c17-670093da57c2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:48:47.890342 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbc7f1fe-c15d-44cd-8c17-670093da57c2 admin admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:48:47.999507 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 989/1977] 10.222.0.13 () {58 vars in 958 bytes} [Wed Aug 31 09:48:47 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:48:47.999507 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d01f8f8-8efb-4613-b3b5-585dccb123bb None None] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:48:48.027741 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d01f8f8-8efb-4613-b3b5-585dccb123bb None None] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:48:48.031721 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 989/1978] 10.222.0.13 () {60 vars in 1153 bytes} [Wed Aug 31 09:48:47 2022] POST /identity/v3/projects => generated 367 bytes in 370 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 09:48:48.033723 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc7f1fe-c15d-44cd-8c17-670093da57c2 admin admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:48:48.136177 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-501f8d8b-c853-4aa0-8a93-60119c90bc80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:48:48.137330 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501f8d8b-c853-4aa0-8a93-60119c90bc80 admin admin] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:48.137938 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501f8d8b-c853-4aa0-8a93-60119c90bc80 admin admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:48.138492 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501f8d8b-c853-4aa0-8a93-60119c90bc80 admin admin] PATH_INFO: `/v3/roles` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:48.139817 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-501f8d8b-c853-4aa0-8a93-60119c90bc80 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:48:48.143005 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-501f8d8b-c853-4aa0-8a93-60119c90bc80 admin admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:48:48.151691 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc7f1fe-c15d-44cd-8c17-670093da57c2 admin admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:48:48.159177 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-862d6006-c935-4621-9100-543cdf1a3373 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d52b69c8e94441dacfcff944326f169', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '49548eb44d62415582af820ab66decee', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:48:48.160836 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862d6006-c935-4621-9100-543cdf1a3373 service cinder] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:48.161301 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862d6006-c935-4621-9100-543cdf1a3373 service cinder] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:48.162092 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862d6006-c935-4621-9100-543cdf1a3373 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:48.170508 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 990/1979] 10.222.0.13 () {58 vars in 1122 bytes} [Wed Aug 31 09:48:47 2022] GET /identity/v3/roles => generated 1420 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 09:48:48.170508 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-501f8d8b-c853-4aa0-8a93-60119c90bc80 admin admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:48:48.188945 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-501f8d8b-c853-4aa0-8a93-60119c90bc80 admin admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:48:48.197964 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-514af791-51c7-4828-ad91-63b9b910c102 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:48:48.199152 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514af791-51c7-4828-ad91-63b9b910c102 admin admin] REQUEST_METHOD: `POST` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:48.199729 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514af791-51c7-4828-ad91-63b9b910c102 admin admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:48.200339 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514af791-51c7-4828-ad91-63b9b910c102 admin admin] PATH_INFO: `/v3/users` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:48.201662 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-514af791-51c7-4828-ad91-63b9b910c102 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:48:48.205716 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-514af791-51c7-4828-ad91-63b9b910c102 admin admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:48:48.242797 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 991/1980] 10.222.0.13 () {60 vars in 1144 bytes} [Wed Aug 31 09:48:48 2022] POST /identity/v3/users => generated 290 bytes in 76 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 09:48:48.244430 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-514af791-51c7-4828-ad91-63b9b910c102 admin admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:48:48.252835 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-862d6006-c935-4621-9100-543cdf1a3373 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:48:48.255049 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-862d6006-c935-4621-9100-543cdf1a3373 service cinder] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:48:48.266399 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 990/1981] 10.222.0.13 () {60 vars in 1365 bytes} [Wed Aug 31 09:48:48 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 234 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:48:48.267357 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-862d6006-c935-4621-9100-543cdf1a3373 service cinder] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:48:48.267880 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-514af791-51c7-4828-ad91-63b9b910c102 admin admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:48:48.277764 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ab36c2-b74c-4d0f-90dc-83b5601f8822 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:48:48.279035 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ab36c2-b74c-4d0f-90dc-83b5601f8822 admin admin] REQUEST_METHOD: `PUT` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:48.279538 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ab36c2-b74c-4d0f-90dc-83b5601f8822 admin admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:48.280043 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ab36c2-b74c-4d0f-90dc-83b5601f8822 admin admin] PATH_INFO: `/v3/projects/6254c9e845bf4d01b15882e06338098b/users/9c401cde27794c009a6e13250daf83dc/roles/5d4ee63a3add47fc9a6b8b16d39e0a1f` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:48.281975 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-862d6006-c935-4621-9100-543cdf1a3373 service cinder] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:48:48.290707 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4322ce54-316a-43d3-b95a-7315a34fb933 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:48:48.291486 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4322ce54-316a-43d3-b95a-7315a34fb933 admin admin] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:48.291764 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4322ce54-316a-43d3-b95a-7315a34fb933 admin admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:48.292011 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4322ce54-316a-43d3-b95a-7315a34fb933 admin admin] PATH_INFO: `/v3/roles` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:48.293595 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4322ce54-316a-43d3-b95a-7315a34fb933 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:48:48.295317 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4322ce54-316a-43d3-b95a-7315a34fb933 admin admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:48:48.317947 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98ab36c2-b74c-4d0f-90dc-83b5601f8822 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6254c9e845bf4d01b15882e06338098b, user_id=9c401cde27794c009a6e13250daf83dc, role_id=5d4ee63a3add47fc9a6b8b16d39e0a1f)` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:48:48.325258 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 991/1982] 10.222.0.13 () {58 vars in 1122 bytes} [Wed Aug 31 09:48:48 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 09:48:48.326177 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98ab36c2-b74c-4d0f-90dc-83b5601f8822 admin admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:48:48.328138 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4322ce54-316a-43d3-b95a-7315a34fb933 admin admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:48:48.352850 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 992/1983] 10.222.0.13 () {60 vars in 1483 bytes} [Wed Aug 31 09:48:48 2022] PUT /identity/v3/projects/6254c9e845bf4d01b15882e06338098b/users/9c401cde27794c009a6e13250daf83dc/roles/5d4ee63a3add47fc9a6b8b16d39e0a1f => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:48:48.355679 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4322ce54-316a-43d3-b95a-7315a34fb933 admin admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:48:48.359454 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ab36c2-b74c-4d0f-90dc-83b5601f8822 admin admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:48:48.474206 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:48:48.505995 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:48:48.515217 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0935b764-5e41-4881-899a-58c0e9da6a8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:48:48.516305 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0935b764-5e41-4881-899a-58c0e9da6a8d admin admin] REQUEST_METHOD: `PUT` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:48.516786 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0935b764-5e41-4881-899a-58c0e9da6a8d admin admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:48.517930 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0935b764-5e41-4881-899a-58c0e9da6a8d admin admin] PATH_INFO: `/v3/projects/fc3b8644859b4e73878b2156defc9563/users/c81e3fd1717c45ceb2082c75bd8a59a3/roles/5d4ee63a3add47fc9a6b8b16d39e0a1f` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:48.527943 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ab36c2-b74c-4d0f-90dc-83b5601f8822 admin admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:48:48.533518 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf57eff-7682-4eb9-add7-d76b4d239c5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:48:48.534104 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf57eff-7682-4eb9-add7-d76b4d239c5a admin admin] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:48.534436 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf57eff-7682-4eb9-add7-d76b4d239c5a admin admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:48.535374 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf57eff-7682-4eb9-add7-d76b4d239c5a admin admin] PATH_INFO: `/v3/roles` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:48.536085 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bf57eff-7682-4eb9-add7-d76b4d239c5a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:48:48.549897 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bf57eff-7682-4eb9-add7-d76b4d239c5a admin admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:48:48.571197 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 993/1984] 10.222.0.13 () {58 vars in 1122 bytes} [Wed Aug 31 09:48:48 2022] GET /identity/v3/roles => generated 1420 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 09:48:48.571860 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0935b764-5e41-4881-899a-58c0e9da6a8d admin admin] RBAC: Authorizing `identity:create_grant(project_id=fc3b8644859b4e73878b2156defc9563, user_id=c81e3fd1717c45ceb2082c75bd8a59a3, role_id=5d4ee63a3add47fc9a6b8b16d39e0a1f)` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:48:48.575858 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf57eff-7682-4eb9-add7-d76b4d239c5a admin admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:48:48.581901 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0935b764-5e41-4881-899a-58c0e9da6a8d admin admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:48:48.589241 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf57eff-7682-4eb9-add7-d76b4d239c5a admin admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:48:48.600817 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c702820-8c01-4ae3-854f-0cc64b5131be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:48:48.602367 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c702820-8c01-4ae3-854f-0cc64b5131be admin admin] REQUEST_METHOD: `PUT` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:48.602886 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c702820-8c01-4ae3-854f-0cc64b5131be admin admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:48.603396 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c702820-8c01-4ae3-854f-0cc64b5131be admin admin] PATH_INFO: `/v3/projects/6254c9e845bf4d01b15882e06338098b/users/9c401cde27794c009a6e13250daf83dc/roles/5d4ee63a3add47fc9a6b8b16d39e0a1f` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:48.607902 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c702820-8c01-4ae3-854f-0cc64b5131be admin admin] RBAC: Authorizing `identity:create_grant(project_id=6254c9e845bf4d01b15882e06338098b, user_id=9c401cde27794c009a6e13250daf83dc, role_id=5d4ee63a3add47fc9a6b8b16d39e0a1f)` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:48:48.614042 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c702820-8c01-4ae3-854f-0cc64b5131be admin admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:48:48.625212 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 992/1985] 10.222.0.13 () {60 vars in 1483 bytes} [Wed Aug 31 09:48:48 2022] PUT /identity/v3/projects/fc3b8644859b4e73878b2156defc9563/users/c81e3fd1717c45ceb2082c75bd8a59a3/roles/5d4ee63a3add47fc9a6b8b16d39e0a1f => generated 0 bytes in 297 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:48:48.627332 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0935b764-5e41-4881-899a-58c0e9da6a8d admin admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:48:48.641870 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 994/1986] 10.222.0.13 () {60 vars in 1483 bytes} [Wed Aug 31 09:48:48 2022] PUT /identity/v3/projects/6254c9e845bf4d01b15882e06338098b/users/9c401cde27794c009a6e13250daf83dc/roles/5d4ee63a3add47fc9a6b8b16d39e0a1f => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 09:48:48.656390 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee9684e-b60b-4764-b71a-4d549f72aebe None None] REQUEST_METHOD: `POST` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:48.656390 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee9684e-b60b-4764-b71a-4d549f72aebe None None] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:48.656390 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee9684e-b60b-4764-b71a-4d549f72aebe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:48.752837 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.auth.core [None req-bee9684e-b60b-4764-b71a-4d549f72aebe None None] MFA Rules not processed for user `9c401cde27794c009a6e13250daf83dc`. Rule list: `[]` (Enabled: `True`). {{(pid=88114) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:48:48.757572 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.fernet_utils [None req-bee9684e-b60b-4764-b71a-4d549f72aebe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88114) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:48:48.816984 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0935b764-5e41-4881-899a-58c0e9da6a8d admin admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:48:48.986507 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e0ec7ad-0700-452a-bd90-71a91fa8d2dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:48:48.987442 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0ec7ad-0700-452a-bd90-71a91fa8d2dc admin admin] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:48.987883 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0ec7ad-0700-452a-bd90-71a91fa8d2dc admin admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:48.988961 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0ec7ad-0700-452a-bd90-71a91fa8d2dc admin admin] PATH_INFO: `/v3/roles` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:48.989091 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 995/1987] 10.222.0.13 () {58 vars in 958 bytes} [Wed Aug 31 09:48:48 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 339 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:48:48.990904 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e0ec7ad-0700-452a-bd90-71a91fa8d2dc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:48:48.993807 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e0ec7ad-0700-452a-bd90-71a91fa8d2dc admin admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:48:49.041660 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35afbdf5-a0f3-47ed-b6b5-fbdfa80af7aa None None] REQUEST_METHOD: `POST` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:49.042727 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35afbdf5-a0f3-47ed-b6b5-fbdfa80af7aa None None] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:49.043166 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35afbdf5-a0f3-47ed-b6b5-fbdfa80af7aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:49.046142 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 993/1988] 10.222.0.13 () {58 vars in 1122 bytes} [Wed Aug 31 09:48:48 2022] GET /identity/v3/roles => generated 1420 bytes in 420 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 09:48:49.052488 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e0ec7ad-0700-452a-bd90-71a91fa8d2dc admin admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:48:49.075105 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e0ec7ad-0700-452a-bd90-71a91fa8d2dc admin admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:48:49.086864 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0925d1e0-41a8-46a9-b6c2-4c61ed37d832 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:48:49.088085 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0925d1e0-41a8-46a9-b6c2-4c61ed37d832 admin admin] REQUEST_METHOD: `PUT` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:49.088481 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0925d1e0-41a8-46a9-b6c2-4c61ed37d832 admin admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:49.088956 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0925d1e0-41a8-46a9-b6c2-4c61ed37d832 admin admin] PATH_INFO: `/v3/projects/fc3b8644859b4e73878b2156defc9563/users/c81e3fd1717c45ceb2082c75bd8a59a3/roles/5d4ee63a3add47fc9a6b8b16d39e0a1f` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:49.093042 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0925d1e0-41a8-46a9-b6c2-4c61ed37d832 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fc3b8644859b4e73878b2156defc9563, user_id=c81e3fd1717c45ceb2082c75bd8a59a3, role_id=5d4ee63a3add47fc9a6b8b16d39e0a1f)` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:48:49.101137 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.auth.core [None req-35afbdf5-a0f3-47ed-b6b5-fbdfa80af7aa None None] MFA Rules not processed for user `c475984de9e84ff18fb96c588a07c80b`. Rule list: `[]` (Enabled: `True`). {{(pid=88114) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:48:49.101585 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0925d1e0-41a8-46a9-b6c2-4c61ed37d832 admin admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:48:49.102556 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.fernet_utils [None req-35afbdf5-a0f3-47ed-b6b5-fbdfa80af7aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88114) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:48:49.121986 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 994/1989] 10.222.0.13 () {60 vars in 1483 bytes} [Wed Aug 31 09:48:49 2022] PUT /identity/v3/projects/fc3b8644859b4e73878b2156defc9563/users/c81e3fd1717c45ceb2082c75bd8a59a3/roles/5d4ee63a3add47fc9a6b8b16d39e0a1f => generated 0 bytes in 70 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:48:49.137497 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0925d1e0-41a8-46a9-b6c2-4c61ed37d832 admin admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:48:49.272555 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 996/1990] 10.222.0.13 () {58 vars in 958 bytes} [Wed Aug 31 09:48:49 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 232 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 09:48:49.278245 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f617e62-8d4c-4baf-8e1d-5607eb1c6f0c None None] REQUEST_METHOD: `POST` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:49.278245 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f617e62-8d4c-4baf-8e1d-5607eb1c6f0c None None] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:49.291787 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f617e62-8d4c-4baf-8e1d-5607eb1c6f0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:49.306080 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0925d1e0-41a8-46a9-b6c2-4c61ed37d832 admin admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:48:49.321076 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49cba5bf-d8f5-4c67-947e-b4f2b0634cbb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d52b69c8e94441dacfcff944326f169', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '49548eb44d62415582af820ab66decee', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:48:49.322899 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49cba5bf-d8f5-4c67-947e-b4f2b0634cbb service cinder] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:49.323767 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49cba5bf-d8f5-4c67-947e-b4f2b0634cbb service cinder] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:49.324478 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49cba5bf-d8f5-4c67-947e-b4f2b0634cbb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:49.354199 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:48:49.464229 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.auth.core [None req-6f617e62-8d4c-4baf-8e1d-5607eb1c6f0c None None] MFA Rules not processed for user `c81e3fd1717c45ceb2082c75bd8a59a3`. Rule list: `[]` (Enabled: `True`). {{(pid=88114) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:48:49.468178 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.fernet_utils [None req-6f617e62-8d4c-4baf-8e1d-5607eb1c6f0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88114) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:48:49.523089 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49cba5bf-d8f5-4c67-947e-b4f2b0634cbb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:48:49.525639 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49cba5bf-d8f5-4c67-947e-b4f2b0634cbb service cinder] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:48:49.548117 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 995/1991] 10.222.0.13 () {60 vars in 1365 bytes} [Wed Aug 31 09:48:49 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 426 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:48:49.551068 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49cba5bf-d8f5-4c67-947e-b4f2b0634cbb service cinder] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:48:49.597963 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:48:49.665449 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 997/1992] 10.222.0.13 () {58 vars in 958 bytes} [Wed Aug 31 09:48:49 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 392 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:48:49.682552 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f617e62-8d4c-4baf-8e1d-5607eb1c6f0c None None] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:48:49.686522 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.fernet_utils [None req-6f617e62-8d4c-4baf-8e1d-5607eb1c6f0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88114) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:48:49.721459 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f617e62-8d4c-4baf-8e1d-5607eb1c6f0c None None] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:48:49.723596 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49cba5bf-d8f5-4c67-947e-b4f2b0634cbb service cinder] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:48:49.745869 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d0289a-4a7a-4ab3-997d-dd7d2767e9dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:48:49.745869 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d0289a-4a7a-4ab3-997d-dd7d2767e9dc admin admin] REQUEST_METHOD: `DELETE` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:49.746181 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f53cf265-1f9f-4688-93f3-885187f43379 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d52b69c8e94441dacfcff944326f169', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '49548eb44d62415582af820ab66decee', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:48:49.746355 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d0289a-4a7a-4ab3-997d-dd7d2767e9dc admin admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:49.746355 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d0289a-4a7a-4ab3-997d-dd7d2767e9dc admin admin] PATH_INFO: `/v3/users/5e935eba26644bffb17aa00255ef99ea` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:49.746633 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53cf265-1f9f-4688-93f3-885187f43379 service cinder] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:49.746633 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53cf265-1f9f-4688-93f3-885187f43379 service cinder] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:49.746633 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53cf265-1f9f-4688-93f3-885187f43379 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:49.746986 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95d0289a-4a7a-4ab3-997d-dd7d2767e9dc admin admin] RBAC: Authorizing `identity:delete_user(user_id=5e935eba26644bffb17aa00255ef99ea)` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:48:49.746986 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95d0289a-4a7a-4ab3-997d-dd7d2767e9dc admin admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:48:49.795726 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f53cf265-1f9f-4688-93f3-885187f43379 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:48:49.798518 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f53cf265-1f9f-4688-93f3-885187f43379 service cinder] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:48:49.827191 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 998/1993] 10.222.0.13 () {60 vars in 1365 bytes} [Wed Aug 31 09:48:49 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 09:48:49.939216 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.notifications [None req-95d0289a-4a7a-4ab3-997d-dd7d2767e9dc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5e935eba26644bffb17aa00255ef99ea'} {{(pid=88113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:48:49.966826 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.notifications [None req-95d0289a-4a7a-4ab3-997d-dd7d2767e9dc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5e935eba26644bffb17aa00255ef99ea'} {{(pid=88113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:48:50.016628 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.notifications [None req-95d0289a-4a7a-4ab3-997d-dd7d2767e9dc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5e935eba26644bffb17aa00255ef99ea'} {{(pid=88113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:48:50.018962 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.notifications [None req-95d0289a-4a7a-4ab3-997d-dd7d2767e9dc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5e935eba26644bffb17aa00255ef99ea'} {{(pid=88113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:48:50.042436 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.notifications [None req-95d0289a-4a7a-4ab3-997d-dd7d2767e9dc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5e935eba26644bffb17aa00255ef99ea'} {{(pid=88113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:48:50.071860 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 996/1994] 10.222.0.13 () {58 vars in 1224 bytes} [Wed Aug 31 09:48:49 2022] DELETE /identity/v3/users/5e935eba26644bffb17aa00255ef99ea => generated 0 bytes in 521 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:48:50.076270 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3262edc2-73f1-4690-b8c9-d69869bcca96 None None] REQUEST_METHOD: `POST` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:50.077260 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3262edc2-73f1-4690-b8c9-d69869bcca96 None None] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:50.078026 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3262edc2-73f1-4690-b8c9-d69869bcca96 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:50.125804 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.auth.core [None req-3262edc2-73f1-4690-b8c9-d69869bcca96 None None] MFA Rules not processed for user `fd836cf590a145df9acf19cf3ae2ac56`. Rule list: `[]` (Enabled: `True`). {{(pid=88114) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:48:50.130138 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.fernet_utils [None req-3262edc2-73f1-4690-b8c9-d69869bcca96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88114) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:48:50.350883 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 999/1995] 10.222.0.13 () {58 vars in 958 bytes} [Wed Aug 31 09:48:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 276 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:48:50.364662 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d0289a-4a7a-4ab3-997d-dd7d2767e9dc admin admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:48:50.368230 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.fernet_utils [None req-95d0289a-4a7a-4ab3-997d-dd7d2767e9dc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:48:50.516781 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:48:50.640682 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d0289a-4a7a-4ab3-997d-dd7d2767e9dc admin admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:48:50.653546 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05957fb8-f241-49d9-b5d0-ea29affba9dc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9bdaacd597442c2bcde2c555f2cc140', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '49548eb44d62415582af820ab66decee', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:48:50.654740 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05957fb8-f241-49d9-b5d0-ea29affba9dc service neutron] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:50.655253 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05957fb8-f241-49d9-b5d0-ea29affba9dc service neutron] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:50.655745 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05957fb8-f241-49d9-b5d0-ea29affba9dc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:50.727609 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05957fb8-f241-49d9-b5d0-ea29affba9dc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:48:50.730605 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05957fb8-f241-49d9-b5d0-ea29affba9dc service neutron] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:48:50.778766 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 997/1996] 10.222.0.13 () {60 vars in 1365 bytes} [Wed Aug 31 09:48:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 415 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:48:51.548221 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3262edc2-73f1-4690-b8c9-d69869bcca96 None None] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:48:51.556270 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.fernet_utils [None req-3262edc2-73f1-4690-b8c9-d69869bcca96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88114) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:48:51.616353 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3262edc2-73f1-4690-b8c9-d69869bcca96 None None] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:48:51.636034 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed2fe248-daa5-4d93-8f83-cc1080089da4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:48:51.637137 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2fe248-daa5-4d93-8f83-cc1080089da4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:51.637860 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2fe248-daa5-4d93-8f83-cc1080089da4 admin admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:51.638629 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2fe248-daa5-4d93-8f83-cc1080089da4 admin admin] PATH_INFO: `/v3/projects/8437124192c34160bc1fba7bdd86e90c` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:51.643394 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed2fe248-daa5-4d93-8f83-cc1080089da4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8437124192c34160bc1fba7bdd86e90c)` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:48:51.651742 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed2fe248-daa5-4d93-8f83-cc1080089da4 admin admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:48:51.899459 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.notifications [None req-ed2fe248-daa5-4d93-8f83-cc1080089da4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8437124192c34160bc1fba7bdd86e90c', 'request_id': 'req-ed2fe248-daa5-4d93-8f83-cc1080089da4'} {{(pid=88114) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:48:51.943737 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.notifications [None req-ed2fe248-daa5-4d93-8f83-cc1080089da4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8437124192c34160bc1fba7bdd86e90c', 'request_id': 'req-ed2fe248-daa5-4d93-8f83-cc1080089da4'} {{(pid=88114) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:48:51.963533 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.notifications [None req-ed2fe248-daa5-4d93-8f83-cc1080089da4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8437124192c34160bc1fba7bdd86e90c', 'request_id': 'req-ed2fe248-daa5-4d93-8f83-cc1080089da4'} {{(pid=88114) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:48:51.997207 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.notifications [None req-ed2fe248-daa5-4d93-8f83-cc1080089da4 admin admin] The token cache is being invalidate because project 8437124192c34160bc1fba7bdd86e90c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88114) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 09:48:51.997533 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.notifications [None req-ed2fe248-daa5-4d93-8f83-cc1080089da4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88114) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:48:52.000847 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.notifications [None req-ed2fe248-daa5-4d93-8f83-cc1080089da4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88114) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:48:52.007212 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 1000/1997] 10.222.0.13 () {58 vars in 1233 bytes} [Wed Aug 31 09:48:51 2022] DELETE /identity/v3/projects/8437124192c34160bc1fba7bdd86e90c => generated 0 bytes in 460 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:48:52.017146 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05957fb8-f241-49d9-b5d0-ea29affba9dc service neutron] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:48:52.067198 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05957fb8-f241-49d9-b5d0-ea29affba9dc service neutron] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:48:52.169400 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eddf4582-4449-4754-a02d-f13904b449c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:48:52.170234 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eddf4582-4449-4754-a02d-f13904b449c4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:52.170643 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eddf4582-4449-4754-a02d-f13904b449c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:52.171000 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eddf4582-4449-4754-a02d-f13904b449c4 admin admin] PATH_INFO: `/v3/users/e5d116b0755440da976b1f782f7c504b` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:52.173201 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eddf4582-4449-4754-a02d-f13904b449c4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e5d116b0755440da976b1f782f7c504b)` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:48:52.176258 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eddf4582-4449-4754-a02d-f13904b449c4 admin admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:48:52.286232 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.notifications [None req-eddf4582-4449-4754-a02d-f13904b449c4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e5d116b0755440da976b1f782f7c504b'} {{(pid=88113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:48:52.330864 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed2fe248-daa5-4d93-8f83-cc1080089da4 admin admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:48:52.331106 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.notifications [None req-eddf4582-4449-4754-a02d-f13904b449c4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e5d116b0755440da976b1f782f7c504b'} {{(pid=88113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:48:52.333771 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.fernet_utils [None req-ed2fe248-daa5-4d93-8f83-cc1080089da4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88114) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:48:52.399087 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.notifications [None req-eddf4582-4449-4754-a02d-f13904b449c4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e5d116b0755440da976b1f782f7c504b'} {{(pid=88113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:48:52.404809 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.notifications [None req-eddf4582-4449-4754-a02d-f13904b449c4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e5d116b0755440da976b1f782f7c504b'} {{(pid=88113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:48:52.439210 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed2fe248-daa5-4d93-8f83-cc1080089da4 admin admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:48:52.443012 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.notifications [None req-eddf4582-4449-4754-a02d-f13904b449c4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e5d116b0755440da976b1f782f7c504b'} {{(pid=88113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:48:52.453553 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 998/1998] 10.222.0.13 () {58 vars in 1224 bytes} [Wed Aug 31 09:48:52 2022] DELETE /identity/v3/users/e5d116b0755440da976b1f782f7c504b => generated 0 bytes in 439 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:48:52.454912 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-744c12e3-a231-4c49-9c0f-3b23abfd7669 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d52b69c8e94441dacfcff944326f169', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '49548eb44d62415582af820ab66decee', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:48:52.456552 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744c12e3-a231-4c49-9c0f-3b23abfd7669 service cinder] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:52.458031 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744c12e3-a231-4c49-9c0f-3b23abfd7669 service cinder] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:52.459143 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744c12e3-a231-4c49-9c0f-3b23abfd7669 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:52.461894 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.fernet_utils [None req-744c12e3-a231-4c49-9c0f-3b23abfd7669 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88114) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:48:52.478396 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:48:52.622367 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-744c12e3-a231-4c49-9c0f-3b23abfd7669 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:48:52.625051 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-744c12e3-a231-4c49-9c0f-3b23abfd7669 service cinder] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:48:52.645032 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 1001/1999] 10.222.0.13 () {60 vars in 1365 bytes} [Wed Aug 31 09:48:52 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:48:53.473143 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eddf4582-4449-4754-a02d-f13904b449c4 admin admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:48:53.475827 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.fernet_utils [None req-eddf4582-4449-4754-a02d-f13904b449c4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:48:53.657257 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:48:53.728372 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eddf4582-4449-4754-a02d-f13904b449c4 admin admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:48:53.738251 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33bff3ba-8d00-4fd6-b27c-74d1331f92df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:48:53.739349 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33bff3ba-8d00-4fd6-b27c-74d1331f92df admin admin] REQUEST_METHOD: `DELETE` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:53.739828 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33bff3ba-8d00-4fd6-b27c-74d1331f92df admin admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:53.740435 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33bff3ba-8d00-4fd6-b27c-74d1331f92df admin admin] PATH_INFO: `/v3/projects/8f9b66eb8c024cbf8ec51f00ffed6787` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:53.743171 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33bff3ba-8d00-4fd6-b27c-74d1331f92df admin admin] RBAC: Authorizing `identity:delete_project(project_id=8f9b66eb8c024cbf8ec51f00ffed6787)` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:48:53.746943 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33bff3ba-8d00-4fd6-b27c-74d1331f92df admin admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:48:53.842892 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.notifications [None req-33bff3ba-8d00-4fd6-b27c-74d1331f92df admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8f9b66eb8c024cbf8ec51f00ffed6787', 'request_id': 'req-33bff3ba-8d00-4fd6-b27c-74d1331f92df'} {{(pid=88113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:48:53.880883 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.notifications [None req-33bff3ba-8d00-4fd6-b27c-74d1331f92df admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8f9b66eb8c024cbf8ec51f00ffed6787', 'request_id': 'req-33bff3ba-8d00-4fd6-b27c-74d1331f92df'} {{(pid=88113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:48:53.903124 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.notifications [None req-33bff3ba-8d00-4fd6-b27c-74d1331f92df admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8f9b66eb8c024cbf8ec51f00ffed6787', 'request_id': 'req-33bff3ba-8d00-4fd6-b27c-74d1331f92df'} {{(pid=88113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:48:53.991752 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.notifications [None req-33bff3ba-8d00-4fd6-b27c-74d1331f92df admin admin] The token cache is being invalidate because project 8f9b66eb8c024cbf8ec51f00ffed6787 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88113) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 09:48:53.991752 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.notifications [None req-33bff3ba-8d00-4fd6-b27c-74d1331f92df admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:48:53.993782 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.notifications [None req-33bff3ba-8d00-4fd6-b27c-74d1331f92df admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:48:53.996821 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 999/2000] 10.222.0.13 () {58 vars in 1233 bytes} [Wed Aug 31 09:48:53 2022] DELETE /identity/v3/projects/8f9b66eb8c024cbf8ec51f00ffed6787 => generated 0 bytes in 525 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:48:54.004779 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df9cde8-8be5-4ea1-9b4c-0d0f4a6388c7 None None] REQUEST_METHOD: `POST` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:54.005613 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df9cde8-8be5-4ea1-9b4c-0d0f4a6388c7 None None] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:54.005790 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df9cde8-8be5-4ea1-9b4c-0d0f4a6388c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:54.061231 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.auth.core [None req-2df9cde8-8be5-4ea1-9b4c-0d0f4a6388c7 None None] MFA Rules not processed for user `fd836cf590a145df9acf19cf3ae2ac56`. Rule list: `[]` (Enabled: `True`). {{(pid=88114) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:48:54.064481 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.fernet_utils [None req-2df9cde8-8be5-4ea1-9b4c-0d0f4a6388c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88114) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:48:54.241341 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 1002/2001] 10.222.0.13 () {58 vars in 958 bytes} [Wed Aug 31 09:48:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 238 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:48:54.272156 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3597fe3b-a9c9-4b7f-a94e-af86ac545d91 None None] REQUEST_METHOD: `POST` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:54.272756 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3597fe3b-a9c9-4b7f-a94e-af86ac545d91 None None] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:54.273930 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3597fe3b-a9c9-4b7f-a94e-af86ac545d91 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:54.321204 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.auth.core [None req-3597fe3b-a9c9-4b7f-a94e-af86ac545d91 None None] MFA Rules not processed for user `fd836cf590a145df9acf19cf3ae2ac56`. Rule list: `[]` (Enabled: `True`). {{(pid=88113) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:48:54.323375 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.fernet_utils [None req-3597fe3b-a9c9-4b7f-a94e-af86ac545d91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:48:54.349878 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 1000/2002] 10.222.0.13 () {58 vars in 958 bytes} [Wed Aug 31 09:48:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:48:54.356207 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df9cde8-8be5-4ea1-9b4c-0d0f4a6388c7 None None] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:48:54.384878 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df9cde8-8be5-4ea1-9b4c-0d0f4a6388c7 None None] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:48:54.402627 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d13f509-0537-4afb-8c52-b0702ee8c963 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:48:54.405009 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d13f509-0537-4afb-8c52-b0702ee8c963 admin admin] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:54.407667 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d13f509-0537-4afb-8c52-b0702ee8c963 admin admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:54.408290 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d13f509-0537-4afb-8c52-b0702ee8c963 admin admin] PATH_INFO: `/v3/domains` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:54.410412 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d13f509-0537-4afb-8c52-b0702ee8c963 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88114) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:48:54.411588 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d13f509-0537-4afb-8c52-b0702ee8c963 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:48:54.414666 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d13f509-0537-4afb-8c52-b0702ee8c963 admin admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:48:54.450455 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 1003/2003] 10.222.0.13 () {58 vars in 1153 bytes} [Wed Aug 31 09:48:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 09:48:54.459222 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3597fe3b-a9c9-4b7f-a94e-af86ac545d91 None None] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:48:54.504182 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3597fe3b-a9c9-4b7f-a94e-af86ac545d91 None None] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:48:54.538414 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6521a11a-f789-4bb7-b37f-8a301155875f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:48:54.539577 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6521a11a-f789-4bb7-b37f-8a301155875f admin admin] REQUEST_METHOD: `POST` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:54.540067 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6521a11a-f789-4bb7-b37f-8a301155875f admin admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:54.540545 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6521a11a-f789-4bb7-b37f-8a301155875f admin admin] PATH_INFO: `/v3/projects` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:54.541815 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6521a11a-f789-4bb7-b37f-8a301155875f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:48:54.545831 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6521a11a-f789-4bb7-b37f-8a301155875f admin admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:48:54.709487 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 1001/2004] 10.222.0.13 () {60 vars in 1153 bytes} [Wed Aug 31 09:48:54 2022] POST /identity/v3/projects => generated 401 bytes in 250 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 09:48:54.718186 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d13f509-0537-4afb-8c52-b0702ee8c963 admin admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:48:54.745815 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d13f509-0537-4afb-8c52-b0702ee8c963 admin admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:48:54.921943 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-875e9ae9-0088-4925-ab1a-e5d2ec14225c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:48:54.922573 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875e9ae9-0088-4925-ab1a-e5d2ec14225c admin admin] REQUEST_METHOD: `POST` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:54.922850 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875e9ae9-0088-4925-ab1a-e5d2ec14225c admin admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:54.923123 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875e9ae9-0088-4925-ab1a-e5d2ec14225c admin admin] PATH_INFO: `/v3/users` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:54.923774 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-875e9ae9-0088-4925-ab1a-e5d2ec14225c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:48:54.926946 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-875e9ae9-0088-4925-ab1a-e5d2ec14225c admin admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:48:54.965554 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 1004/2005] 10.222.0.13 () {60 vars in 1144 bytes} [Wed Aug 31 09:48:54 2022] POST /identity/v3/users => generated 301 bytes in 248 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 09:48:54.974002 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6521a11a-f789-4bb7-b37f-8a301155875f admin admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:48:55.037701 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6521a11a-f789-4bb7-b37f-8a301155875f admin admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:48:55.058646 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-875e9ae9-0088-4925-ab1a-e5d2ec14225c admin admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:48:55.061814 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.fernet_utils [None req-875e9ae9-0088-4925-ab1a-e5d2ec14225c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88114) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:48:55.076031 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f842481d-f5fb-426c-8c9d-cc41ebb53264 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:48:55.077309 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f842481d-f5fb-426c-8c9d-cc41ebb53264 admin admin] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:55.078072 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f842481d-f5fb-426c-8c9d-cc41ebb53264 admin admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:55.078556 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f842481d-f5fb-426c-8c9d-cc41ebb53264 admin admin] PATH_INFO: `/v3/roles` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:55.079988 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f842481d-f5fb-426c-8c9d-cc41ebb53264 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:48:55.093572 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f842481d-f5fb-426c-8c9d-cc41ebb53264 admin admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:48:55.139526 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 1002/2006] 10.222.0.13 () {58 vars in 1122 bytes} [Wed Aug 31 09:48:54 2022] GET /identity/v3/roles => generated 1420 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 09:48:55.141755 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f842481d-f5fb-426c-8c9d-cc41ebb53264 admin admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:48:55.161174 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f842481d-f5fb-426c-8c9d-cc41ebb53264 admin admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:48:55.170809 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e9fed0f-54ee-408b-b799-7c5e02593cbf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:48:55.171868 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9fed0f-54ee-408b-b799-7c5e02593cbf admin admin] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:55.172357 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9fed0f-54ee-408b-b799-7c5e02593cbf admin admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:55.173016 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9fed0f-54ee-408b-b799-7c5e02593cbf admin admin] PATH_INFO: `/v3/roles` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:55.174547 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e9fed0f-54ee-408b-b799-7c5e02593cbf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:48:55.176987 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:48:55.179106 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e9fed0f-54ee-408b-b799-7c5e02593cbf admin admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:48:55.204371 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 1003/2007] 10.222.0.13 () {58 vars in 1122 bytes} [Wed Aug 31 09:48:55 2022] GET /identity/v3/roles => generated 1420 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 09:48:55.210464 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e9fed0f-54ee-408b-b799-7c5e02593cbf admin admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:48:55.226546 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e9fed0f-54ee-408b-b799-7c5e02593cbf admin admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:48:55.232322 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd5edd7-4763-43a6-a4f5-e0311e582e22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:48:55.233567 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd5edd7-4763-43a6-a4f5-e0311e582e22 admin admin] REQUEST_METHOD: `PUT` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:55.233876 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd5edd7-4763-43a6-a4f5-e0311e582e22 admin admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:55.234245 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd5edd7-4763-43a6-a4f5-e0311e582e22 admin admin] PATH_INFO: `/v3/projects/6ddbec14e9e54228acbf4d80ea432639/users/42c211044366426e92b558a8332194da/roles/6c99f5d9f1f243eaa134d8540f21095d` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:55.243620 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-875e9ae9-0088-4925-ab1a-e5d2ec14225c admin admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:48:55.252583 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86bd02d0-d7cc-4d80-baa9-8149553c9182 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d52b69c8e94441dacfcff944326f169', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '49548eb44d62415582af820ab66decee', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:48:55.254898 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fd5edd7-4763-43a6-a4f5-e0311e582e22 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6ddbec14e9e54228acbf4d80ea432639, user_id=42c211044366426e92b558a8332194da, role_id=6c99f5d9f1f243eaa134d8540f21095d)` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:48:55.255271 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86bd02d0-d7cc-4d80-baa9-8149553c9182 service cinder] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:55.256048 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86bd02d0-d7cc-4d80-baa9-8149553c9182 service cinder] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:55.256777 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86bd02d0-d7cc-4d80-baa9-8149553c9182 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:55.257872 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fd5edd7-4763-43a6-a4f5-e0311e582e22 admin admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:48:55.260764 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.fernet_utils [None req-86bd02d0-d7cc-4d80-baa9-8149553c9182 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88114) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:48:55.277660 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 1004/2008] 10.222.0.13 () {60 vars in 1483 bytes} [Wed Aug 31 09:48:55 2022] PUT /identity/v3/projects/6ddbec14e9e54228acbf4d80ea432639/users/42c211044366426e92b558a8332194da/roles/6c99f5d9f1f243eaa134d8540f21095d => generated 0 bytes in 69 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 09:48:55.310862 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e49b45-ce3f-40d1-a746-c22bf74fc86c None None] REQUEST_METHOD: `POST` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:55.311236 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e49b45-ce3f-40d1-a746-c22bf74fc86c None None] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:55.311589 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e49b45-ce3f-40d1-a746-c22bf74fc86c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:55.402294 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.auth.core [None req-79e49b45-ce3f-40d1-a746-c22bf74fc86c None None] MFA Rules not processed for user `42c211044366426e92b558a8332194da`. Rule list: `[]` (Enabled: `True`). {{(pid=88113) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:48:55.404174 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.fernet_utils [None req-79e49b45-ce3f-40d1-a746-c22bf74fc86c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:48:55.472242 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86bd02d0-d7cc-4d80-baa9-8149553c9182 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:48:55.477550 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86bd02d0-d7cc-4d80-baa9-8149553c9182 service cinder] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:48:55.496004 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 1005/2009] 10.222.0.13 () {60 vars in 1365 bytes} [Wed Aug 31 09:48:55 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 440 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:48:55.505957 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:48:55.532894 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 1005/2010] 10.222.0.13 () {58 vars in 958 bytes} [Wed Aug 31 09:48:55 2022] POST /identity/v3/auth/tokens => generated 3278 bytes in 220 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:48:55.535580 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86bd02d0-d7cc-4d80-baa9-8149553c9182 service cinder] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:48:55.548498 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86bd02d0-d7cc-4d80-baa9-8149553c9182 service cinder] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:48:55.648975 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab8183d-d864-4657-9b91-7bcbd0b3f1b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:48:55.651153 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab8183d-d864-4657-9b91-7bcbd0b3f1b4 admin admin] REQUEST_METHOD: `POST` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:55.652169 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab8183d-d864-4657-9b91-7bcbd0b3f1b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:55.652674 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab8183d-d864-4657-9b91-7bcbd0b3f1b4 admin admin] PATH_INFO: `/v3/projects` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:55.653762 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ab8183d-d864-4657-9b91-7bcbd0b3f1b4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:48:55.657430 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ab8183d-d864-4657-9b91-7bcbd0b3f1b4 admin admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:48:55.709533 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 1006/2011] 10.222.0.13 () {60 vars in 1153 bytes} [Wed Aug 31 09:48:55 2022] POST /identity/v3/projects => generated 399 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 09:48:55.715327 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e49b45-ce3f-40d1-a746-c22bf74fc86c None None] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:48:55.733456 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e49b45-ce3f-40d1-a746-c22bf74fc86c None None] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:48:55.846941 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-672001b5-b28e-41ca-a19a-a603de5f4ba6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:48:55.847835 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672001b5-b28e-41ca-a19a-a603de5f4ba6 admin admin] REQUEST_METHOD: `POST` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:55.848320 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672001b5-b28e-41ca-a19a-a603de5f4ba6 admin admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:55.848907 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672001b5-b28e-41ca-a19a-a603de5f4ba6 admin admin] PATH_INFO: `/v3/users` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:55.849609 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-672001b5-b28e-41ca-a19a-a603de5f4ba6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:48:55.852123 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-672001b5-b28e-41ca-a19a-a603de5f4ba6 admin admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:48:55.887969 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 1006/2012] 10.222.0.13 () {60 vars in 1144 bytes} [Wed Aug 31 09:48:55 2022] POST /identity/v3/users => generated 306 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 09:48:55.899870 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab8183d-d864-4657-9b91-7bcbd0b3f1b4 admin admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:48:55.919790 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab8183d-d864-4657-9b91-7bcbd0b3f1b4 admin admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:48:55.929772 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c3df83-8631-4b87-8ff1-fff63c55667b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:48:55.930890 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c3df83-8631-4b87-8ff1-fff63c55667b admin admin] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:55.931474 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c3df83-8631-4b87-8ff1-fff63c55667b admin admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:55.931844 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c3df83-8631-4b87-8ff1-fff63c55667b admin admin] PATH_INFO: `/v3/roles` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:55.933016 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47c3df83-8631-4b87-8ff1-fff63c55667b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:48:55.936172 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47c3df83-8631-4b87-8ff1-fff63c55667b admin admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:48:55.977406 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 1007/2013] 10.222.0.13 () {58 vars in 1122 bytes} [Wed Aug 31 09:48:55 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 09:48:55.983053 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-672001b5-b28e-41ca-a19a-a603de5f4ba6 admin admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:48:56.004810 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-672001b5-b28e-41ca-a19a-a603de5f4ba6 admin admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:48:56.014056 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d9ff4e0-1946-4b92-81da-e5b78c67e20c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:48:56.015400 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9ff4e0-1946-4b92-81da-e5b78c67e20c admin admin] REQUEST_METHOD: `PUT` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:56.016015 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9ff4e0-1946-4b92-81da-e5b78c67e20c admin admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:56.016742 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9ff4e0-1946-4b92-81da-e5b78c67e20c admin admin] PATH_INFO: `/v3/projects/1e294af9d7204453adcd286755f57962/users/446307197bda44649c566c16a013de27/roles/5d4ee63a3add47fc9a6b8b16d39e0a1f` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:56.061416 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d9ff4e0-1946-4b92-81da-e5b78c67e20c admin admin] RBAC: Authorizing `identity:create_grant(project_id=1e294af9d7204453adcd286755f57962, user_id=446307197bda44649c566c16a013de27, role_id=5d4ee63a3add47fc9a6b8b16d39e0a1f)` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:48:56.061416 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d9ff4e0-1946-4b92-81da-e5b78c67e20c admin admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:48:56.092215 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 1007/2014] 10.222.0.13 () {60 vars in 1483 bytes} [Wed Aug 31 09:48:55 2022] PUT /identity/v3/projects/1e294af9d7204453adcd286755f57962/users/446307197bda44649c566c16a013de27/roles/5d4ee63a3add47fc9a6b8b16d39e0a1f => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:48:56.101746 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c3df83-8631-4b87-8ff1-fff63c55667b admin admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:48:56.115533 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c3df83-8631-4b87-8ff1-fff63c55667b admin admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:48:56.189438 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:48:56.232197 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0353d0c0-df6b-43d3-ab92-fe4ec7d20d06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:48:56.233717 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0353d0c0-df6b-43d3-ab92-fe4ec7d20d06 admin admin] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:56.235095 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0353d0c0-df6b-43d3-ab92-fe4ec7d20d06 admin admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:56.235734 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0353d0c0-df6b-43d3-ab92-fe4ec7d20d06 admin admin] PATH_INFO: `/v3/roles` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:56.236661 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0353d0c0-df6b-43d3-ab92-fe4ec7d20d06 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:48:56.239726 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0353d0c0-df6b-43d3-ab92-fe4ec7d20d06 admin admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:48:56.256931 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 1008/2015] 10.222.0.13 () {58 vars in 1122 bytes} [Wed Aug 31 09:48:56 2022] GET /identity/v3/roles => generated 1420 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 09:48:56.262262 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d9ff4e0-1946-4b92-81da-e5b78c67e20c admin admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:48:56.280097 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d9ff4e0-1946-4b92-81da-e5b78c67e20c admin admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:48:56.290346 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba145d49-f7de-464a-88c8-023709e2cda5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:48:56.291120 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba145d49-f7de-464a-88c8-023709e2cda5 admin admin] REQUEST_METHOD: `PUT` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:56.291394 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba145d49-f7de-464a-88c8-023709e2cda5 admin admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:56.292324 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba145d49-f7de-464a-88c8-023709e2cda5 admin admin] PATH_INFO: `/v3/projects/1e294af9d7204453adcd286755f57962/users/446307197bda44649c566c16a013de27/roles/5d4ee63a3add47fc9a6b8b16d39e0a1f` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:56.295165 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba145d49-f7de-464a-88c8-023709e2cda5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1e294af9d7204453adcd286755f57962, user_id=446307197bda44649c566c16a013de27, role_id=5d4ee63a3add47fc9a6b8b16d39e0a1f)` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:48:56.298779 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba145d49-f7de-464a-88c8-023709e2cda5 admin admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:48:56.318398 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 1008/2016] 10.222.0.13 () {60 vars in 1483 bytes} [Wed Aug 31 09:48:56 2022] PUT /identity/v3/projects/1e294af9d7204453adcd286755f57962/users/446307197bda44649c566c16a013de27/roles/5d4ee63a3add47fc9a6b8b16d39e0a1f => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:48:56.329805 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9d9dc6-2043-435d-91e2-057271dd72f2 None None] REQUEST_METHOD: `POST` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:56.330342 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9d9dc6-2043-435d-91e2-057271dd72f2 None None] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:56.331379 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9d9dc6-2043-435d-91e2-057271dd72f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:56.386349 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.auth.core [None req-fb9d9dc6-2043-435d-91e2-057271dd72f2 None None] MFA Rules not processed for user `446307197bda44649c566c16a013de27`. Rule list: `[]` (Enabled: `True`). {{(pid=88114) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:48:56.391690 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.fernet_utils [None req-fb9d9dc6-2043-435d-91e2-057271dd72f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88114) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:48:56.541559 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 1009/2017] 10.222.0.13 () {58 vars in 958 bytes} [Wed Aug 31 09:48:56 2022] POST /identity/v3/auth/tokens => generated 3343 bytes in 213 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:48:56.554198 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba145d49-f7de-464a-88c8-023709e2cda5 admin admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:48:56.639536 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba145d49-f7de-464a-88c8-023709e2cda5 admin admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:48:56.644623 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec9757ad-d97e-4f30-a240-fce96400b231 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d52b69c8e94441dacfcff944326f169', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '49548eb44d62415582af820ab66decee', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:48:56.645259 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9757ad-d97e-4f30-a240-fce96400b231 service cinder] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:56.645568 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9757ad-d97e-4f30-a240-fce96400b231 service cinder] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:56.645848 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9757ad-d97e-4f30-a240-fce96400b231 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:56.658477 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec9757ad-d97e-4f30-a240-fce96400b231 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:48:56.661165 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec9757ad-d97e-4f30-a240-fce96400b231 service cinder] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:48:56.673064 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 1009/2018] 10.222.0.13 () {60 vars in 1365 bytes} [Wed Aug 31 09:48:56 2022] GET /identity/v3/auth/tokens => generated 3343 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 09:48:56.849105 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb9d9dc6-2043-435d-91e2-057271dd72f2 None None] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:48:56.860781 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb9d9dc6-2043-435d-91e2-057271dd72f2 None None] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:48:56.867321 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1612927e-12a0-4669-b409-c6b8b917e88a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d52b69c8e94441dacfcff944326f169', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '49548eb44d62415582af820ab66decee', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:48:56.867950 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1612927e-12a0-4669-b409-c6b8b917e88a service cinder] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:56.868227 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1612927e-12a0-4669-b409-c6b8b917e88a service cinder] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:56.868501 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1612927e-12a0-4669-b409-c6b8b917e88a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:56.901802 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec9757ad-d97e-4f30-a240-fce96400b231 service cinder] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:48:56.903522 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.fernet_utils [None req-ec9757ad-d97e-4f30-a240-fce96400b231 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:48:57.015747 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1612927e-12a0-4669-b409-c6b8b917e88a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:48:57.021084 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1612927e-12a0-4669-b409-c6b8b917e88a service cinder] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:48:57.040572 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:48:57.064460 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 1010/2019] 10.222.0.13 () {60 vars in 1365 bytes} [Wed Aug 31 09:48:56 2022] GET /identity/v3/auth/tokens => generated 3278 bytes in 216 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:48:57.071998 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec9757ad-d97e-4f30-a240-fce96400b231 service cinder] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:48:57.084705 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-895e58e7-d523-4fe9-81af-1eaecab7a44c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:48:57.088167 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895e58e7-d523-4fe9-81af-1eaecab7a44c admin admin] REQUEST_METHOD: `DELETE` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:57.088495 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895e58e7-d523-4fe9-81af-1eaecab7a44c admin admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:57.088773 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895e58e7-d523-4fe9-81af-1eaecab7a44c admin admin] PATH_INFO: `/v3/users/bf8707b404674f8f999976d2160f92ea` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:57.090583 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-895e58e7-d523-4fe9-81af-1eaecab7a44c admin admin] RBAC: Authorizing `identity:delete_user(user_id=bf8707b404674f8f999976d2160f92ea)` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:48:57.092971 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-895e58e7-d523-4fe9-81af-1eaecab7a44c admin admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:48:57.262096 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.notifications [None req-895e58e7-d523-4fe9-81af-1eaecab7a44c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bf8707b404674f8f999976d2160f92ea'} {{(pid=88113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:48:57.293610 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.notifications [None req-895e58e7-d523-4fe9-81af-1eaecab7a44c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bf8707b404674f8f999976d2160f92ea'} {{(pid=88113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:48:57.334381 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.notifications [None req-895e58e7-d523-4fe9-81af-1eaecab7a44c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bf8707b404674f8f999976d2160f92ea'} {{(pid=88113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:48:57.335231 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.notifications [None req-895e58e7-d523-4fe9-81af-1eaecab7a44c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bf8707b404674f8f999976d2160f92ea'} {{(pid=88113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:48:57.355797 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.notifications [None req-895e58e7-d523-4fe9-81af-1eaecab7a44c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bf8707b404674f8f999976d2160f92ea'} {{(pid=88113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:48:57.370438 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 1010/2020] 10.222.0.13 () {58 vars in 1224 bytes} [Wed Aug 31 09:48:56 2022] DELETE /identity/v3/users/bf8707b404674f8f999976d2160f92ea => generated 0 bytes in 464 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:48:57.371291 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60fb22b8-a1c2-45e2-9dac-ad823f652a54 None None] REQUEST_METHOD: `POST` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:57.372629 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60fb22b8-a1c2-45e2-9dac-ad823f652a54 None None] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:57.373300 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60fb22b8-a1c2-45e2-9dac-ad823f652a54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:57.413629 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.auth.core [None req-60fb22b8-a1c2-45e2-9dac-ad823f652a54 None None] MFA Rules not processed for user `fd836cf590a145df9acf19cf3ae2ac56`. Rule list: `[]` (Enabled: `True`). {{(pid=88114) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:48:57.415954 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.fernet_utils [None req-60fb22b8-a1c2-45e2-9dac-ad823f652a54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88114) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:48:57.513983 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 1011/2021] 10.222.0.13 () {58 vars in 958 bytes} [Wed Aug 31 09:48:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 09:48:57.531079 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-895e58e7-d523-4fe9-81af-1eaecab7a44c admin admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:48:57.534785 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.fernet_utils [None req-895e58e7-d523-4fe9-81af-1eaecab7a44c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:48:57.802051 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-895e58e7-d523-4fe9-81af-1eaecab7a44c admin admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:48:57.811917 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-359a7bcd-b298-46c4-b77e-ac945e41c3a2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9bdaacd597442c2bcde2c555f2cc140', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '49548eb44d62415582af820ab66decee', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:48:57.823943 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359a7bcd-b298-46c4-b77e-ac945e41c3a2 service neutron] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:57.824905 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359a7bcd-b298-46c4-b77e-ac945e41c3a2 service neutron] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:57.825532 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359a7bcd-b298-46c4-b77e-ac945e41c3a2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:57.861048 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-359a7bcd-b298-46c4-b77e-ac945e41c3a2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:48:57.865033 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-359a7bcd-b298-46c4-b77e-ac945e41c3a2 service neutron] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:48:57.899681 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 1011/2022] 10.222.0.13 () {60 vars in 1365 bytes} [Wed Aug 31 09:48:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 370 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:48:57.964171 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60fb22b8-a1c2-45e2-9dac-ad823f652a54 None None] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:48:57.966597 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.fernet_utils [None req-60fb22b8-a1c2-45e2-9dac-ad823f652a54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88114) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:48:57.994364 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60fb22b8-a1c2-45e2-9dac-ad823f652a54 None None] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:48:58.001063 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ef71ac-636b-471e-b69b-e14035460714 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:48:58.002223 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ef71ac-636b-471e-b69b-e14035460714 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:58.002558 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ef71ac-636b-471e-b69b-e14035460714 admin admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:58.003590 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ef71ac-636b-471e-b69b-e14035460714 admin admin] PATH_INFO: `/v3/users/a79193bb8fb04786aeb2acfb9d1a77b6` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:58.005220 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63ef71ac-636b-471e-b69b-e14035460714 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a79193bb8fb04786aeb2acfb9d1a77b6)` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:48:58.007973 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63ef71ac-636b-471e-b69b-e14035460714 admin admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:48:58.015183 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=100 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:48:58.103060 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.notifications [None req-63ef71ac-636b-471e-b69b-e14035460714 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a79193bb8fb04786aeb2acfb9d1a77b6'} {{(pid=88114) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:48:58.125203 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.notifications [None req-63ef71ac-636b-471e-b69b-e14035460714 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a79193bb8fb04786aeb2acfb9d1a77b6'} {{(pid=88114) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:48:58.158356 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.notifications [None req-63ef71ac-636b-471e-b69b-e14035460714 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a79193bb8fb04786aeb2acfb9d1a77b6'} {{(pid=88114) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:48:58.159778 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.notifications [None req-63ef71ac-636b-471e-b69b-e14035460714 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a79193bb8fb04786aeb2acfb9d1a77b6'} {{(pid=88114) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:48:58.225760 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.notifications [None req-63ef71ac-636b-471e-b69b-e14035460714 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a79193bb8fb04786aeb2acfb9d1a77b6'} {{(pid=88114) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:48:58.259536 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 1012/2023] 10.222.0.13 () {58 vars in 1224 bytes} [Wed Aug 31 09:48:57 2022] DELETE /identity/v3/users/a79193bb8fb04786aeb2acfb9d1a77b6 => generated 0 bytes in 296 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:48:58.264796 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77862fb-16a7-4ea0-a370-6e521f75240b None None] REQUEST_METHOD: `POST` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:58.265449 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77862fb-16a7-4ea0-a370-6e521f75240b None None] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:58.265874 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77862fb-16a7-4ea0-a370-6e521f75240b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:58.310703 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.auth.core [None req-e77862fb-16a7-4ea0-a370-6e521f75240b None None] MFA Rules not processed for user `fd836cf590a145df9acf19cf3ae2ac56`. Rule list: `[]` (Enabled: `True`). {{(pid=88113) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:48:58.312105 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.fernet_utils [None req-e77862fb-16a7-4ea0-a370-6e521f75240b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:48:58.414891 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 1012/2024] 10.222.0.13 () {58 vars in 958 bytes} [Wed Aug 31 09:48:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:48:58.427824 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ef71ac-636b-471e-b69b-e14035460714 admin admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:48:58.429875 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.fernet_utils [None req-63ef71ac-636b-471e-b69b-e14035460714 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88114) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:48:58.467241 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e77862fb-16a7-4ea0-a370-6e521f75240b None None] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:48:58.470656 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.fernet_utils [None req-e77862fb-16a7-4ea0-a370-6e521f75240b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:48:58.503506 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e77862fb-16a7-4ea0-a370-6e521f75240b None None] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:48:58.527883 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7054d0ef-42f5-480b-ae76-6af6d882a773 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:48:58.528655 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7054d0ef-42f5-480b-ae76-6af6d882a773 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:58.529019 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7054d0ef-42f5-480b-ae76-6af6d882a773 admin admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:58.529367 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7054d0ef-42f5-480b-ae76-6af6d882a773 admin admin] PATH_INFO: `/v3/projects/737dc8a01dfe4ba1ba9239e4d183eccd` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:58.531531 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7054d0ef-42f5-480b-ae76-6af6d882a773 admin admin] RBAC: Authorizing `identity:delete_project(project_id=737dc8a01dfe4ba1ba9239e4d183eccd)` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:48:58.534107 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ef71ac-636b-471e-b69b-e14035460714 admin admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:48:58.536195 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7054d0ef-42f5-480b-ae76-6af6d882a773 admin admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:48:58.538883 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a033e831-6102-49a4-8f2e-03902d3cc896 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9bdaacd597442c2bcde2c555f2cc140', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '49548eb44d62415582af820ab66decee', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:48:58.540159 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a033e831-6102-49a4-8f2e-03902d3cc896 service neutron] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:58.540159 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a033e831-6102-49a4-8f2e-03902d3cc896 service neutron] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:58.540159 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a033e831-6102-49a4-8f2e-03902d3cc896 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:58.553281 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a033e831-6102-49a4-8f2e-03902d3cc896 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:48:58.555084 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a033e831-6102-49a4-8f2e-03902d3cc896 service neutron] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:48:58.558833 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:48:58.568343 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 1013/2025] 10.222.0.13 () {60 vars in 1365 bytes} [Wed Aug 31 09:48:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:48:58.629817 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.notifications [None req-7054d0ef-42f5-480b-ae76-6af6d882a773 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '737dc8a01dfe4ba1ba9239e4d183eccd', 'request_id': 'req-7054d0ef-42f5-480b-ae76-6af6d882a773'} {{(pid=88113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:48:58.639709 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.notifications [None req-7054d0ef-42f5-480b-ae76-6af6d882a773 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '737dc8a01dfe4ba1ba9239e4d183eccd', 'request_id': 'req-7054d0ef-42f5-480b-ae76-6af6d882a773'} {{(pid=88113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:48:58.650907 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.notifications [None req-7054d0ef-42f5-480b-ae76-6af6d882a773 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '737dc8a01dfe4ba1ba9239e4d183eccd', 'request_id': 'req-7054d0ef-42f5-480b-ae76-6af6d882a773'} {{(pid=88113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:48:58.668965 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.notifications [None req-7054d0ef-42f5-480b-ae76-6af6d882a773 admin admin] The token cache is being invalidate because project 737dc8a01dfe4ba1ba9239e4d183eccd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88113) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 09:48:58.669294 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.notifications [None req-7054d0ef-42f5-480b-ae76-6af6d882a773 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:48:58.670432 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.notifications [None req-7054d0ef-42f5-480b-ae76-6af6d882a773 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:48:58.675966 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 1013/2026] 10.222.0.13 () {58 vars in 1233 bytes} [Wed Aug 31 09:48:58 2022] DELETE /identity/v3/projects/737dc8a01dfe4ba1ba9239e4d183eccd => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:48:58.678192 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a033e831-6102-49a4-8f2e-03902d3cc896 service neutron] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:48:58.694748 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a033e831-6102-49a4-8f2e-03902d3cc896 service neutron] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:48:58.830025 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6cba414-8b0b-440f-8aae-816749395c0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:48:58.830697 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6cba414-8b0b-440f-8aae-816749395c0e admin admin] REQUEST_METHOD: `DELETE` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:58.831062 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6cba414-8b0b-440f-8aae-816749395c0e admin admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:58.831382 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6cba414-8b0b-440f-8aae-816749395c0e admin admin] PATH_INFO: `/v3/users/c81e3fd1717c45ceb2082c75bd8a59a3` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:58.833051 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6cba414-8b0b-440f-8aae-816749395c0e admin admin] RBAC: Authorizing `identity:delete_user(user_id=c81e3fd1717c45ceb2082c75bd8a59a3)` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:48:58.835112 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6cba414-8b0b-440f-8aae-816749395c0e admin admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:48:58.890041 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:48:58.948926 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.notifications [None req-e6cba414-8b0b-440f-8aae-816749395c0e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c81e3fd1717c45ceb2082c75bd8a59a3'} {{(pid=88114) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:48:58.978506 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.notifications [None req-e6cba414-8b0b-440f-8aae-816749395c0e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c81e3fd1717c45ceb2082c75bd8a59a3'} {{(pid=88114) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:48:58.999640 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.notifications [None req-e6cba414-8b0b-440f-8aae-816749395c0e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c81e3fd1717c45ceb2082c75bd8a59a3'} {{(pid=88114) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:48:59.000764 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.notifications [None req-e6cba414-8b0b-440f-8aae-816749395c0e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c81e3fd1717c45ceb2082c75bd8a59a3'} {{(pid=88114) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:48:59.043430 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.notifications [None req-e6cba414-8b0b-440f-8aae-816749395c0e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c81e3fd1717c45ceb2082c75bd8a59a3'} {{(pid=88114) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:48:59.064607 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 1014/2027] 10.222.0.13 () {58 vars in 1224 bytes} [Wed Aug 31 09:48:58 2022] DELETE /identity/v3/users/c81e3fd1717c45ceb2082c75bd8a59a3 => generated 0 bytes in 388 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 09:48:59.197978 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7054d0ef-42f5-480b-ae76-6af6d882a773 admin admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:48:59.199591 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.fernet_utils [None req-7054d0ef-42f5-480b-ae76-6af6d882a773 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:48:59.227336 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7054d0ef-42f5-480b-ae76-6af6d882a773 admin admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:48:59.378157 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe1a77c-2968-4370-952b-268040d3e642 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:48:59.379309 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe1a77c-2968-4370-952b-268040d3e642 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:59.379820 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe1a77c-2968-4370-952b-268040d3e642 admin admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:59.380282 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe1a77c-2968-4370-952b-268040d3e642 admin admin] PATH_INFO: `/v3/projects/f132219bdfb14b5395c5cf3811b1b57c` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:59.383017 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbe1a77c-2968-4370-952b-268040d3e642 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f132219bdfb14b5395c5cf3811b1b57c)` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:48:59.385829 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbe1a77c-2968-4370-952b-268040d3e642 admin admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:48:59.500941 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.notifications [None req-fbe1a77c-2968-4370-952b-268040d3e642 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f132219bdfb14b5395c5cf3811b1b57c', 'request_id': 'req-fbe1a77c-2968-4370-952b-268040d3e642'} {{(pid=88113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:48:59.512479 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.notifications [None req-fbe1a77c-2968-4370-952b-268040d3e642 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f132219bdfb14b5395c5cf3811b1b57c', 'request_id': 'req-fbe1a77c-2968-4370-952b-268040d3e642'} {{(pid=88113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:48:59.525088 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.notifications [None req-fbe1a77c-2968-4370-952b-268040d3e642 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f132219bdfb14b5395c5cf3811b1b57c', 'request_id': 'req-fbe1a77c-2968-4370-952b-268040d3e642'} {{(pid=88113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:48:59.545195 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.notifications [None req-fbe1a77c-2968-4370-952b-268040d3e642 admin admin] The token cache is being invalidate because project f132219bdfb14b5395c5cf3811b1b57c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88113) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 09:48:59.547148 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.notifications [None req-fbe1a77c-2968-4370-952b-268040d3e642 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:48:59.548391 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.notifications [None req-fbe1a77c-2968-4370-952b-268040d3e642 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:48:59.553078 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 1014/2028] 10.222.0.13 () {58 vars in 1233 bytes} [Wed Aug 31 09:48:59 2022] DELETE /identity/v3/projects/f132219bdfb14b5395c5cf3811b1b57c => generated 0 bytes in 359 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:48:59.557773 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6cba414-8b0b-440f-8aae-816749395c0e admin admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:48:59.562051 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.fernet_utils [None req-e6cba414-8b0b-440f-8aae-816749395c0e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88114) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:48:59.628189 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6cba414-8b0b-440f-8aae-816749395c0e admin admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:48:59.861028 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1426e7c-59fe-45b9-8d60-812bcade43f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:48:59.862783 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1426e7c-59fe-45b9-8d60-812bcade43f9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:48:59.863449 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1426e7c-59fe-45b9-8d60-812bcade43f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:48:59.863931 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1426e7c-59fe-45b9-8d60-812bcade43f9 admin admin] PATH_INFO: `/v3/users/1d7b408a9b804ccca57fce60fc527e7c` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:48:59.866356 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1426e7c-59fe-45b9-8d60-812bcade43f9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1d7b408a9b804ccca57fce60fc527e7c)` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:48:59.872726 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1426e7c-59fe-45b9-8d60-812bcade43f9 admin admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:48:59.990513 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.notifications [None req-e1426e7c-59fe-45b9-8d60-812bcade43f9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1d7b408a9b804ccca57fce60fc527e7c'} {{(pid=88114) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:49:00.011198 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.notifications [None req-e1426e7c-59fe-45b9-8d60-812bcade43f9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1d7b408a9b804ccca57fce60fc527e7c'} {{(pid=88114) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:49:00.032951 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.notifications [None req-e1426e7c-59fe-45b9-8d60-812bcade43f9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1d7b408a9b804ccca57fce60fc527e7c'} {{(pid=88114) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:49:00.033933 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.notifications [None req-e1426e7c-59fe-45b9-8d60-812bcade43f9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1d7b408a9b804ccca57fce60fc527e7c'} {{(pid=88114) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:49:00.054219 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.notifications [None req-e1426e7c-59fe-45b9-8d60-812bcade43f9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1d7b408a9b804ccca57fce60fc527e7c'} {{(pid=88114) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:49:00.079580 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 1015/2029] 10.222.0.13 () {58 vars in 1224 bytes} [Wed Aug 31 09:48:59 2022] DELETE /identity/v3/users/1d7b408a9b804ccca57fce60fc527e7c => generated 0 bytes in 523 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:49:00.277426 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe1a77c-2968-4370-952b-268040d3e642 admin admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:49:00.280841 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.fernet_utils [None req-fbe1a77c-2968-4370-952b-268040d3e642 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:49:00.373098 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe1a77c-2968-4370-952b-268040d3e642 admin admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:49:00.486735 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:49:00.659734 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13988dc4-0729-4c61-9291-50ef89828e68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:49:00.660944 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13988dc4-0729-4c61-9291-50ef89828e68 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:00.661505 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13988dc4-0729-4c61-9291-50ef89828e68 admin admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:00.662101 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13988dc4-0729-4c61-9291-50ef89828e68 admin admin] PATH_INFO: `/v3/projects/fc3b8644859b4e73878b2156defc9563` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:00.664318 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13988dc4-0729-4c61-9291-50ef89828e68 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fc3b8644859b4e73878b2156defc9563)` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:49:00.666993 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13988dc4-0729-4c61-9291-50ef89828e68 admin admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:49:00.769771 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.notifications [None req-13988dc4-0729-4c61-9291-50ef89828e68 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fc3b8644859b4e73878b2156defc9563', 'request_id': 'req-13988dc4-0729-4c61-9291-50ef89828e68'} {{(pid=88113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:49:00.781210 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.notifications [None req-13988dc4-0729-4c61-9291-50ef89828e68 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fc3b8644859b4e73878b2156defc9563', 'request_id': 'req-13988dc4-0729-4c61-9291-50ef89828e68'} {{(pid=88113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:49:00.798153 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.notifications [None req-13988dc4-0729-4c61-9291-50ef89828e68 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fc3b8644859b4e73878b2156defc9563', 'request_id': 'req-13988dc4-0729-4c61-9291-50ef89828e68'} {{(pid=88113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:49:00.827661 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.notifications [None req-13988dc4-0729-4c61-9291-50ef89828e68 admin admin] The token cache is being invalidate because project fc3b8644859b4e73878b2156defc9563 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88113) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 09:49:00.828171 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.notifications [None req-13988dc4-0729-4c61-9291-50ef89828e68 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:49:00.829367 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.notifications [None req-13988dc4-0729-4c61-9291-50ef89828e68 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:49:00.837718 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 1015/2030] 10.222.0.13 () {58 vars in 1233 bytes} [Wed Aug 31 09:49:00 2022] DELETE /identity/v3/projects/fc3b8644859b4e73878b2156defc9563 => generated 0 bytes in 557 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:49:00.840384 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2de98b0-0aab-42db-a679-f41f72dc42ee None None] REQUEST_METHOD: `POST` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:00.840911 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2de98b0-0aab-42db-a679-f41f72dc42ee None None] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:00.840911 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2de98b0-0aab-42db-a679-f41f72dc42ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:00.886119 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.auth.core [None req-e2de98b0-0aab-42db-a679-f41f72dc42ee None None] MFA Rules not processed for user `fd836cf590a145df9acf19cf3ae2ac56`. Rule list: `[]` (Enabled: `True`). {{(pid=88114) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:49:00.887364 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.fernet_utils [None req-e2de98b0-0aab-42db-a679-f41f72dc42ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88114) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:49:00.925523 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:49:01.001918 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 1016/2031] 10.222.0.13 () {58 vars in 958 bytes} [Wed Aug 31 09:49:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 09:49:01.009234 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a679a44c-2f04-43a5-ac1b-4f62abbcead8 None None] REQUEST_METHOD: `POST` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:01.009896 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a679a44c-2f04-43a5-ac1b-4f62abbcead8 None None] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:01.010417 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a679a44c-2f04-43a5-ac1b-4f62abbcead8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:01.037902 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.auth.core [None req-a679a44c-2f04-43a5-ac1b-4f62abbcead8 None None] MFA Rules not processed for user `fd836cf590a145df9acf19cf3ae2ac56`. Rule list: `[]` (Enabled: `True`). {{(pid=88113) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:49:01.039303 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.fernet_utils [None req-a679a44c-2f04-43a5-ac1b-4f62abbcead8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:49:01.049242 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 1016/2032] 10.222.0.13 () {58 vars in 958 bytes} [Wed Aug 31 09:49:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:49:01.053458 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2de98b0-0aab-42db-a679-f41f72dc42ee None None] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:49:01.067468 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2de98b0-0aab-42db-a679-f41f72dc42ee None None] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:49:01.073250 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a7b987-4fce-4034-80a0-3a858b5c6fc3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:49:01.074186 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a7b987-4fce-4034-80a0-3a858b5c6fc3 admin admin] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:01.074591 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a7b987-4fce-4034-80a0-3a858b5c6fc3 admin admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:01.077797 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a7b987-4fce-4034-80a0-3a858b5c6fc3 admin admin] PATH_INFO: `/v3/domains` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:01.078669 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13a7b987-4fce-4034-80a0-3a858b5c6fc3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88114) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:49:01.079068 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13a7b987-4fce-4034-80a0-3a858b5c6fc3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:49:01.082202 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13a7b987-4fce-4034-80a0-3a858b5c6fc3 admin admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:49:01.103436 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 1017/2033] 10.222.0.13 () {58 vars in 1153 bytes} [Wed Aug 31 09:49:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:49:01.111318 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a679a44c-2f04-43a5-ac1b-4f62abbcead8 None None] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:49:01.122767 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a679a44c-2f04-43a5-ac1b-4f62abbcead8 None None] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:49:01.129081 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c0b81f-83c4-4893-aa8b-f0ad4cb68764 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:49:01.130016 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c0b81f-83c4-4893-aa8b-f0ad4cb68764 admin admin] REQUEST_METHOD: `POST` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:01.130372 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c0b81f-83c4-4893-aa8b-f0ad4cb68764 admin admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:01.130780 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c0b81f-83c4-4893-aa8b-f0ad4cb68764 admin admin] PATH_INFO: `/v3/projects` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:01.131521 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3c0b81f-83c4-4893-aa8b-f0ad4cb68764 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:49:01.133761 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3c0b81f-83c4-4893-aa8b-f0ad4cb68764 admin admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:49:01.174449 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 1017/2034] 10.222.0.13 () {60 vars in 1153 bytes} [Wed Aug 31 09:49:01 2022] POST /identity/v3/projects => generated 387 bytes in 63 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 09:49:01.179152 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a7b987-4fce-4034-80a0-3a858b5c6fc3 admin admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:49:01.196508 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a7b987-4fce-4034-80a0-3a858b5c6fc3 admin admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:49:01.293405 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-419fb879-8030-4216-942b-d8cc42a2142c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:49:01.294286 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419fb879-8030-4216-942b-d8cc42a2142c admin admin] REQUEST_METHOD: `POST` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:01.294597 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419fb879-8030-4216-942b-d8cc42a2142c admin admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:01.295298 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419fb879-8030-4216-942b-d8cc42a2142c admin admin] PATH_INFO: `/v3/users` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:01.296706 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-419fb879-8030-4216-942b-d8cc42a2142c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:49:01.299529 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-419fb879-8030-4216-942b-d8cc42a2142c admin admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:49:01.349151 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 1018/2035] 10.222.0.13 () {60 vars in 1144 bytes} [Wed Aug 31 09:49:01 2022] POST /identity/v3/users => generated 294 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 09:49:01.353866 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c0b81f-83c4-4893-aa8b-f0ad4cb68764 admin admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:49:01.366435 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c0b81f-83c4-4893-aa8b-f0ad4cb68764 admin admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:49:01.372009 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e70a9a-0399-4d5a-87a7-cfc1c33ec069 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:49:01.372660 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e70a9a-0399-4d5a-87a7-cfc1c33ec069 admin admin] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:01.373007 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e70a9a-0399-4d5a-87a7-cfc1c33ec069 admin admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:01.373221 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e70a9a-0399-4d5a-87a7-cfc1c33ec069 admin admin] PATH_INFO: `/v3/roles` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:01.374017 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8e70a9a-0399-4d5a-87a7-cfc1c33ec069 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:49:01.375613 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8e70a9a-0399-4d5a-87a7-cfc1c33ec069 admin admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:49:01.393787 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-419fb879-8030-4216-942b-d8cc42a2142c admin admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:49:01.400524 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 1018/2036] 10.222.0.13 () {58 vars in 1122 bytes} [Wed Aug 31 09:49:01 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 09:49:01.413272 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-419fb879-8030-4216-942b-d8cc42a2142c admin admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:49:01.423917 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c72f8e09-f020-43b8-bae4-f8e717753a52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:49:01.425189 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72f8e09-f020-43b8-bae4-f8e717753a52 admin admin] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:01.425893 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72f8e09-f020-43b8-bae4-f8e717753a52 admin admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:01.426504 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72f8e09-f020-43b8-bae4-f8e717753a52 admin admin] PATH_INFO: `/v3/roles` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:01.427039 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c72f8e09-f020-43b8-bae4-f8e717753a52 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:49:01.428842 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c72f8e09-f020-43b8-bae4-f8e717753a52 admin admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:49:01.462145 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e70a9a-0399-4d5a-87a7-cfc1c33ec069 admin admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:49:01.463262 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 1019/2037] 10.222.0.13 () {58 vars in 1122 bytes} [Wed Aug 31 09:49:01 2022] GET /identity/v3/roles => generated 1420 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 09:49:01.477547 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e70a9a-0399-4d5a-87a7-cfc1c33ec069 admin admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:49:01.483177 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a3ccb6-ea88-4571-b5d2-c4932bea105e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:49:01.483887 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a3ccb6-ea88-4571-b5d2-c4932bea105e admin admin] REQUEST_METHOD: `PUT` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:01.484170 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a3ccb6-ea88-4571-b5d2-c4932bea105e admin admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:01.484444 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a3ccb6-ea88-4571-b5d2-c4932bea105e admin admin] PATH_INFO: `/v3/projects/2eb58e59f570457b955ec9fde3af6944/users/53f4390bd6b7419b892997cb01755d16/roles/6c99f5d9f1f243eaa134d8540f21095d` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:01.488337 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c72f8e09-f020-43b8-bae4-f8e717753a52 admin admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:49:01.492432 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.fernet_utils [None req-c72f8e09-f020-43b8-bae4-f8e717753a52 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88114) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:49:01.504980 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25a3ccb6-ea88-4571-b5d2-c4932bea105e admin admin] RBAC: Authorizing `identity:create_grant(project_id=2eb58e59f570457b955ec9fde3af6944, user_id=53f4390bd6b7419b892997cb01755d16, role_id=6c99f5d9f1f243eaa134d8540f21095d)` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:49:01.508730 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25a3ccb6-ea88-4571-b5d2-c4932bea105e admin admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:49:01.547248 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 1019/2038] 10.222.0.13 () {60 vars in 1483 bytes} [Wed Aug 31 09:49:01 2022] PUT /identity/v3/projects/2eb58e59f570457b955ec9fde3af6944/users/53f4390bd6b7419b892997cb01755d16/roles/6c99f5d9f1f243eaa134d8540f21095d => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 09:49:01.547248 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677676a6-e3e4-4a0c-80fe-d2772da45690 None None] REQUEST_METHOD: `POST` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:01.547248 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677676a6-e3e4-4a0c-80fe-d2772da45690 None None] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:01.547248 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677676a6-e3e4-4a0c-80fe-d2772da45690 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:01.550674 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c72f8e09-f020-43b8-bae4-f8e717753a52 admin admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:49:01.584979 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.auth.core [None req-677676a6-e3e4-4a0c-80fe-d2772da45690 None None] MFA Rules not processed for user `53f4390bd6b7419b892997cb01755d16`. Rule list: `[]` (Enabled: `True`). {{(pid=88113) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:49:01.586917 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.fernet_utils [None req-677676a6-e3e4-4a0c-80fe-d2772da45690 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:49:01.641758 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:49:01.718510 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 1020/2039] 10.222.0.13 () {58 vars in 958 bytes} [Wed Aug 31 09:49:01 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:49:01.720289 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-677676a6-e3e4-4a0c-80fe-d2772da45690 None None] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:49:01.754970 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9068b29-f0c4-46ef-9bdb-8f467e82de27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:49:01.756080 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9068b29-f0c4-46ef-9bdb-8f467e82de27 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:01.756533 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9068b29-f0c4-46ef-9bdb-8f467e82de27 admin admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:01.757315 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9068b29-f0c4-46ef-9bdb-8f467e82de27 admin admin] PATH_INFO: `/v3/projects/e2d1e8d7858849eaaf92de0406282ac8` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:01.761428 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9068b29-f0c4-46ef-9bdb-8f467e82de27 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e2d1e8d7858849eaaf92de0406282ac8)` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:49:01.767604 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9068b29-f0c4-46ef-9bdb-8f467e82de27 admin admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:49:01.834083 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-677676a6-e3e4-4a0c-80fe-d2772da45690 None None] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:49:01.847965 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d41954-55a2-4f09-8363-9525dec97801 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:49:01.852775 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d41954-55a2-4f09-8363-9525dec97801 admin admin] REQUEST_METHOD: `POST` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:01.853311 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d41954-55a2-4f09-8363-9525dec97801 admin admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:01.854282 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d41954-55a2-4f09-8363-9525dec97801 admin admin] PATH_INFO: `/v3/projects` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:01.856569 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66d41954-55a2-4f09-8363-9525dec97801 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:49:01.859279 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66d41954-55a2-4f09-8363-9525dec97801 admin admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:49:01.887148 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=95,keystone:INSERT=3,keystone:DELETE=2 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:49:01.912839 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 1021/2040] 10.222.0.13 () {60 vars in 1153 bytes} [Wed Aug 31 09:49:01 2022] POST /identity/v3/projects => generated 387 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 09:49:01.912839 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d41954-55a2-4f09-8363-9525dec97801 admin admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:49:01.917016 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.fernet_utils [None req-66d41954-55a2-4f09-8363-9525dec97801 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:49:01.935766 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.notifications [None req-b9068b29-f0c4-46ef-9bdb-8f467e82de27 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e2d1e8d7858849eaaf92de0406282ac8', 'request_id': 'req-b9068b29-f0c4-46ef-9bdb-8f467e82de27'} {{(pid=88114) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:49:02.001605 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.notifications [None req-b9068b29-f0c4-46ef-9bdb-8f467e82de27 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e2d1e8d7858849eaaf92de0406282ac8', 'request_id': 'req-b9068b29-f0c4-46ef-9bdb-8f467e82de27'} {{(pid=88114) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:49:02.026775 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.notifications [None req-b9068b29-f0c4-46ef-9bdb-8f467e82de27 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e2d1e8d7858849eaaf92de0406282ac8', 'request_id': 'req-b9068b29-f0c4-46ef-9bdb-8f467e82de27'} {{(pid=88114) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:49:02.066355 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.notifications [None req-b9068b29-f0c4-46ef-9bdb-8f467e82de27 admin admin] The token cache is being invalidate because project e2d1e8d7858849eaaf92de0406282ac8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88114) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 09:49:02.066816 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d41954-55a2-4f09-8363-9525dec97801 admin admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:49:02.068637 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.notifications [None req-b9068b29-f0c4-46ef-9bdb-8f467e82de27 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88114) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:49:02.070154 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.notifications [None req-b9068b29-f0c4-46ef-9bdb-8f467e82de27 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88114) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:49:02.073910 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 1020/2041] 10.222.0.13 () {58 vars in 1233 bytes} [Wed Aug 31 09:49:01 2022] DELETE /identity/v3/projects/e2d1e8d7858849eaaf92de0406282ac8 => generated 0 bytes in 588 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:49:02.075776 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9068b29-f0c4-46ef-9bdb-8f467e82de27 admin admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:49:02.078860 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.fernet_utils [None req-b9068b29-f0c4-46ef-9bdb-8f467e82de27 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88114) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:49:02.079415 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e75bbf-71c1-4217-becb-a033c2926d8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:49:02.080096 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e75bbf-71c1-4217-becb-a033c2926d8d admin admin] REQUEST_METHOD: `DELETE` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:02.080378 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e75bbf-71c1-4217-becb-a033c2926d8d admin admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:02.080713 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e75bbf-71c1-4217-becb-a033c2926d8d admin admin] PATH_INFO: `/v3/users/9ca662ced8784e2185764bcbdf921ec4` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:02.082886 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16e75bbf-71c1-4217-becb-a033c2926d8d admin admin] RBAC: Authorizing `identity:delete_user(user_id=9ca662ced8784e2185764bcbdf921ec4)` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:49:02.084856 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16e75bbf-71c1-4217-becb-a033c2926d8d admin admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:49:02.207356 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.notifications [None req-16e75bbf-71c1-4217-becb-a033c2926d8d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9ca662ced8784e2185764bcbdf921ec4'} {{(pid=88113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:49:02.224810 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.notifications [None req-16e75bbf-71c1-4217-becb-a033c2926d8d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9ca662ced8784e2185764bcbdf921ec4'} {{(pid=88113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:49:02.242834 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9068b29-f0c4-46ef-9bdb-8f467e82de27 admin admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:49:02.246816 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=90,keystone:INSERT=1,keystone:DELETE=9 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:49:02.249144 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e072b4-8ce0-4c37-87ba-ef52cd6559c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:49:02.249761 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.notifications [None req-16e75bbf-71c1-4217-becb-a033c2926d8d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9ca662ced8784e2185764bcbdf921ec4'} {{(pid=88113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:49:02.250661 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e072b4-8ce0-4c37-87ba-ef52cd6559c2 admin admin] REQUEST_METHOD: `POST` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:02.251100 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e072b4-8ce0-4c37-87ba-ef52cd6559c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:02.251815 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e072b4-8ce0-4c37-87ba-ef52cd6559c2 admin admin] PATH_INFO: `/v3/users` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:02.253130 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8e072b4-8ce0-4c37-87ba-ef52cd6559c2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:49:02.253620 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.notifications [None req-16e75bbf-71c1-4217-becb-a033c2926d8d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9ca662ced8784e2185764bcbdf921ec4'} {{(pid=88113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:49:02.255793 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8e072b4-8ce0-4c37-87ba-ef52cd6559c2 admin admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:49:02.274667 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.notifications [None req-16e75bbf-71c1-4217-becb-a033c2926d8d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9ca662ced8784e2185764bcbdf921ec4'} {{(pid=88113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:49:02.292694 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 1022/2042] 10.222.0.13 () {58 vars in 1224 bytes} [Wed Aug 31 09:49:01 2022] DELETE /identity/v3/users/9ca662ced8784e2185764bcbdf921ec4 => generated 0 bytes in 384 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:49:02.295034 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a71be71-8226-45e3-94bb-88f5758f8b97 None None] REQUEST_METHOD: `POST` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:02.295626 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a71be71-8226-45e3-94bb-88f5758f8b97 None None] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:02.296223 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a71be71-8226-45e3-94bb-88f5758f8b97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:02.307781 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 1021/2043] 10.222.0.13 () {60 vars in 1144 bytes} [Wed Aug 31 09:49:02 2022] POST /identity/v3/users => generated 294 bytes in 233 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 09:49:02.309262 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8e52f4-314f-43b6-bef3-9bab7e5469ce None None] REQUEST_METHOD: `POST` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:02.309567 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8e52f4-314f-43b6-bef3-9bab7e5469ce None None] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:02.309845 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8e52f4-314f-43b6-bef3-9bab7e5469ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:02.328178 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.auth.core [None req-2a71be71-8226-45e3-94bb-88f5758f8b97 None None] MFA Rules not processed for user `fd836cf590a145df9acf19cf3ae2ac56`. Rule list: `[]` (Enabled: `True`). {{(pid=88113) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:49:02.330064 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.fernet_utils [None req-2a71be71-8226-45e3-94bb-88f5758f8b97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:49:02.332498 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.auth.core [None req-dc8e52f4-314f-43b6-bef3-9bab7e5469ce None None] MFA Rules not processed for user `fd836cf590a145df9acf19cf3ae2ac56`. Rule list: `[]` (Enabled: `True`). {{(pid=88114) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:49:02.334484 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.fernet_utils [None req-dc8e52f4-314f-43b6-bef3-9bab7e5469ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88114) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:49:02.453000 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 1022/2044] 10.222.0.13 () {58 vars in 958 bytes} [Wed Aug 31 09:49:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:49:02.454422 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc8e52f4-314f-43b6-bef3-9bab7e5469ce None None] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:49:02.456284 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.fernet_utils [None req-dc8e52f4-314f-43b6-bef3-9bab7e5469ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88114) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:49:02.474595 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 1023/2045] 10.222.0.13 () {58 vars in 958 bytes} [Wed Aug 31 09:49:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:49:02.474595 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a71be71-8226-45e3-94bb-88f5758f8b97 None None] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:49:02.474595 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.fernet_utils [None req-2a71be71-8226-45e3-94bb-88f5758f8b97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:49:02.481262 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc8e52f4-314f-43b6-bef3-9bab7e5469ce None None] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:49:02.492550 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29dcdb9b-2483-4dd2-a38e-7e460c34834a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:49:02.494184 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29dcdb9b-2483-4dd2-a38e-7e460c34834a admin admin] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:02.495255 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29dcdb9b-2483-4dd2-a38e-7e460c34834a admin admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:02.496058 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29dcdb9b-2483-4dd2-a38e-7e460c34834a admin admin] PATH_INFO: `/v3/roles` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:02.496731 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29dcdb9b-2483-4dd2-a38e-7e460c34834a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:49:02.498592 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29dcdb9b-2483-4dd2-a38e-7e460c34834a admin admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:49:02.519300 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 1023/2046] 10.222.0.13 () {58 vars in 1122 bytes} [Wed Aug 31 09:49:02 2022] GET /identity/v3/roles => generated 1420 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 09:49:02.520785 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0bcdb9-053f-4c77-ae17-16c1dd565873 None None] REQUEST_METHOD: `POST` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:02.521079 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0bcdb9-053f-4c77-ae17-16c1dd565873 None None] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:02.521531 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0bcdb9-053f-4c77-ae17-16c1dd565873 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:02.549970 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.auth.core [None req-1a0bcdb9-053f-4c77-ae17-16c1dd565873 None None] MFA Rules not processed for user `fd836cf590a145df9acf19cf3ae2ac56`. Rule list: `[]` (Enabled: `True`). {{(pid=88114) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:49:02.551651 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.fernet_utils [None req-1a0bcdb9-053f-4c77-ae17-16c1dd565873 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88114) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:49:02.562287 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 1024/2047] 10.222.0.13 () {58 vars in 958 bytes} [Wed Aug 31 09:49:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:49:02.562657 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a0bcdb9-053f-4c77-ae17-16c1dd565873 None None] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:49:02.575540 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a0bcdb9-053f-4c77-ae17-16c1dd565873 None None] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:49:02.582677 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c300fa24-5912-4996-8bf9-67c5d69eaf5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:49:02.583309 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c300fa24-5912-4996-8bf9-67c5d69eaf5e admin admin] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:02.583583 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c300fa24-5912-4996-8bf9-67c5d69eaf5e admin admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:02.583851 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c300fa24-5912-4996-8bf9-67c5d69eaf5e admin admin] PATH_INFO: `/v3/roles` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:02.584512 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c300fa24-5912-4996-8bf9-67c5d69eaf5e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:49:02.587539 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c300fa24-5912-4996-8bf9-67c5d69eaf5e admin admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:49:02.604813 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 1025/2048] 10.222.0.13 () {58 vars in 1122 bytes} [Wed Aug 31 09:49:02 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 09:49:02.607117 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a71be71-8226-45e3-94bb-88f5758f8b97 None None] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:49:02.609035 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c300fa24-5912-4996-8bf9-67c5d69eaf5e admin admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:49:02.614633 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c300f6-a492-4ccd-b577-03ee76c74423 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9bdaacd597442c2bcde2c555f2cc140', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '49548eb44d62415582af820ab66decee', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:49:02.616338 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c300f6-a492-4ccd-b577-03ee76c74423 service neutron] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:02.616820 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c300f6-a492-4ccd-b577-03ee76c74423 service neutron] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:02.617761 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c300f6-a492-4ccd-b577-03ee76c74423 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:02.619843 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c300fa24-5912-4996-8bf9-67c5d69eaf5e admin admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:49:02.626073 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c7ab2c-f5ee-4806-ac65-dce288746e59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:49:02.627361 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c7ab2c-f5ee-4806-ac65-dce288746e59 admin admin] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:02.628354 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c7ab2c-f5ee-4806-ac65-dce288746e59 admin admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:02.628479 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c7ab2c-f5ee-4806-ac65-dce288746e59 admin admin] PATH_INFO: `/v3/domains` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:02.629118 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78c7ab2c-f5ee-4806-ac65-dce288746e59 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88114) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:49:02.629438 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78c7ab2c-f5ee-4806-ac65-dce288746e59 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:49:02.631298 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78c7ab2c-f5ee-4806-ac65-dce288746e59 admin admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:49:02.637756 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31c300f6-a492-4ccd-b577-03ee76c74423 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:49:02.641914 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31c300f6-a492-4ccd-b577-03ee76c74423 service neutron] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:49:02.654763 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 1026/2049] 10.222.0.13 () {58 vars in 1153 bytes} [Wed Aug 31 09:49:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:49:02.654763 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c7ab2c-f5ee-4806-ac65-dce288746e59 admin admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:49:02.679465 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c7ab2c-f5ee-4806-ac65-dce288746e59 admin admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:49:02.683066 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 1024/2050] 10.222.0.13 () {60 vars in 1365 bytes} [Wed Aug 31 09:49:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 215 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:49:02.684273 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c300f6-a492-4ccd-b577-03ee76c74423 service neutron] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:49:02.692406 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad4ae9c-ca76-4f6a-a354-aa3b26667b90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:49:02.693123 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad4ae9c-ca76-4f6a-a354-aa3b26667b90 admin admin] REQUEST_METHOD: `PUT` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:02.693425 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad4ae9c-ca76-4f6a-a354-aa3b26667b90 admin admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:02.693697 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad4ae9c-ca76-4f6a-a354-aa3b26667b90 admin admin] PATH_INFO: `/v3/projects/047132299e3a41c4b1e610a19666b929/users/473970514dab4dbb9a046737ecc73b7e/roles/6c99f5d9f1f243eaa134d8540f21095d` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:02.697326 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c300f6-a492-4ccd-b577-03ee76c74423 service neutron] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:49:02.703931 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f8442e-e485-4b7f-aee7-0c9e1b78d776 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:49:02.704509 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:49:02.707065 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f8442e-e485-4b7f-aee7-0c9e1b78d776 admin admin] REQUEST_METHOD: `POST` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:02.707653 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f8442e-e485-4b7f-aee7-0c9e1b78d776 admin admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:02.708106 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f8442e-e485-4b7f-aee7-0c9e1b78d776 admin admin] PATH_INFO: `/v3/projects` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:02.709094 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4f8442e-e485-4b7f-aee7-0c9e1b78d776 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:49:02.711743 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4f8442e-e485-4b7f-aee7-0c9e1b78d776 admin admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:49:02.730649 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ad4ae9c-ca76-4f6a-a354-aa3b26667b90 admin admin] RBAC: Authorizing `identity:create_grant(project_id=047132299e3a41c4b1e610a19666b929, user_id=473970514dab4dbb9a046737ecc73b7e, role_id=6c99f5d9f1f243eaa134d8540f21095d)` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:49:02.734799 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ad4ae9c-ca76-4f6a-a354-aa3b26667b90 admin admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:49:02.770713 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 1027/2051] 10.222.0.13 () {60 vars in 1483 bytes} [Wed Aug 31 09:49:02 2022] PUT /identity/v3/projects/047132299e3a41c4b1e610a19666b929/users/473970514dab4dbb9a046737ecc73b7e/roles/6c99f5d9f1f243eaa134d8540f21095d => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:49:02.771795 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad4ae9c-ca76-4f6a-a354-aa3b26667b90 admin admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:49:02.777782 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.fernet_utils [None req-0ad4ae9c-ca76-4f6a-a354-aa3b26667b90 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88114) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:49:02.804836 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 1025/2052] 10.222.0.13 () {60 vars in 1153 bytes} [Wed Aug 31 09:49:02 2022] POST /identity/v3/projects => generated 363 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 09:49:02.813512 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4312910-7c94-4a34-b1fb-1d3cbae1cc50 None None] REQUEST_METHOD: `POST` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:02.818374 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4312910-7c94-4a34-b1fb-1d3cbae1cc50 None None] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:02.820925 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4312910-7c94-4a34-b1fb-1d3cbae1cc50 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:02.903832 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.auth.core [None req-a4312910-7c94-4a34-b1fb-1d3cbae1cc50 None None] MFA Rules not processed for user `473970514dab4dbb9a046737ecc73b7e`. Rule list: `[]` (Enabled: `True`). {{(pid=88113) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:49:02.905231 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.fernet_utils [None req-a4312910-7c94-4a34-b1fb-1d3cbae1cc50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:49:02.913724 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad4ae9c-ca76-4f6a-a354-aa3b26667b90 admin admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:49:02.988804 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-672ffe06-1903-48d5-ba3d-4a0ae90f4e73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:49:02.989897 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672ffe06-1903-48d5-ba3d-4a0ae90f4e73 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:02.990246 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672ffe06-1903-48d5-ba3d-4a0ae90f4e73 admin admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:02.990532 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672ffe06-1903-48d5-ba3d-4a0ae90f4e73 admin admin] PATH_INFO: `/v3/users/42c211044366426e92b558a8332194da` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:02.994831 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-672ffe06-1903-48d5-ba3d-4a0ae90f4e73 admin admin] RBAC: Authorizing `identity:delete_user(user_id=42c211044366426e92b558a8332194da)` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:49:02.996855 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-672ffe06-1903-48d5-ba3d-4a0ae90f4e73 admin admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:49:03.054289 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=97,keystone:INSERT=2,keystone:DELETE=1 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:49:03.060655 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 1026/2053] 10.222.0.13 () {58 vars in 958 bytes} [Wed Aug 31 09:49:02 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 255 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:49:03.062432 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4312910-7c94-4a34-b1fb-1d3cbae1cc50 None None] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:49:03.082711 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.notifications [None req-672ffe06-1903-48d5-ba3d-4a0ae90f4e73 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '42c211044366426e92b558a8332194da'} {{(pid=88114) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:49:03.089808 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4312910-7c94-4a34-b1fb-1d3cbae1cc50 None None] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:49:03.099010 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b146f68-efc3-4c74-870d-ca2ff8e30fed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:49:03.100120 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b146f68-efc3-4c74-870d-ca2ff8e30fed admin admin] REQUEST_METHOD: `POST` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:03.100764 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b146f68-efc3-4c74-870d-ca2ff8e30fed admin admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:03.101208 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b146f68-efc3-4c74-870d-ca2ff8e30fed admin admin] PATH_INFO: `/v3/users` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:03.101600 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.notifications [None req-672ffe06-1903-48d5-ba3d-4a0ae90f4e73 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '42c211044366426e92b558a8332194da'} {{(pid=88114) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:49:03.102334 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b146f68-efc3-4c74-870d-ca2ff8e30fed admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:49:03.105186 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b146f68-efc3-4c74-870d-ca2ff8e30fed admin admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:49:03.119696 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.notifications [None req-672ffe06-1903-48d5-ba3d-4a0ae90f4e73 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '42c211044366426e92b558a8332194da'} {{(pid=88114) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:49:03.121914 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.notifications [None req-672ffe06-1903-48d5-ba3d-4a0ae90f4e73 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '42c211044366426e92b558a8332194da'} {{(pid=88114) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:49:03.147485 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 1027/2054] 10.222.0.13 () {60 vars in 1144 bytes} [Wed Aug 31 09:49:03 2022] POST /identity/v3/users => generated 282 bytes in 83 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 09:49:03.147485 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b146f68-efc3-4c74-870d-ca2ff8e30fed admin admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:49:03.152021 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.notifications [None req-672ffe06-1903-48d5-ba3d-4a0ae90f4e73 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '42c211044366426e92b558a8332194da'} {{(pid=88114) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:49:03.153840 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.fernet_utils [None req-5b146f68-efc3-4c74-870d-ca2ff8e30fed admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:49:03.162820 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 1028/2055] 10.222.0.13 () {58 vars in 1224 bytes} [Wed Aug 31 09:49:02 2022] DELETE /identity/v3/users/42c211044366426e92b558a8332194da => generated 0 bytes in 391 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:49:03.163847 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-672ffe06-1903-48d5-ba3d-4a0ae90f4e73 admin admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:49:03.165328 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.fernet_utils [None req-672ffe06-1903-48d5-ba3d-4a0ae90f4e73 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88114) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:49:03.186334 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b146f68-efc3-4c74-870d-ca2ff8e30fed admin admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:49:03.194588 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:49:03.332962 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40daff31-f9be-47e4-a032-4ed371013939 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:49:03.334024 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40daff31-f9be-47e4-a032-4ed371013939 admin admin] REQUEST_METHOD: `POST` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:03.334467 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40daff31-f9be-47e4-a032-4ed371013939 admin admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:03.334925 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40daff31-f9be-47e4-a032-4ed371013939 admin admin] PATH_INFO: `/v3/projects` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:03.336038 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40daff31-f9be-47e4-a032-4ed371013939 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:49:03.347734 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40daff31-f9be-47e4-a032-4ed371013939 admin admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:49:03.352713 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-672ffe06-1903-48d5-ba3d-4a0ae90f4e73 admin admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:49:03.367242 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-990d0d49-1aec-4246-96fb-ddf6621f53db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:49:03.368616 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990d0d49-1aec-4246-96fb-ddf6621f53db admin admin] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:03.369068 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990d0d49-1aec-4246-96fb-ddf6621f53db admin admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:03.369591 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990d0d49-1aec-4246-96fb-ddf6621f53db admin admin] PATH_INFO: `/v3/roles` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:03.376702 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-990d0d49-1aec-4246-96fb-ddf6621f53db admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:49:03.379908 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-990d0d49-1aec-4246-96fb-ddf6621f53db admin admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:49:03.395184 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 1028/2056] 10.222.0.13 () {60 vars in 1153 bytes} [Wed Aug 31 09:49:03 2022] POST /identity/v3/projects => generated 385 bytes in 250 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 09:49:03.396649 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f290488d-7366-4c94-8d6a-e9fbc8471c99 None None] REQUEST_METHOD: `POST` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:03.397007 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f290488d-7366-4c94-8d6a-e9fbc8471c99 None None] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:03.397352 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f290488d-7366-4c94-8d6a-e9fbc8471c99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:03.410912 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 1029/2057] 10.222.0.13 () {58 vars in 1122 bytes} [Wed Aug 31 09:49:03 2022] GET /identity/v3/roles => generated 1420 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 09:49:03.412199 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-990d0d49-1aec-4246-96fb-ddf6621f53db admin admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:49:03.418685 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.fernet_utils [None req-990d0d49-1aec-4246-96fb-ddf6621f53db admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88114) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:49:03.427464 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.auth.core [None req-f290488d-7366-4c94-8d6a-e9fbc8471c99 None None] MFA Rules not processed for user `fd836cf590a145df9acf19cf3ae2ac56`. Rule list: `[]` (Enabled: `True`). {{(pid=88113) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:49:03.428774 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.fernet_utils [None req-f290488d-7366-4c94-8d6a-e9fbc8471c99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:49:03.437727 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-990d0d49-1aec-4246-96fb-ddf6621f53db admin admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:49:03.539292 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 1029/2058] 10.222.0.13 () {58 vars in 958 bytes} [Wed Aug 31 09:49:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:49:03.540642 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f290488d-7366-4c94-8d6a-e9fbc8471c99 None None] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:49:03.540913 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ebed16-680f-43b2-8472-b870751742a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:49:03.541731 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ebed16-680f-43b2-8472-b870751742a0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:03.542074 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ebed16-680f-43b2-8472-b870751742a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:03.542408 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ebed16-680f-43b2-8472-b870751742a0 admin admin] PATH_INFO: `/v3/projects/1c9972e700f4459dba1afc3060ca9a39` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:03.544688 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35ebed16-680f-43b2-8472-b870751742a0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1c9972e700f4459dba1afc3060ca9a39)` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:49:03.546941 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35ebed16-680f-43b2-8472-b870751742a0 admin admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:49:03.563414 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f290488d-7366-4c94-8d6a-e9fbc8471c99 None None] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:49:03.574189 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48033bd1-3b4c-42c3-b39b-4b8d81b14844 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:49:03.575360 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48033bd1-3b4c-42c3-b39b-4b8d81b14844 admin admin] REQUEST_METHOD: `POST` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:03.576642 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48033bd1-3b4c-42c3-b39b-4b8d81b14844 admin admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:03.577227 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48033bd1-3b4c-42c3-b39b-4b8d81b14844 admin admin] PATH_INFO: `/v3/users` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:03.579058 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48033bd1-3b4c-42c3-b39b-4b8d81b14844 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:49:03.584176 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48033bd1-3b4c-42c3-b39b-4b8d81b14844 admin admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:49:03.632149 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.notifications [None req-35ebed16-680f-43b2-8472-b870751742a0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1c9972e700f4459dba1afc3060ca9a39', 'request_id': 'req-35ebed16-680f-43b2-8472-b870751742a0'} {{(pid=88114) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:49:03.633240 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 1030/2059] 10.222.0.13 () {60 vars in 1144 bytes} [Wed Aug 31 09:49:03 2022] POST /identity/v3/users => generated 299 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 09:49:03.638873 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48033bd1-3b4c-42c3-b39b-4b8d81b14844 admin admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:49:03.646928 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.notifications [None req-35ebed16-680f-43b2-8472-b870751742a0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1c9972e700f4459dba1afc3060ca9a39', 'request_id': 'req-35ebed16-680f-43b2-8472-b870751742a0'} {{(pid=88114) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:49:03.656656 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48033bd1-3b4c-42c3-b39b-4b8d81b14844 admin admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:49:03.659792 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.notifications [None req-35ebed16-680f-43b2-8472-b870751742a0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1c9972e700f4459dba1afc3060ca9a39', 'request_id': 'req-35ebed16-680f-43b2-8472-b870751742a0'} {{(pid=88114) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:49:03.682220 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.notifications [None req-35ebed16-680f-43b2-8472-b870751742a0 admin admin] The token cache is being invalidate because project 1c9972e700f4459dba1afc3060ca9a39 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88114) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 09:49:03.682618 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.notifications [None req-35ebed16-680f-43b2-8472-b870751742a0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88114) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:49:03.683555 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.notifications [None req-35ebed16-680f-43b2-8472-b870751742a0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88114) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:49:03.686440 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 1030/2060] 10.222.0.13 () {58 vars in 1233 bytes} [Wed Aug 31 09:49:03 2022] DELETE /identity/v3/projects/1c9972e700f4459dba1afc3060ca9a39 => generated 0 bytes in 275 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:49:03.687834 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ebed16-680f-43b2-8472-b870751742a0 admin admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:49:03.690409 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.fernet_utils [None req-35ebed16-680f-43b2-8472-b870751742a0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88114) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:49:03.761210 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:49:03.767547 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:49:03.781316 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1add5ff-2585-455f-82f8-46b7124fcffa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:49:03.782005 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1add5ff-2585-455f-82f8-46b7124fcffa admin admin] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:03.782476 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1add5ff-2585-455f-82f8-46b7124fcffa admin admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:03.782904 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1add5ff-2585-455f-82f8-46b7124fcffa admin admin] PATH_INFO: `/v3/roles` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:03.783436 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1add5ff-2585-455f-82f8-46b7124fcffa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:49:03.786269 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1add5ff-2585-455f-82f8-46b7124fcffa admin admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:49:03.799128 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 1031/2061] 10.222.0.13 () {58 vars in 1122 bytes} [Wed Aug 31 09:49:03 2022] GET /identity/v3/roles => generated 1420 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 09:49:03.799996 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1add5ff-2585-455f-82f8-46b7124fcffa admin admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:49:03.802121 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.fernet_utils [None req-e1add5ff-2585-455f-82f8-46b7124fcffa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:49:03.805358 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ebed16-680f-43b2-8472-b870751742a0 admin admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:49:03.810609 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c68fb454-af1a-4e29-9107-0c6bfdba1c5e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9bdaacd597442c2bcde2c555f2cc140', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '49548eb44d62415582af820ab66decee', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:49:03.811388 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68fb454-af1a-4e29-9107-0c6bfdba1c5e service neutron] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:03.811816 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68fb454-af1a-4e29-9107-0c6bfdba1c5e service neutron] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:03.812280 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68fb454-af1a-4e29-9107-0c6bfdba1c5e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:03.815012 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.fernet_utils [None req-c68fb454-af1a-4e29-9107-0c6bfdba1c5e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88114) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:49:03.817010 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1add5ff-2585-455f-82f8-46b7124fcffa admin admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:49:03.822257 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d2a0cd7-5f94-43da-aafc-f1c8d2683d0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:49:03.822894 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2a0cd7-5f94-43da-aafc-f1c8d2683d0d admin admin] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:03.823176 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2a0cd7-5f94-43da-aafc-f1c8d2683d0d admin admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:03.823444 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2a0cd7-5f94-43da-aafc-f1c8d2683d0d admin admin] PATH_INFO: `/v3/roles` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:03.824111 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d2a0cd7-5f94-43da-aafc-f1c8d2683d0d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:49:03.825823 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d2a0cd7-5f94-43da-aafc-f1c8d2683d0d admin admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:49:03.836115 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c68fb454-af1a-4e29-9107-0c6bfdba1c5e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:49:03.838043 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c68fb454-af1a-4e29-9107-0c6bfdba1c5e service neutron] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:49:03.844890 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 1032/2062] 10.222.0.13 () {58 vars in 1122 bytes} [Wed Aug 31 09:49:03 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 09:49:03.844890 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d2a0cd7-5f94-43da-aafc-f1c8d2683d0d admin admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:49:03.846917 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.fernet_utils [None req-1d2a0cd7-5f94-43da-aafc-f1c8d2683d0d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:49:03.855312 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 1031/2063] 10.222.0.13 () {60 vars in 1365 bytes} [Wed Aug 31 09:49:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:49:03.858296 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c68fb454-af1a-4e29-9107-0c6bfdba1c5e service neutron] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:49:03.860724 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.fernet_utils [None req-c68fb454-af1a-4e29-9107-0c6bfdba1c5e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88114) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:49:03.869992 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d2a0cd7-5f94-43da-aafc-f1c8d2683d0d admin admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:49:03.877024 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-983e01e0-82b3-44b0-8501-ab8464987461 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:49:03.877883 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983e01e0-82b3-44b0-8501-ab8464987461 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:03.878361 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983e01e0-82b3-44b0-8501-ab8464987461 admin admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:03.878807 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983e01e0-82b3-44b0-8501-ab8464987461 admin admin] PATH_INFO: `/v3/users/d95b5de8d1f4471ab8bd563150447b95` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:03.880143 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-983e01e0-82b3-44b0-8501-ab8464987461 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d95b5de8d1f4471ab8bd563150447b95)` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:49:03.882188 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-983e01e0-82b3-44b0-8501-ab8464987461 admin admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:49:03.882691 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c68fb454-af1a-4e29-9107-0c6bfdba1c5e service neutron] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:49:03.894132 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c84a2610-1ef6-4394-a3dc-f3eb2fc99e05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:49:03.895699 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84a2610-1ef6-4394-a3dc-f3eb2fc99e05 admin admin] REQUEST_METHOD: `PUT` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:03.897601 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84a2610-1ef6-4394-a3dc-f3eb2fc99e05 admin admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:03.897601 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84a2610-1ef6-4394-a3dc-f3eb2fc99e05 admin admin] PATH_INFO: `/v3/projects/1bb839c560c64bed81e75e12bec0d7aa/users/1e9acc8fe90c4c7ea63f86dfd468809c/roles/6c99f5d9f1f243eaa134d8540f21095d` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:03.932785 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c84a2610-1ef6-4394-a3dc-f3eb2fc99e05 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1bb839c560c64bed81e75e12bec0d7aa, user_id=1e9acc8fe90c4c7ea63f86dfd468809c, role_id=6c99f5d9f1f243eaa134d8540f21095d)` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:49:03.936662 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c84a2610-1ef6-4394-a3dc-f3eb2fc99e05 admin admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:49:03.949949 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.notifications [None req-983e01e0-82b3-44b0-8501-ab8464987461 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd95b5de8d1f4471ab8bd563150447b95'} {{(pid=88113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:49:03.974593 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.notifications [None req-983e01e0-82b3-44b0-8501-ab8464987461 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd95b5de8d1f4471ab8bd563150447b95'} {{(pid=88113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:49:03.978386 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 1032/2064] 10.222.0.13 () {60 vars in 1483 bytes} [Wed Aug 31 09:49:03 2022] PUT /identity/v3/projects/1bb839c560c64bed81e75e12bec0d7aa/users/1e9acc8fe90c4c7ea63f86dfd468809c/roles/6c99f5d9f1f243eaa134d8540f21095d => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:49:03.979563 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c84a2610-1ef6-4394-a3dc-f3eb2fc99e05 admin admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:49:04.015370 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.notifications [None req-983e01e0-82b3-44b0-8501-ab8464987461 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd95b5de8d1f4471ab8bd563150447b95'} {{(pid=88113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:49:04.016798 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.notifications [None req-983e01e0-82b3-44b0-8501-ab8464987461 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd95b5de8d1f4471ab8bd563150447b95'} {{(pid=88113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:49:04.028501 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.notifications [None req-983e01e0-82b3-44b0-8501-ab8464987461 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd95b5de8d1f4471ab8bd563150447b95'} {{(pid=88113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:49:04.040986 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 1033/2065] 10.222.0.13 () {58 vars in 1224 bytes} [Wed Aug 31 09:49:03 2022] DELETE /identity/v3/users/d95b5de8d1f4471ab8bd563150447b95 => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:49:04.042719 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e70cf3-23f1-4346-ae89-5d411f65ae8d None None] REQUEST_METHOD: `POST` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:04.044029 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e70cf3-23f1-4346-ae89-5d411f65ae8d None None] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:04.044029 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e70cf3-23f1-4346-ae89-5d411f65ae8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:04.080865 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.auth.core [None req-f6e70cf3-23f1-4346-ae89-5d411f65ae8d None None] MFA Rules not processed for user `1e9acc8fe90c4c7ea63f86dfd468809c`. Rule list: `[]` (Enabled: `True`). {{(pid=88113) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:49:04.082560 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.fernet_utils [None req-f6e70cf3-23f1-4346-ae89-5d411f65ae8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:49:04.119620 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c84a2610-1ef6-4394-a3dc-f3eb2fc99e05 admin admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:49:04.187689 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 1034/2066] 10.222.0.13 () {58 vars in 958 bytes} [Wed Aug 31 09:49:04 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:49:04.193371 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e70cf3-23f1-4346-ae89-5d411f65ae8d None None] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:49:04.199475 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.fernet_utils [None req-f6e70cf3-23f1-4346-ae89-5d411f65ae8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:49:04.271365 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d741e62-032a-4df2-837b-1d5f1342df63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:49:04.272208 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d741e62-032a-4df2-837b-1d5f1342df63 admin admin] REQUEST_METHOD: `PUT` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:04.272570 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d741e62-032a-4df2-837b-1d5f1342df63 admin admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:04.274031 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d741e62-032a-4df2-837b-1d5f1342df63 admin admin] PATH_INFO: `/v3/projects/49b47df28ff94283948257611a2fe330/users/712beab5eea84a2a815f5bc4914c8558/roles/5d4ee63a3add47fc9a6b8b16d39e0a1f` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:04.332417 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d741e62-032a-4df2-837b-1d5f1342df63 admin admin] RBAC: Authorizing `identity:create_grant(project_id=49b47df28ff94283948257611a2fe330, user_id=712beab5eea84a2a815f5bc4914c8558, role_id=5d4ee63a3add47fc9a6b8b16d39e0a1f)` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:49:04.343825 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d741e62-032a-4df2-837b-1d5f1342df63 admin admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:49:04.363888 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:49:04.407391 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 1033/2067] 10.222.0.13 () {60 vars in 1483 bytes} [Wed Aug 31 09:49:03 2022] PUT /identity/v3/projects/49b47df28ff94283948257611a2fe330/users/712beab5eea84a2a815f5bc4914c8558/roles/5d4ee63a3add47fc9a6b8b16d39e0a1f => generated 0 bytes in 426 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:49:04.410753 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d741e62-032a-4df2-837b-1d5f1342df63 admin admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:49:04.414383 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.fernet_utils [None req-5d741e62-032a-4df2-837b-1d5f1342df63 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88114) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:49:04.463200 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e70cf3-23f1-4346-ae89-5d411f65ae8d None None] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:49:04.571266 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:49:04.579276 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f15b119-181f-48b8-81c4-91c0ef7a3a37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:49:04.580463 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f15b119-181f-48b8-81c4-91c0ef7a3a37 admin admin] REQUEST_METHOD: `POST` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:04.581582 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f15b119-181f-48b8-81c4-91c0ef7a3a37 admin admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:04.582057 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f15b119-181f-48b8-81c4-91c0ef7a3a37 admin admin] PATH_INFO: `/v3/projects` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:04.583261 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f15b119-181f-48b8-81c4-91c0ef7a3a37 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:49:04.587197 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f15b119-181f-48b8-81c4-91c0ef7a3a37 admin admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:49:04.651276 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 1035/2068] 10.222.0.13 () {60 vars in 1153 bytes} [Wed Aug 31 09:49:04 2022] POST /identity/v3/projects => generated 365 bytes in 459 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 09:49:04.653407 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d741e62-032a-4df2-837b-1d5f1342df63 admin admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:49:04.656699 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f15b119-181f-48b8-81c4-91c0ef7a3a37 admin admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:49:04.780471 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f653316-1a83-4bee-93df-7f9a81e71d9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:49:04.781964 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f653316-1a83-4bee-93df-7f9a81e71d9a admin admin] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:04.782678 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f653316-1a83-4bee-93df-7f9a81e71d9a admin admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:04.783169 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f653316-1a83-4bee-93df-7f9a81e71d9a admin admin] PATH_INFO: `/v3/roles` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:04.786080 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f653316-1a83-4bee-93df-7f9a81e71d9a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:49:04.788688 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f653316-1a83-4bee-93df-7f9a81e71d9a admin admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:49:04.808183 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 1034/2069] 10.222.0.13 () {58 vars in 1122 bytes} [Wed Aug 31 09:49:04 2022] GET /identity/v3/roles => generated 1420 bytes in 400 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 09:49:04.809524 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f653316-1a83-4bee-93df-7f9a81e71d9a admin admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:49:04.829417 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f15b119-181f-48b8-81c4-91c0ef7a3a37 admin admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:49:04.836641 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f653316-1a83-4bee-93df-7f9a81e71d9a admin admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:49:04.843512 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec23265-1a62-4d1b-a464-2f1a36978aa3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:49:04.844776 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec23265-1a62-4d1b-a464-2f1a36978aa3 admin admin] REQUEST_METHOD: `PUT` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:04.845294 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec23265-1a62-4d1b-a464-2f1a36978aa3 admin admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:04.846024 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec23265-1a62-4d1b-a464-2f1a36978aa3 admin admin] PATH_INFO: `/v3/projects/49b47df28ff94283948257611a2fe330/users/712beab5eea84a2a815f5bc4914c8558/roles/5d4ee63a3add47fc9a6b8b16d39e0a1f` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:04.851654 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ec23265-1a62-4d1b-a464-2f1a36978aa3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=49b47df28ff94283948257611a2fe330, user_id=712beab5eea84a2a815f5bc4914c8558, role_id=5d4ee63a3add47fc9a6b8b16d39e0a1f)` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:49:04.855511 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ec23265-1a62-4d1b-a464-2f1a36978aa3 admin admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:49:04.857868 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a7bf9ec-6a71-4940-8f12-2d9534d4a6ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:49:04.859750 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7bf9ec-6a71-4940-8f12-2d9534d4a6ff admin admin] REQUEST_METHOD: `POST` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:04.861365 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7bf9ec-6a71-4940-8f12-2d9534d4a6ff admin admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:04.862830 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7bf9ec-6a71-4940-8f12-2d9534d4a6ff admin admin] PATH_INFO: `/v3/users` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:04.864800 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a7bf9ec-6a71-4940-8f12-2d9534d4a6ff admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:49:04.870705 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a7bf9ec-6a71-4940-8f12-2d9534d4a6ff admin admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:49:04.884481 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 1035/2070] 10.222.0.13 () {60 vars in 1483 bytes} [Wed Aug 31 09:49:04 2022] PUT /identity/v3/projects/49b47df28ff94283948257611a2fe330/users/712beab5eea84a2a815f5bc4914c8558/roles/5d4ee63a3add47fc9a6b8b16d39e0a1f => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:49:04.891376 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff8da1d-b150-4e7b-a2c5-531e1be731c9 None None] REQUEST_METHOD: `POST` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:04.891923 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff8da1d-b150-4e7b-a2c5-531e1be731c9 None None] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:04.892411 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff8da1d-b150-4e7b-a2c5-531e1be731c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:04.926215 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 1036/2071] 10.222.0.13 () {60 vars in 1144 bytes} [Wed Aug 31 09:49:04 2022] POST /identity/v3/users => generated 289 bytes in 270 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 09:49:04.930385 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a7bf9ec-6a71-4940-8f12-2d9534d4a6ff admin admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:49:04.954803 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a7bf9ec-6a71-4940-8f12-2d9534d4a6ff admin admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:49:04.963934 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.auth.core [None req-3ff8da1d-b150-4e7b-a2c5-531e1be731c9 None None] MFA Rules not processed for user `712beab5eea84a2a815f5bc4914c8558`. Rule list: `[]` (Enabled: `True`). {{(pid=88114) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:49:04.966139 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.fernet_utils [None req-3ff8da1d-b150-4e7b-a2c5-531e1be731c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88114) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:49:05.079214 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d8ca0f-de4d-42fe-a041-9a9b8b443e29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:49:05.079997 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d8ca0f-de4d-42fe-a041-9a9b8b443e29 admin admin] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:05.080186 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d8ca0f-de4d-42fe-a041-9a9b8b443e29 admin admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:05.080464 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d8ca0f-de4d-42fe-a041-9a9b8b443e29 admin admin] PATH_INFO: `/v3/roles` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:05.081186 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06d8ca0f-de4d-42fe-a041-9a9b8b443e29 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:49:05.082951 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06d8ca0f-de4d-42fe-a041-9a9b8b443e29 admin admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:49:05.084720 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:49:05.108077 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 1036/2072] 10.222.0.13 () {58 vars in 958 bytes} [Wed Aug 31 09:49:04 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 218 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:49:05.109053 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff8da1d-b150-4e7b-a2c5-531e1be731c9 None None] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:49:05.110511 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.fernet_utils [None req-3ff8da1d-b150-4e7b-a2c5-531e1be731c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88114) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:49:05.121605 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 1037/2073] 10.222.0.13 () {58 vars in 1122 bytes} [Wed Aug 31 09:49:04 2022] GET /identity/v3/roles => generated 1420 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 09:49:05.126022 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d8ca0f-de4d-42fe-a041-9a9b8b443e29 admin admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:49:05.129329 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.fernet_utils [None req-06d8ca0f-de4d-42fe-a041-9a9b8b443e29 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:49:05.130145 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff8da1d-b150-4e7b-a2c5-531e1be731c9 None None] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:49:05.135583 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb479b2-4735-44db-97f1-7884d2159ea3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:49:05.136283 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb479b2-4735-44db-97f1-7884d2159ea3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:05.136637 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb479b2-4735-44db-97f1-7884d2159ea3 admin admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:05.136981 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb479b2-4735-44db-97f1-7884d2159ea3 admin admin] PATH_INFO: `/v3/projects/79bb1d5c92494216a2805a642f4e8084` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:05.138617 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbb479b2-4735-44db-97f1-7884d2159ea3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=79bb1d5c92494216a2805a642f4e8084)` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:49:05.140863 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbb479b2-4735-44db-97f1-7884d2159ea3 admin admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:49:05.146972 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d8ca0f-de4d-42fe-a041-9a9b8b443e29 admin admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:49:05.155581 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa3f3b1b-60fb-41d5-8981-0a970005dd49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:49:05.156337 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3f3b1b-60fb-41d5-8981-0a970005dd49 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:05.156946 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3f3b1b-60fb-41d5-8981-0a970005dd49 admin admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:05.157308 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3f3b1b-60fb-41d5-8981-0a970005dd49 admin admin] PATH_INFO: `/v3/projects/6ddbec14e9e54228acbf4d80ea432639` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:05.158462 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa3f3b1b-60fb-41d5-8981-0a970005dd49 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6ddbec14e9e54228acbf4d80ea432639)` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:49:05.161070 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa3f3b1b-60fb-41d5-8981-0a970005dd49 admin admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:49:05.196410 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.notifications [None req-dbb479b2-4735-44db-97f1-7884d2159ea3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '79bb1d5c92494216a2805a642f4e8084', 'request_id': 'req-dbb479b2-4735-44db-97f1-7884d2159ea3'} {{(pid=88114) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:49:05.204983 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.notifications [None req-dbb479b2-4735-44db-97f1-7884d2159ea3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '79bb1d5c92494216a2805a642f4e8084', 'request_id': 'req-dbb479b2-4735-44db-97f1-7884d2159ea3'} {{(pid=88114) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:49:05.208417 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=94,keystone:INSERT=1,keystone:DELETE=5 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:49:05.216411 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.notifications [None req-dbb479b2-4735-44db-97f1-7884d2159ea3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '79bb1d5c92494216a2805a642f4e8084', 'request_id': 'req-dbb479b2-4735-44db-97f1-7884d2159ea3'} {{(pid=88114) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:49:05.223796 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.notifications [None req-fa3f3b1b-60fb-41d5-8981-0a970005dd49 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6ddbec14e9e54228acbf4d80ea432639', 'request_id': 'req-fa3f3b1b-60fb-41d5-8981-0a970005dd49'} {{(pid=88113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:49:05.231704 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.notifications [None req-fa3f3b1b-60fb-41d5-8981-0a970005dd49 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6ddbec14e9e54228acbf4d80ea432639', 'request_id': 'req-fa3f3b1b-60fb-41d5-8981-0a970005dd49'} {{(pid=88113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:49:05.233012 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.notifications [None req-dbb479b2-4735-44db-97f1-7884d2159ea3 admin admin] The token cache is being invalidate because project 79bb1d5c92494216a2805a642f4e8084 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88114) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 09:49:05.233483 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.notifications [None req-dbb479b2-4735-44db-97f1-7884d2159ea3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88114) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:49:05.234383 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.notifications [None req-dbb479b2-4735-44db-97f1-7884d2159ea3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88114) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:49:05.236030 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 1037/2074] 10.222.0.13 () {58 vars in 1233 bytes} [Wed Aug 31 09:49:05 2022] DELETE /identity/v3/projects/79bb1d5c92494216a2805a642f4e8084 => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:49:05.237057 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb479b2-4735-44db-97f1-7884d2159ea3 admin admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:49:05.238662 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.fernet_utils [None req-dbb479b2-4735-44db-97f1-7884d2159ea3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88114) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:49:05.245183 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.notifications [None req-fa3f3b1b-60fb-41d5-8981-0a970005dd49 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6ddbec14e9e54228acbf4d80ea432639', 'request_id': 'req-fa3f3b1b-60fb-41d5-8981-0a970005dd49'} {{(pid=88113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:49:05.259219 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.notifications [None req-fa3f3b1b-60fb-41d5-8981-0a970005dd49 admin admin] The token cache is being invalidate because project 6ddbec14e9e54228acbf4d80ea432639 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88113) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 09:49:05.259601 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.notifications [None req-fa3f3b1b-60fb-41d5-8981-0a970005dd49 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:49:05.260384 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.notifications [None req-fa3f3b1b-60fb-41d5-8981-0a970005dd49 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:49:05.262812 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 1038/2075] 10.222.0.13 () {58 vars in 1233 bytes} [Wed Aug 31 09:49:05 2022] DELETE /identity/v3/projects/6ddbec14e9e54228acbf4d80ea432639 => generated 0 bytes in 137 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:49:05.264212 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa3f3b1b-60fb-41d5-8981-0a970005dd49 admin admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:49:05.267277 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.fernet_utils [None req-fa3f3b1b-60fb-41d5-8981-0a970005dd49 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:49:05.338946 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb479b2-4735-44db-97f1-7884d2159ea3 admin admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:49:05.347922 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa3f3b1b-60fb-41d5-8981-0a970005dd49 admin admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:49:05.354163 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc21f5c-c100-459b-849e-25cedf0a84b2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d52b69c8e94441dacfcff944326f169', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '49548eb44d62415582af820ab66decee', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:49:05.354900 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc21f5c-c100-459b-849e-25cedf0a84b2 service cinder] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:05.355185 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc21f5c-c100-459b-849e-25cedf0a84b2 service cinder] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:05.355607 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc21f5c-c100-459b-849e-25cedf0a84b2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:05.359830 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.fernet_utils [None req-fbc21f5c-c100-459b-849e-25cedf0a84b2 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:49:05.410985 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef83127-976f-4a8a-9be9-6d944dfb0eb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:49:05.411814 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef83127-976f-4a8a-9be9-6d944dfb0eb2 admin admin] REQUEST_METHOD: `PUT` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:05.412149 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef83127-976f-4a8a-9be9-6d944dfb0eb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:05.412475 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef83127-976f-4a8a-9be9-6d944dfb0eb2 admin admin] PATH_INFO: `/v3/projects/3d9546d7f106424f96461fc562e94a61/users/69878209c5e94774a21296a004dea3a7/roles/5d4ee63a3add47fc9a6b8b16d39e0a1f` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:05.431157 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ef83127-976f-4a8a-9be9-6d944dfb0eb2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3d9546d7f106424f96461fc562e94a61, user_id=69878209c5e94774a21296a004dea3a7, role_id=5d4ee63a3add47fc9a6b8b16d39e0a1f)` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:49:05.434607 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ef83127-976f-4a8a-9be9-6d944dfb0eb2 admin admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:49:05.450404 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbc21f5c-c100-459b-849e-25cedf0a84b2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:49:05.452649 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 1038/2076] 10.222.0.13 () {60 vars in 1483 bytes} [Wed Aug 31 09:49:05 2022] PUT /identity/v3/projects/3d9546d7f106424f96461fc562e94a61/users/69878209c5e94774a21296a004dea3a7/roles/5d4ee63a3add47fc9a6b8b16d39e0a1f => generated 0 bytes in 216 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:49:05.453177 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbc21f5c-c100-459b-849e-25cedf0a84b2 service cinder] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:49:05.453628 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef83127-976f-4a8a-9be9-6d944dfb0eb2 admin admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:49:05.459022 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.fernet_utils [None req-1ef83127-976f-4a8a-9be9-6d944dfb0eb2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88114) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:49:05.473823 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 1039/2077] 10.222.0.13 () {60 vars in 1365 bytes} [Wed Aug 31 09:49:05 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 210 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:49:05.474748 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc21f5c-c100-459b-849e-25cedf0a84b2 service cinder] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:49:05.476266 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.fernet_utils [None req-fbc21f5c-c100-459b-849e-25cedf0a84b2 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:49:05.510005 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc21f5c-c100-459b-849e-25cedf0a84b2 service cinder] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:49:05.564238 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef83127-976f-4a8a-9be9-6d944dfb0eb2 admin admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:49:05.574771 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a2e4115-dc2c-4ee8-a0e6-a6192d241c92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:49:05.575886 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2e4115-dc2c-4ee8-a0e6-a6192d241c92 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:05.576385 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2e4115-dc2c-4ee8-a0e6-a6192d241c92 admin admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:05.576868 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2e4115-dc2c-4ee8-a0e6-a6192d241c92 admin admin] PATH_INFO: `/v3/users/9c401cde27794c009a6e13250daf83dc` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:05.578769 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a2e4115-dc2c-4ee8-a0e6-a6192d241c92 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9c401cde27794c009a6e13250daf83dc)` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:49:05.581881 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a2e4115-dc2c-4ee8-a0e6-a6192d241c92 admin admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:49:05.584221 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:49:05.616146 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=95,keystone:DELETE=3,keystone:INSERT=2 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:49:05.653824 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e4d8eaf-8dd2-4757-8ca7-e3653506deae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:49:05.655378 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4d8eaf-8dd2-4757-8ca7-e3653506deae admin admin] REQUEST_METHOD: `DELETE` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:05.656282 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4d8eaf-8dd2-4757-8ca7-e3653506deae admin admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:05.656780 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4d8eaf-8dd2-4757-8ca7-e3653506deae admin admin] PATH_INFO: `/v3/users/446307197bda44649c566c16a013de27` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:05.659091 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e4d8eaf-8dd2-4757-8ca7-e3653506deae admin admin] RBAC: Authorizing `identity:delete_user(user_id=446307197bda44649c566c16a013de27)` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:49:05.663004 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e4d8eaf-8dd2-4757-8ca7-e3653506deae admin admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:49:05.673989 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.notifications [None req-2a2e4115-dc2c-4ee8-a0e6-a6192d241c92 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9c401cde27794c009a6e13250daf83dc'} {{(pid=88114) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:49:05.685855 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.notifications [None req-2a2e4115-dc2c-4ee8-a0e6-a6192d241c92 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9c401cde27794c009a6e13250daf83dc'} {{(pid=88114) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:49:05.709005 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.notifications [None req-2a2e4115-dc2c-4ee8-a0e6-a6192d241c92 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9c401cde27794c009a6e13250daf83dc'} {{(pid=88114) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:49:05.710195 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.notifications [None req-2a2e4115-dc2c-4ee8-a0e6-a6192d241c92 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9c401cde27794c009a6e13250daf83dc'} {{(pid=88114) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:49:05.728790 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.notifications [None req-2a2e4115-dc2c-4ee8-a0e6-a6192d241c92 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9c401cde27794c009a6e13250daf83dc'} {{(pid=88114) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:49:05.752207 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 1039/2078] 10.222.0.13 () {58 vars in 1224 bytes} [Wed Aug 31 09:49:05 2022] DELETE /identity/v3/users/9c401cde27794c009a6e13250daf83dc => generated 0 bytes in 299 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:49:05.754041 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a2e4115-dc2c-4ee8-a0e6-a6192d241c92 admin admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:49:05.761040 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.notifications [None req-0e4d8eaf-8dd2-4757-8ca7-e3653506deae admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '446307197bda44649c566c16a013de27'} {{(pid=88113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:49:05.762737 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.fernet_utils [None req-2a2e4115-dc2c-4ee8-a0e6-a6192d241c92 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88114) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:49:05.790884 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.notifications [None req-0e4d8eaf-8dd2-4757-8ca7-e3653506deae admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '446307197bda44649c566c16a013de27'} {{(pid=88113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:49:05.841985 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.notifications [None req-0e4d8eaf-8dd2-4757-8ca7-e3653506deae admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '446307197bda44649c566c16a013de27'} {{(pid=88113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:49:05.844456 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.notifications [None req-0e4d8eaf-8dd2-4757-8ca7-e3653506deae admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '446307197bda44649c566c16a013de27'} {{(pid=88113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:49:05.863926 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.notifications [None req-0e4d8eaf-8dd2-4757-8ca7-e3653506deae admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '446307197bda44649c566c16a013de27'} {{(pid=88113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:49:05.890290 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 1040/2079] 10.222.0.13 () {58 vars in 1224 bytes} [Wed Aug 31 09:49:05 2022] DELETE /identity/v3/users/446307197bda44649c566c16a013de27 => generated 0 bytes in 413 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:49:05.894208 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a2e4115-dc2c-4ee8-a0e6-a6192d241c92 admin admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:49:05.989326 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ae051f-8f21-42e8-a60d-a97eb6ce054b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:49:05.990050 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ae051f-8f21-42e8-a60d-a97eb6ce054b admin admin] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:05.990416 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ae051f-8f21-42e8-a60d-a97eb6ce054b admin admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:05.990693 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ae051f-8f21-42e8-a60d-a97eb6ce054b admin admin] PATH_INFO: `/v3/roles` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:05.991380 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08ae051f-8f21-42e8-a60d-a97eb6ce054b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:49:05.993057 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08ae051f-8f21-42e8-a60d-a97eb6ce054b admin admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:49:06.007403 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 1040/2080] 10.222.0.13 () {58 vars in 1122 bytes} [Wed Aug 31 09:49:05 2022] GET /identity/v3/roles => generated 1420 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 09:49:06.015242 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e4d8eaf-8dd2-4757-8ca7-e3653506deae admin admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:49:06.021409 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.fernet_utils [None req-0e4d8eaf-8dd2-4757-8ca7-e3653506deae admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:49:06.055637 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e4d8eaf-8dd2-4757-8ca7-e3653506deae admin admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:49:06.070837 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b5241db-9e6c-451e-a533-ce9585984ff9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:49:06.072353 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5241db-9e6c-451e-a533-ce9585984ff9 admin admin] REQUEST_METHOD: `PUT` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:06.073128 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5241db-9e6c-451e-a533-ce9585984ff9 admin admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:06.073895 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5241db-9e6c-451e-a533-ce9585984ff9 admin admin] PATH_INFO: `/v3/projects/3d9546d7f106424f96461fc562e94a61/users/69878209c5e94774a21296a004dea3a7/roles/5d4ee63a3add47fc9a6b8b16d39e0a1f` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:06.080051 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b5241db-9e6c-451e-a533-ce9585984ff9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3d9546d7f106424f96461fc562e94a61, user_id=69878209c5e94774a21296a004dea3a7, role_id=5d4ee63a3add47fc9a6b8b16d39e0a1f)` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:49:06.088315 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b5241db-9e6c-451e-a533-ce9585984ff9 admin admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:49:06.137431 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 1041/2081] 10.222.0.13 () {60 vars in 1483 bytes} [Wed Aug 31 09:49:06 2022] PUT /identity/v3/projects/3d9546d7f106424f96461fc562e94a61/users/69878209c5e94774a21296a004dea3a7/roles/5d4ee63a3add47fc9a6b8b16d39e0a1f => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:49:06.139687 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f0ed2a-3787-491c-9146-159fa6394cd7 None None] REQUEST_METHOD: `POST` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:06.140012 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f0ed2a-3787-491c-9146-159fa6394cd7 None None] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:06.140446 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f0ed2a-3787-491c-9146-159fa6394cd7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:06.181775 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.auth.core [None req-f9f0ed2a-3787-491c-9146-159fa6394cd7 None None] MFA Rules not processed for user `69878209c5e94774a21296a004dea3a7`. Rule list: `[]` (Enabled: `True`). {{(pid=88114) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:49:06.183282 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.fernet_utils [None req-f9f0ed2a-3787-491c-9146-159fa6394cd7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88114) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:49:06.256160 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:DELETE=9,keystone:SELECT=90,keystone:INSERT=1 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:49:06.386761 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 1041/2082] 10.222.0.13 () {58 vars in 958 bytes} [Wed Aug 31 09:49:06 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 248 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:49:06.415359 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b5241db-9e6c-451e-a533-ce9585984ff9 admin admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:49:06.420903 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.fernet_utils [None req-1b5241db-9e6c-451e-a533-ce9585984ff9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:49:06.433830 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f0ed2a-3787-491c-9146-159fa6394cd7 None None] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:49:06.436690 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.fernet_utils [None req-f9f0ed2a-3787-491c-9146-159fa6394cd7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88114) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:49:06.566790 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b5241db-9e6c-451e-a533-ce9585984ff9 admin admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:49:06.571788 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-783fddec-efa9-472c-bb1d-aeff0a89c751 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d52b69c8e94441dacfcff944326f169', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '49548eb44d62415582af820ab66decee', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:49:06.572633 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783fddec-efa9-472c-bb1d-aeff0a89c751 service cinder] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:06.572988 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783fddec-efa9-472c-bb1d-aeff0a89c751 service cinder] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:06.573336 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783fddec-efa9-472c-bb1d-aeff0a89c751 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:06.588812 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-783fddec-efa9-472c-bb1d-aeff0a89c751 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:49:06.590684 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-783fddec-efa9-472c-bb1d-aeff0a89c751 service cinder] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:49:06.606576 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 1042/2083] 10.222.0.13 () {60 vars in 1365 bytes} [Wed Aug 31 09:49:06 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 193 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:49:06.608701 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-783fddec-efa9-472c-bb1d-aeff0a89c751 service cinder] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:49:06.613477 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.fernet_utils [None req-783fddec-efa9-472c-bb1d-aeff0a89c751 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:49:06.632900 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f0ed2a-3787-491c-9146-159fa6394cd7 None None] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:49:06.639766 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-783fddec-efa9-472c-bb1d-aeff0a89c751 service cinder] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:49:06.643817 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c87efa64-f4ff-47f1-9c49-5fc4015cbc30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:49:06.644501 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87efa64-f4ff-47f1-9c49-5fc4015cbc30 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:06.644781 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87efa64-f4ff-47f1-9c49-5fc4015cbc30 admin admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:06.645076 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87efa64-f4ff-47f1-9c49-5fc4015cbc30 admin admin] PATH_INFO: `/v3/projects/6254c9e845bf4d01b15882e06338098b` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:06.646810 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c87efa64-f4ff-47f1-9c49-5fc4015cbc30 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6254c9e845bf4d01b15882e06338098b)` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:49:06.648854 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c87efa64-f4ff-47f1-9c49-5fc4015cbc30 admin admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:49:06.655492 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d36b60da-62bb-4263-b099-afcbaf133488 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:49:06.655999 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36b60da-62bb-4263-b099-afcbaf133488 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:06.656178 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36b60da-62bb-4263-b099-afcbaf133488 admin admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:06.656460 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36b60da-62bb-4263-b099-afcbaf133488 admin admin] PATH_INFO: `/v3/projects/1e294af9d7204453adcd286755f57962` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:06.657801 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d36b60da-62bb-4263-b099-afcbaf133488 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1e294af9d7204453adcd286755f57962)` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:49:06.660036 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d36b60da-62bb-4263-b099-afcbaf133488 admin admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:49:06.722033 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:49:06.764087 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.notifications [None req-d36b60da-62bb-4263-b099-afcbaf133488 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1e294af9d7204453adcd286755f57962', 'request_id': 'req-d36b60da-62bb-4263-b099-afcbaf133488'} {{(pid=88113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:49:06.786103 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.notifications [None req-d36b60da-62bb-4263-b099-afcbaf133488 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1e294af9d7204453adcd286755f57962', 'request_id': 'req-d36b60da-62bb-4263-b099-afcbaf133488'} {{(pid=88113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:49:06.804663 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.notifications [None req-d36b60da-62bb-4263-b099-afcbaf133488 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1e294af9d7204453adcd286755f57962', 'request_id': 'req-d36b60da-62bb-4263-b099-afcbaf133488'} {{(pid=88113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:49:06.817176 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.notifications [None req-c87efa64-f4ff-47f1-9c49-5fc4015cbc30 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6254c9e845bf4d01b15882e06338098b', 'request_id': 'req-c87efa64-f4ff-47f1-9c49-5fc4015cbc30'} {{(pid=88114) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:49:06.828584 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.notifications [None req-d36b60da-62bb-4263-b099-afcbaf133488 admin admin] The token cache is being invalidate because project 1e294af9d7204453adcd286755f57962 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88113) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 09:49:06.829813 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.notifications [None req-d36b60da-62bb-4263-b099-afcbaf133488 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:49:06.831717 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.notifications [None req-d36b60da-62bb-4263-b099-afcbaf133488 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:49:06.836510 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.notifications [None req-c87efa64-f4ff-47f1-9c49-5fc4015cbc30 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6254c9e845bf4d01b15882e06338098b', 'request_id': 'req-c87efa64-f4ff-47f1-9c49-5fc4015cbc30'} {{(pid=88114) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:49:06.843014 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 1043/2084] 10.222.0.13 () {58 vars in 1233 bytes} [Wed Aug 31 09:49:06 2022] DELETE /identity/v3/projects/1e294af9d7204453adcd286755f57962 => generated 0 bytes in 234 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:49:06.843444 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d36b60da-62bb-4263-b099-afcbaf133488 admin admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:49:06.851536 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.fernet_utils [None req-d36b60da-62bb-4263-b099-afcbaf133488 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:49:06.860286 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.notifications [None req-c87efa64-f4ff-47f1-9c49-5fc4015cbc30 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6254c9e845bf4d01b15882e06338098b', 'request_id': 'req-c87efa64-f4ff-47f1-9c49-5fc4015cbc30'} {{(pid=88114) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:49:06.886733 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.notifications [None req-c87efa64-f4ff-47f1-9c49-5fc4015cbc30 admin admin] The token cache is being invalidate because project 6254c9e845bf4d01b15882e06338098b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88114) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 09:49:06.888226 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.notifications [None req-c87efa64-f4ff-47f1-9c49-5fc4015cbc30 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88114) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:49:06.889564 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.notifications [None req-c87efa64-f4ff-47f1-9c49-5fc4015cbc30 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88114) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:49:06.892705 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 1042/2085] 10.222.0.13 () {58 vars in 1233 bytes} [Wed Aug 31 09:49:06 2022] DELETE /identity/v3/projects/6254c9e845bf4d01b15882e06338098b => generated 0 bytes in 460 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:49:06.894600 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c87efa64-f4ff-47f1-9c49-5fc4015cbc30 admin admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:49:06.898933 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.fernet_utils [None req-c87efa64-f4ff-47f1-9c49-5fc4015cbc30 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88114) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:49:06.991213 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d36b60da-62bb-4263-b099-afcbaf133488 admin admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:49:07.023065 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c87efa64-f4ff-47f1-9c49-5fc4015cbc30 admin admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:49:07.028083 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e90237-ebfc-4321-8f3d-00d83619ebd6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d52b69c8e94441dacfcff944326f169', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '49548eb44d62415582af820ab66decee', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:49:07.028825 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e90237-ebfc-4321-8f3d-00d83619ebd6 service cinder] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:07.030262 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e90237-ebfc-4321-8f3d-00d83619ebd6 service cinder] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:07.030262 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e90237-ebfc-4321-8f3d-00d83619ebd6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:07.032617 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.fernet_utils [None req-d4e90237-ebfc-4321-8f3d-00d83619ebd6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88114) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:49:07.040548 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:49:07.130156 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1450ab5-7002-49ad-9a5b-19b0b00ebbee service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d52b69c8e94441dacfcff944326f169', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '49548eb44d62415582af820ab66decee', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:49:07.131623 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1450ab5-7002-49ad-9a5b-19b0b00ebbee service cinder] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:07.131623 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1450ab5-7002-49ad-9a5b-19b0b00ebbee service cinder] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:07.132860 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1450ab5-7002-49ad-9a5b-19b0b00ebbee service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:07.137188 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.fernet_utils [None req-b1450ab5-7002-49ad-9a5b-19b0b00ebbee service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:49:07.141905 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4e90237-ebfc-4321-8f3d-00d83619ebd6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:49:07.144040 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4e90237-ebfc-4321-8f3d-00d83619ebd6 service cinder] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:49:07.157771 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 1043/2086] 10.222.0.13 () {60 vars in 1365 bytes} [Wed Aug 31 09:49:06 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 263 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:49:07.160321 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14534768-ada9-4578-8285-8a3d58e85808 None None] REQUEST_METHOD: `POST` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:07.160778 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14534768-ada9-4578-8285-8a3d58e85808 None None] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:07.161190 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14534768-ada9-4578-8285-8a3d58e85808 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:07.186842 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.auth.core [None req-14534768-ada9-4578-8285-8a3d58e85808 None None] MFA Rules not processed for user `fd836cf590a145df9acf19cf3ae2ac56`. Rule list: `[]` (Enabled: `True`). {{(pid=88114) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:49:07.188490 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.fernet_utils [None req-14534768-ada9-4578-8285-8a3d58e85808 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88114) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:49:07.250136 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1450ab5-7002-49ad-9a5b-19b0b00ebbee service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:49:07.253127 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1450ab5-7002-49ad-9a5b-19b0b00ebbee service cinder] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:49:07.281880 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 1044/2087] 10.222.0.13 () {60 vars in 1365 bytes} [Wed Aug 31 09:49:06 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 440 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:49:07.286845 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0282bf60-8870-493a-954b-345514f8ba6b None None] REQUEST_METHOD: `POST` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:07.286845 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0282bf60-8870-493a-954b-345514f8ba6b None None] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:07.287530 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0282bf60-8870-493a-954b-345514f8ba6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:07.352241 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.auth.core [None req-0282bf60-8870-493a-954b-345514f8ba6b None None] MFA Rules not processed for user `fd836cf590a145df9acf19cf3ae2ac56`. Rule list: `[]` (Enabled: `True`). {{(pid=88113) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:49:07.356665 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.fernet_utils [None req-0282bf60-8870-493a-954b-345514f8ba6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:49:07.410097 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 1044/2088] 10.222.0.13 () {58 vars in 958 bytes} [Wed Aug 31 09:49:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 252 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:49:07.424769 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1894c63-39e8-4cc2-be2e-8c88809a84f1 None None] REQUEST_METHOD: `POST` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:07.425490 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1894c63-39e8-4cc2-be2e-8c88809a84f1 None None] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:07.426358 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1894c63-39e8-4cc2-be2e-8c88809a84f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:07.449265 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:49:07.503344 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.auth.core [None req-e1894c63-39e8-4cc2-be2e-8c88809a84f1 None None] MFA Rules not processed for user `fd836cf590a145df9acf19cf3ae2ac56`. Rule list: `[]` (Enabled: `True`). {{(pid=88114) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:49:07.506552 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.fernet_utils [None req-e1894c63-39e8-4cc2-be2e-8c88809a84f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88114) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:49:07.538375 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 1045/2089] 10.222.0.13 () {58 vars in 958 bytes} [Wed Aug 31 09:49:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 09:49:07.540195 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 1045/2090] 10.222.0.13 () {58 vars in 958 bytes} [Wed Aug 31 09:49:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 258 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:49:07.551333 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1894c63-39e8-4cc2-be2e-8c88809a84f1 None None] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:49:07.553058 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7be2005-8de9-4517-9463-884c48ae2380 None None] REQUEST_METHOD: `POST` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:07.554132 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7be2005-8de9-4517-9463-884c48ae2380 None None] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:07.554132 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7be2005-8de9-4517-9463-884c48ae2380 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:07.573484 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1894c63-39e8-4cc2-be2e-8c88809a84f1 None None] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:49:07.590072 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-628322db-0f81-488c-89bc-7e6dae48cb78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:49:07.590940 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628322db-0f81-488c-89bc-7e6dae48cb78 admin admin] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:07.591699 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628322db-0f81-488c-89bc-7e6dae48cb78 admin admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:07.593244 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628322db-0f81-488c-89bc-7e6dae48cb78 admin admin] PATH_INFO: `/v3/domains` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:07.594135 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-628322db-0f81-488c-89bc-7e6dae48cb78 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88114) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:49:07.594918 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.auth.core [None req-f7be2005-8de9-4517-9463-884c48ae2380 None None] MFA Rules not processed for user `fd836cf590a145df9acf19cf3ae2ac56`. Rule list: `[]` (Enabled: `True`). {{(pid=88113) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:49:07.595782 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-628322db-0f81-488c-89bc-7e6dae48cb78 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:49:07.596652 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.fernet_utils [None req-f7be2005-8de9-4517-9463-884c48ae2380 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:49:07.599836 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-628322db-0f81-488c-89bc-7e6dae48cb78 admin admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:49:07.612889 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 1046/2091] 10.222.0.13 () {58 vars in 958 bytes} [Wed Aug 31 09:49:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 61 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:49:07.615096 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7be2005-8de9-4517-9463-884c48ae2380 None None] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:49:07.635877 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 1046/2092] 10.222.0.13 () {58 vars in 1153 bytes} [Wed Aug 31 09:49:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:49:07.642864 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-628322db-0f81-488c-89bc-7e6dae48cb78 admin admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:49:07.655276 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-628322db-0f81-488c-89bc-7e6dae48cb78 admin admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:49:07.659637 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7be2005-8de9-4517-9463-884c48ae2380 None None] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:49:07.665267 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f6ecd1-33d9-4c31-8aab-9313d8f988e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:49:07.665633 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f6ecd1-33d9-4c31-8aab-9313d8f988e5 admin admin] REQUEST_METHOD: `POST` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:07.665843 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f6ecd1-33d9-4c31-8aab-9313d8f988e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:07.666165 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f6ecd1-33d9-4c31-8aab-9313d8f988e5 admin admin] PATH_INFO: `/v3/projects` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:07.666882 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29f6ecd1-33d9-4c31-8aab-9313d8f988e5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:49:07.669962 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29f6ecd1-33d9-4c31-8aab-9313d8f988e5 admin admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:49:07.680382 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e19698fc-2cc3-4375-b61f-c9e9423681f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:49:07.682154 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19698fc-2cc3-4375-b61f-c9e9423681f9 admin admin] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:07.682756 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19698fc-2cc3-4375-b61f-c9e9423681f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:07.683419 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19698fc-2cc3-4375-b61f-c9e9423681f9 admin admin] PATH_INFO: `/v3/domains` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:07.684427 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e19698fc-2cc3-4375-b61f-c9e9423681f9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88113) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:49:07.684979 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e19698fc-2cc3-4375-b61f-c9e9423681f9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:49:07.690829 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e19698fc-2cc3-4375-b61f-c9e9423681f9 admin admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:49:07.717699 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 1047/2093] 10.222.0.13 () {58 vars in 1153 bytes} [Wed Aug 31 09:49:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 104 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 09:49:07.723726 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e19698fc-2cc3-4375-b61f-c9e9423681f9 admin admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:49:07.739522 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e19698fc-2cc3-4375-b61f-c9e9423681f9 admin admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:49:07.741590 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 1047/2094] 10.222.0.13 () {60 vars in 1153 bytes} [Wed Aug 31 09:49:07 2022] POST /identity/v3/projects => generated 383 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 09:49:07.747928 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f6ecd1-33d9-4c31-8aab-9313d8f988e5 admin admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:49:07.926680 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f6ecd1-33d9-4c31-8aab-9313d8f988e5 admin admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:49:07.937658 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa14d76-8097-4408-99c5-a16ef46d59ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:49:07.937658 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa14d76-8097-4408-99c5-a16ef46d59ae admin admin] REQUEST_METHOD: `POST` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:07.937658 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa14d76-8097-4408-99c5-a16ef46d59ae admin admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:07.937658 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa14d76-8097-4408-99c5-a16ef46d59ae admin admin] PATH_INFO: `/v3/users` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:07.938054 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aa14d76-8097-4408-99c5-a16ef46d59ae admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:49:07.943578 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aa14d76-8097-4408-99c5-a16ef46d59ae admin admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:49:07.950890 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59daf72e-2ac2-4c5b-9139-56b99d563a32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:49:07.951915 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59daf72e-2ac2-4c5b-9139-56b99d563a32 admin admin] REQUEST_METHOD: `POST` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:07.952184 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59daf72e-2ac2-4c5b-9139-56b99d563a32 admin admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:07.952532 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59daf72e-2ac2-4c5b-9139-56b99d563a32 admin admin] PATH_INFO: `/v3/projects` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:07.953767 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59daf72e-2ac2-4c5b-9139-56b99d563a32 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:49:07.956234 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59daf72e-2ac2-4c5b-9139-56b99d563a32 admin admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:49:07.958341 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:49:07.978557 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 1048/2095] 10.222.0.13 () {60 vars in 1144 bytes} [Wed Aug 31 09:49:07 2022] POST /identity/v3/users => generated 292 bytes in 232 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 09:49:07.979554 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa14d76-8097-4408-99c5-a16ef46d59ae admin admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:49:07.992217 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa14d76-8097-4408-99c5-a16ef46d59ae admin admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:49:07.998262 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e8d5e30-4dda-4173-8e3f-fe876519fdfb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:49:07.998871 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8d5e30-4dda-4173-8e3f-fe876519fdfb admin admin] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:07.999147 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8d5e30-4dda-4173-8e3f-fe876519fdfb admin admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:07.999515 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8d5e30-4dda-4173-8e3f-fe876519fdfb admin admin] PATH_INFO: `/v3/roles` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:08.000675 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e8d5e30-4dda-4173-8e3f-fe876519fdfb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:49:08.002780 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e8d5e30-4dda-4173-8e3f-fe876519fdfb admin admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:49:08.009438 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 1048/2096] 10.222.0.13 () {60 vars in 1153 bytes} [Wed Aug 31 09:49:07 2022] POST /identity/v3/projects => generated 363 bytes in 287 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 09:49:08.014558 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59daf72e-2ac2-4c5b-9139-56b99d563a32 admin admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:49:08.021786 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 1049/2097] 10.222.0.13 () {58 vars in 1122 bytes} [Wed Aug 31 09:49:07 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 09:49:08.026329 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e8d5e30-4dda-4173-8e3f-fe876519fdfb admin admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:49:08.039656 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e8d5e30-4dda-4173-8e3f-fe876519fdfb admin admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:49:08.114675 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5321f61-144b-4fd8-93ec-c4de955908f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:49:08.117585 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5321f61-144b-4fd8-93ec-c4de955908f5 admin admin] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:08.117585 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5321f61-144b-4fd8-93ec-c4de955908f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:08.117585 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5321f61-144b-4fd8-93ec-c4de955908f5 admin admin] PATH_INFO: `/v3/roles` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:08.117585 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5321f61-144b-4fd8-93ec-c4de955908f5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:49:08.118362 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5321f61-144b-4fd8-93ec-c4de955908f5 admin admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:49:08.123877 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59daf72e-2ac2-4c5b-9139-56b99d563a32 admin admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:49:08.130702 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84bfd22f-79d2-4f83-ba52-e87b6f76c99b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:49:08.132084 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 1050/2098] 10.222.0.13 () {58 vars in 1122 bytes} [Wed Aug 31 09:49:08 2022] GET /identity/v3/roles => generated 1420 bytes in 106 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 09:49:08.134714 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84bfd22f-79d2-4f83-ba52-e87b6f76c99b admin admin] REQUEST_METHOD: `POST` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:08.135003 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84bfd22f-79d2-4f83-ba52-e87b6f76c99b admin admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:08.135277 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84bfd22f-79d2-4f83-ba52-e87b6f76c99b admin admin] PATH_INFO: `/v3/users` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:08.135870 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5321f61-144b-4fd8-93ec-c4de955908f5 admin admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:49:08.136063 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84bfd22f-79d2-4f83-ba52-e87b6f76c99b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:49:08.137949 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84bfd22f-79d2-4f83-ba52-e87b6f76c99b admin admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:49:08.148549 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5321f61-144b-4fd8-93ec-c4de955908f5 admin admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:49:08.154504 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8837fec-e9e2-4af7-8917-196eb63e7836 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:49:08.155457 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8837fec-e9e2-4af7-8917-196eb63e7836 admin admin] REQUEST_METHOD: `PUT` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:08.155639 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8837fec-e9e2-4af7-8917-196eb63e7836 admin admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:08.156082 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8837fec-e9e2-4af7-8917-196eb63e7836 admin admin] PATH_INFO: `/v3/projects/468a19f9c2614aef94b0069456bc9192/users/3c3dcd5532f5411995782e61840e90fe/roles/6c99f5d9f1f243eaa134d8540f21095d` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:08.168737 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 1049/2099] 10.222.0.13 () {60 vars in 1144 bytes} [Wed Aug 31 09:49:08 2022] POST /identity/v3/users => generated 282 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 09:49:08.169551 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84bfd22f-79d2-4f83-ba52-e87b6f76c99b admin admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:49:08.179895 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8837fec-e9e2-4af7-8917-196eb63e7836 admin admin] RBAC: Authorizing `identity:create_grant(project_id=468a19f9c2614aef94b0069456bc9192, user_id=3c3dcd5532f5411995782e61840e90fe, role_id=6c99f5d9f1f243eaa134d8540f21095d)` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:49:08.182565 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8837fec-e9e2-4af7-8917-196eb63e7836 admin admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:49:08.183183 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84bfd22f-79d2-4f83-ba52-e87b6f76c99b admin admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:49:08.189528 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c787509-3fba-4a2f-af54-0e7292b6d80d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:49:08.190690 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c787509-3fba-4a2f-af54-0e7292b6d80d admin admin] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:08.191207 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c787509-3fba-4a2f-af54-0e7292b6d80d admin admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:08.191898 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c787509-3fba-4a2f-af54-0e7292b6d80d admin admin] PATH_INFO: `/v3/roles` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:08.192765 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c787509-3fba-4a2f-af54-0e7292b6d80d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:49:08.195724 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c787509-3fba-4a2f-af54-0e7292b6d80d admin admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:49:08.200645 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 1051/2100] 10.222.0.13 () {60 vars in 1483 bytes} [Wed Aug 31 09:49:08 2022] PUT /identity/v3/projects/468a19f9c2614aef94b0069456bc9192/users/3c3dcd5532f5411995782e61840e90fe/roles/6c99f5d9f1f243eaa134d8540f21095d => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 09:49:08.211696 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b492e953-12ab-4949-a534-c56b2169156d None None] REQUEST_METHOD: `POST` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:08.212073 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b492e953-12ab-4949-a534-c56b2169156d None None] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:08.212294 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b492e953-12ab-4949-a534-c56b2169156d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:08.217029 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 1050/2101] 10.222.0.13 () {58 vars in 1122 bytes} [Wed Aug 31 09:49:08 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 09:49:08.222122 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c787509-3fba-4a2f-af54-0e7292b6d80d admin admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:49:08.235868 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c787509-3fba-4a2f-af54-0e7292b6d80d admin admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:49:08.272819 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:49:08.273996 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.auth.core [None req-b492e953-12ab-4949-a534-c56b2169156d None None] MFA Rules not processed for user `3c3dcd5532f5411995782e61840e90fe`. Rule list: `[]` (Enabled: `True`). {{(pid=88114) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:49:08.275364 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.fernet_utils [None req-b492e953-12ab-4949-a534-c56b2169156d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88114) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:49:08.368590 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-225fab45-4003-46a4-aa6b-b208cc3a8c98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:49:08.369406 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225fab45-4003-46a4-aa6b-b208cc3a8c98 admin admin] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:08.369695 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225fab45-4003-46a4-aa6b-b208cc3a8c98 admin admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:08.369967 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225fab45-4003-46a4-aa6b-b208cc3a8c98 admin admin] PATH_INFO: `/v3/roles` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:08.370628 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-225fab45-4003-46a4-aa6b-b208cc3a8c98 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:49:08.372972 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-225fab45-4003-46a4-aa6b-b208cc3a8c98 admin admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:49:08.399102 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 1051/2102] 10.222.0.13 () {58 vars in 1122 bytes} [Wed Aug 31 09:49:08 2022] GET /identity/v3/roles => generated 1420 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 09:49:08.402421 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-225fab45-4003-46a4-aa6b-b208cc3a8c98 admin admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:49:08.423509 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 1052/2103] 10.222.0.13 () {58 vars in 958 bytes} [Wed Aug 31 09:49:08 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:49:08.427935 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-225fab45-4003-46a4-aa6b-b208cc3a8c98 admin admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:49:08.437614 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b492e953-12ab-4949-a534-c56b2169156d None None] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:49:08.440533 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.fernet_utils [None req-b492e953-12ab-4949-a534-c56b2169156d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88114) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:49:08.443129 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea3be3cd-b170-4662-a7da-21fd42e1ef07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:49:08.444808 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3be3cd-b170-4662-a7da-21fd42e1ef07 admin admin] REQUEST_METHOD: `PUT` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:08.445294 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3be3cd-b170-4662-a7da-21fd42e1ef07 admin admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:08.445835 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3be3cd-b170-4662-a7da-21fd42e1ef07 admin admin] PATH_INFO: `/v3/projects/77311e37f4df4dbb8b44aecd8ec83df1/users/053705cd245541bd8be740b7e3d00d3f/roles/6c99f5d9f1f243eaa134d8540f21095d` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:08.481174 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea3be3cd-b170-4662-a7da-21fd42e1ef07 admin admin] RBAC: Authorizing `identity:create_grant(project_id=77311e37f4df4dbb8b44aecd8ec83df1, user_id=053705cd245541bd8be740b7e3d00d3f, role_id=6c99f5d9f1f243eaa134d8540f21095d)` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:49:08.484193 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea3be3cd-b170-4662-a7da-21fd42e1ef07 admin admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:49:08.504002 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 1052/2104] 10.222.0.13 () {60 vars in 1483 bytes} [Wed Aug 31 09:49:08 2022] PUT /identity/v3/projects/77311e37f4df4dbb8b44aecd8ec83df1/users/053705cd245541bd8be740b7e3d00d3f/roles/6c99f5d9f1f243eaa134d8540f21095d => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:49:08.515693 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1310afa-ae29-46d4-a187-a1a99957830e None None] REQUEST_METHOD: `POST` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:08.516586 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1310afa-ae29-46d4-a187-a1a99957830e None None] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:08.516859 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1310afa-ae29-46d4-a187-a1a99957830e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:08.536936 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:49:08.544533 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b492e953-12ab-4949-a534-c56b2169156d None None] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:49:08.556217 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.auth.core [None req-b1310afa-ae29-46d4-a187-a1a99957830e None None] MFA Rules not processed for user `053705cd245541bd8be740b7e3d00d3f`. Rule list: `[]` (Enabled: `True`). {{(pid=88113) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:49:08.557824 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.fernet_utils [None req-b1310afa-ae29-46d4-a187-a1a99957830e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:49:08.624654 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d66b69-2477-4898-8571-54a14289f3f3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d52b69c8e94441dacfcff944326f169', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '49548eb44d62415582af820ab66decee', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:49:08.625785 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d66b69-2477-4898-8571-54a14289f3f3 service cinder] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:08.626724 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d66b69-2477-4898-8571-54a14289f3f3 service cinder] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:08.627297 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d66b69-2477-4898-8571-54a14289f3f3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:08.665229 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 1053/2105] 10.222.0.13 () {58 vars in 958 bytes} [Wed Aug 31 09:49:08 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 09:49:08.738995 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1310afa-ae29-46d4-a187-a1a99957830e None None] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:49:08.752848 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1310afa-ae29-46d4-a187-a1a99957830e None None] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:49:08.758629 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-624c671a-fb7d-4e2b-9982-4cecc7b12f4a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d52b69c8e94441dacfcff944326f169', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '49548eb44d62415582af820ab66decee', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:49:08.759261 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624c671a-fb7d-4e2b-9982-4cecc7b12f4a service cinder] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:08.759589 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624c671a-fb7d-4e2b-9982-4cecc7b12f4a service cinder] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:08.760978 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624c671a-fb7d-4e2b-9982-4cecc7b12f4a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:08.774153 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-624c671a-fb7d-4e2b-9982-4cecc7b12f4a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:49:08.775625 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-624c671a-fb7d-4e2b-9982-4cecc7b12f4a service cinder] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:49:08.799967 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29d66b69-2477-4898-8571-54a14289f3f3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:49:08.805653 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 1054/2106] 10.222.0.13 () {60 vars in 1365 bytes} [Wed Aug 31 09:49:08 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 66 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 09:49:08.821729 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29d66b69-2477-4898-8571-54a14289f3f3 service cinder] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:49:08.843004 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 1053/2107] 10.222.0.13 () {60 vars in 1365 bytes} [Wed Aug 31 09:49:08 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 406 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:49:08.891814 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-624c671a-fb7d-4e2b-9982-4cecc7b12f4a service cinder] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:49:09.151789 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-624c671a-fb7d-4e2b-9982-4cecc7b12f4a service cinder] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:49:09.169885 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df3a80f-853f-4f31-9143-118e0388515d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:49:09.170551 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df3a80f-853f-4f31-9143-118e0388515d admin admin] REQUEST_METHOD: `DELETE` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:09.170831 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df3a80f-853f-4f31-9143-118e0388515d admin admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:09.171105 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df3a80f-853f-4f31-9143-118e0388515d admin admin] PATH_INFO: `/v3/users/3c3dcd5532f5411995782e61840e90fe` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:09.172975 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5df3a80f-853f-4f31-9143-118e0388515d admin admin] RBAC: Authorizing `identity:delete_user(user_id=3c3dcd5532f5411995782e61840e90fe)` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:49:09.176464 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5df3a80f-853f-4f31-9143-118e0388515d admin admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:49:09.192569 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:49:09.364948 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.notifications [None req-5df3a80f-853f-4f31-9143-118e0388515d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3c3dcd5532f5411995782e61840e90fe'} {{(pid=88113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:49:09.383878 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.notifications [None req-5df3a80f-853f-4f31-9143-118e0388515d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3c3dcd5532f5411995782e61840e90fe'} {{(pid=88113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:49:09.408430 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.notifications [None req-5df3a80f-853f-4f31-9143-118e0388515d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3c3dcd5532f5411995782e61840e90fe'} {{(pid=88113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:49:09.417986 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.notifications [None req-5df3a80f-853f-4f31-9143-118e0388515d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3c3dcd5532f5411995782e61840e90fe'} {{(pid=88113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:49:09.437109 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.notifications [None req-5df3a80f-853f-4f31-9143-118e0388515d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3c3dcd5532f5411995782e61840e90fe'} {{(pid=88113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:49:09.445616 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 1055/2108] 10.222.0.13 () {58 vars in 1224 bytes} [Wed Aug 31 09:49:08 2022] DELETE /identity/v3/users/3c3dcd5532f5411995782e61840e90fe => generated 0 bytes in 554 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:49:09.450406 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baaa03c5-8738-4b4a-8478-6987892a9cbc None None] REQUEST_METHOD: `POST` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:09.450937 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baaa03c5-8738-4b4a-8478-6987892a9cbc None None] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:09.450937 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baaa03c5-8738-4b4a-8478-6987892a9cbc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:09.485210 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.auth.core [None req-baaa03c5-8738-4b4a-8478-6987892a9cbc None None] MFA Rules not processed for user `fd836cf590a145df9acf19cf3ae2ac56`. Rule list: `[]` (Enabled: `True`). {{(pid=88114) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:49:09.485210 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.fernet_utils [None req-baaa03c5-8738-4b4a-8478-6987892a9cbc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88114) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:49:09.579962 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 1054/2109] 10.222.0.13 () {58 vars in 958 bytes} [Wed Aug 31 09:49:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:49:09.588649 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df3a80f-853f-4f31-9143-118e0388515d admin admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:49:09.591304 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.fernet_utils [None req-5df3a80f-853f-4f31-9143-118e0388515d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:49:09.794559 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df3a80f-853f-4f31-9143-118e0388515d admin admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:49:09.801771 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd3f6bae-77d1-499a-a7b4-033dd3a22acc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9bdaacd597442c2bcde2c555f2cc140', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '49548eb44d62415582af820ab66decee', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:49:09.802741 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3f6bae-77d1-499a-a7b4-033dd3a22acc service neutron] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:09.803243 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3f6bae-77d1-499a-a7b4-033dd3a22acc service neutron] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:09.803702 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3f6bae-77d1-499a-a7b4-033dd3a22acc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:09.822271 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd3f6bae-77d1-499a-a7b4-033dd3a22acc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:49:09.823878 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd3f6bae-77d1-499a-a7b4-033dd3a22acc service neutron] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:49:09.849753 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 1056/2110] 10.222.0.13 () {60 vars in 1365 bytes} [Wed Aug 31 09:49:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 262 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:49:10.403416 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baaa03c5-8738-4b4a-8478-6987892a9cbc None None] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:49:10.406171 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.fernet_utils [None req-baaa03c5-8738-4b4a-8478-6987892a9cbc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88114) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:49:10.436768 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baaa03c5-8738-4b4a-8478-6987892a9cbc None None] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:49:10.445978 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef6f146-0cc1-4808-8331-c51610f5d7f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:49:10.447050 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef6f146-0cc1-4808-8331-c51610f5d7f1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:10.447861 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef6f146-0cc1-4808-8331-c51610f5d7f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:10.448716 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef6f146-0cc1-4808-8331-c51610f5d7f1 admin admin] PATH_INFO: `/v3/projects/468a19f9c2614aef94b0069456bc9192` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:10.451595 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cef6f146-0cc1-4808-8331-c51610f5d7f1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=468a19f9c2614aef94b0069456bc9192)` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:49:10.456827 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cef6f146-0cc1-4808-8331-c51610f5d7f1 admin admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:49:10.552817 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.notifications [None req-cef6f146-0cc1-4808-8331-c51610f5d7f1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '468a19f9c2614aef94b0069456bc9192', 'request_id': 'req-cef6f146-0cc1-4808-8331-c51610f5d7f1'} {{(pid=88114) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:49:10.556986 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:49:10.566071 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.notifications [None req-cef6f146-0cc1-4808-8331-c51610f5d7f1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '468a19f9c2614aef94b0069456bc9192', 'request_id': 'req-cef6f146-0cc1-4808-8331-c51610f5d7f1'} {{(pid=88114) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:49:10.577719 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.notifications [None req-cef6f146-0cc1-4808-8331-c51610f5d7f1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '468a19f9c2614aef94b0069456bc9192', 'request_id': 'req-cef6f146-0cc1-4808-8331-c51610f5d7f1'} {{(pid=88114) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:49:10.602115 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.notifications [None req-cef6f146-0cc1-4808-8331-c51610f5d7f1 admin admin] The token cache is being invalidate because project 468a19f9c2614aef94b0069456bc9192 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88114) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 09:49:10.602615 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.notifications [None req-cef6f146-0cc1-4808-8331-c51610f5d7f1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88114) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:49:10.603764 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.notifications [None req-cef6f146-0cc1-4808-8331-c51610f5d7f1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88114) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:49:10.606477 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 1055/2111] 10.222.0.13 () {58 vars in 1233 bytes} [Wed Aug 31 09:49:10 2022] DELETE /identity/v3/projects/468a19f9c2614aef94b0069456bc9192 => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:49:10.616592 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf5e60a-23d8-4291-a19b-8e643fffd052 None None] REQUEST_METHOD: `POST` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:10.616970 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf5e60a-23d8-4291-a19b-8e643fffd052 None None] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:10.617785 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf5e60a-23d8-4291-a19b-8e643fffd052 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:10.647913 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.auth.core [None req-5cf5e60a-23d8-4291-a19b-8e643fffd052 None None] MFA Rules not processed for user `fd836cf590a145df9acf19cf3ae2ac56`. Rule list: `[]` (Enabled: `True`). {{(pid=88113) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:49:10.649289 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.fernet_utils [None req-5cf5e60a-23d8-4291-a19b-8e643fffd052 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:49:10.778685 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 1057/2112] 10.222.0.13 () {58 vars in 958 bytes} [Wed Aug 31 09:49:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:49:10.789898 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917f2880-322d-4dc4-a783-09f59d8eb62d None None] REQUEST_METHOD: `POST` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:10.790539 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917f2880-322d-4dc4-a783-09f59d8eb62d None None] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:10.791164 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917f2880-322d-4dc4-a783-09f59d8eb62d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:10.855729 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.auth.core [None req-917f2880-322d-4dc4-a783-09f59d8eb62d None None] MFA Rules not processed for user `fd836cf590a145df9acf19cf3ae2ac56`. Rule list: `[]` (Enabled: `True`). {{(pid=88114) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:49:10.858804 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.fernet_utils [None req-917f2880-322d-4dc4-a783-09f59d8eb62d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88114) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:49:10.881450 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 1056/2113] 10.222.0.13 () {58 vars in 958 bytes} [Wed Aug 31 09:49:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:49:10.883727 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf5e60a-23d8-4291-a19b-8e643fffd052 None None] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:49:10.903799 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf5e60a-23d8-4291-a19b-8e643fffd052 None None] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:49:10.914329 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a1ad7f-07d5-4e43-a537-dbddcb090ec6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:49:10.915557 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a1ad7f-07d5-4e43-a537-dbddcb090ec6 admin admin] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:10.916168 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a1ad7f-07d5-4e43-a537-dbddcb090ec6 admin admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:10.916737 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a1ad7f-07d5-4e43-a537-dbddcb090ec6 admin admin] PATH_INFO: `/v3/domains` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:10.918086 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6a1ad7f-07d5-4e43-a537-dbddcb090ec6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88113) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:49:10.918922 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6a1ad7f-07d5-4e43-a537-dbddcb090ec6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:49:10.922014 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6a1ad7f-07d5-4e43-a537-dbddcb090ec6 admin admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:49:10.961795 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 1058/2114] 10.222.0.13 () {58 vars in 1153 bytes} [Wed Aug 31 09:49:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:49:10.961969 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-917f2880-322d-4dc4-a783-09f59d8eb62d None None] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:49:10.968652 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-917f2880-322d-4dc4-a783-09f59d8eb62d None None] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:49:10.980092 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c5b7da9-e980-4511-9240-3590ae25add7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:49:10.980969 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5b7da9-e980-4511-9240-3590ae25add7 admin admin] REQUEST_METHOD: `POST` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:10.981447 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5b7da9-e980-4511-9240-3590ae25add7 admin admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:10.981895 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5b7da9-e980-4511-9240-3590ae25add7 admin admin] PATH_INFO: `/v3/projects` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:10.983007 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c5b7da9-e980-4511-9240-3590ae25add7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:49:10.988178 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c5b7da9-e980-4511-9240-3590ae25add7 admin admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:49:11.042153 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 1057/2115] 10.222.0.13 () {60 vars in 1153 bytes} [Wed Aug 31 09:49:10 2022] POST /identity/v3/projects => generated 359 bytes in 86 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 09:49:11.046709 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a1ad7f-07d5-4e43-a537-dbddcb090ec6 admin admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:49:11.067055 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a1ad7f-07d5-4e43-a537-dbddcb090ec6 admin admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:49:11.108722 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:49:11.198640 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-098383d4-6da6-4f9c-a940-2fa354296edc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:49:11.199736 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098383d4-6da6-4f9c-a940-2fa354296edc admin admin] REQUEST_METHOD: `POST` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:11.200172 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098383d4-6da6-4f9c-a940-2fa354296edc admin admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:11.200567 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098383d4-6da6-4f9c-a940-2fa354296edc admin admin] PATH_INFO: `/v3/users` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:11.201844 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-098383d4-6da6-4f9c-a940-2fa354296edc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:49:11.204857 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-098383d4-6da6-4f9c-a940-2fa354296edc admin admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:49:11.267328 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 1059/2116] 10.222.0.13 () {60 vars in 1144 bytes} [Wed Aug 31 09:49:11 2022] POST /identity/v3/users => generated 280 bytes in 218 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 09:49:11.269893 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c5b7da9-e980-4511-9240-3590ae25add7 admin admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:49:11.295876 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c5b7da9-e980-4511-9240-3590ae25add7 admin admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:49:11.304176 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9443f2d1-68df-4caa-8254-122a345861f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:49:11.305848 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9443f2d1-68df-4caa-8254-122a345861f8 admin admin] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:11.306136 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9443f2d1-68df-4caa-8254-122a345861f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:11.306498 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9443f2d1-68df-4caa-8254-122a345861f8 admin admin] PATH_INFO: `/v3/roles` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:11.307125 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9443f2d1-68df-4caa-8254-122a345861f8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:49:11.308750 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9443f2d1-68df-4caa-8254-122a345861f8 admin admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:49:11.329112 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 1058/2117] 10.222.0.13 () {58 vars in 1122 bytes} [Wed Aug 31 09:49:11 2022] GET /identity/v3/roles => generated 1420 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 09:49:11.338629 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-098383d4-6da6-4f9c-a940-2fa354296edc admin admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:49:11.369301 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-098383d4-6da6-4f9c-a940-2fa354296edc admin admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:49:11.380678 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd39ad5-0c6c-452d-a0ae-ae498964533a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:49:11.381991 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd39ad5-0c6c-452d-a0ae-ae498964533a admin admin] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:11.383440 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd39ad5-0c6c-452d-a0ae-ae498964533a admin admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:11.384769 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd39ad5-0c6c-452d-a0ae-ae498964533a admin admin] PATH_INFO: `/v3/roles` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:11.386158 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cd39ad5-0c6c-452d-a0ae-ae498964533a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:49:11.389453 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cd39ad5-0c6c-452d-a0ae-ae498964533a admin admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:49:11.414968 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 1060/2118] 10.222.0.13 () {58 vars in 1122 bytes} [Wed Aug 31 09:49:11 2022] GET /identity/v3/roles => generated 1420 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 09:49:11.421951 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9443f2d1-68df-4caa-8254-122a345861f8 admin admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:49:11.450002 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9443f2d1-68df-4caa-8254-122a345861f8 admin admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:49:11.466243 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8362c27f-2b99-4cb0-9e14-0329ac2818b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:49:11.467225 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8362c27f-2b99-4cb0-9e14-0329ac2818b0 admin admin] REQUEST_METHOD: `PUT` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:11.467695 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8362c27f-2b99-4cb0-9e14-0329ac2818b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:11.468170 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8362c27f-2b99-4cb0-9e14-0329ac2818b0 admin admin] PATH_INFO: `/v3/projects/09a3b3c80f6f404fbbf39bdf479bfdf4/users/104eb2f609e64a90bafffdb6029a15fe/roles/6c99f5d9f1f243eaa134d8540f21095d` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:11.505211 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8362c27f-2b99-4cb0-9e14-0329ac2818b0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=09a3b3c80f6f404fbbf39bdf479bfdf4, user_id=104eb2f609e64a90bafffdb6029a15fe, role_id=6c99f5d9f1f243eaa134d8540f21095d)` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:49:11.521908 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8362c27f-2b99-4cb0-9e14-0329ac2818b0 admin admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:49:11.560890 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 1059/2119] 10.222.0.13 () {60 vars in 1483 bytes} [Wed Aug 31 09:49:11 2022] PUT /identity/v3/projects/09a3b3c80f6f404fbbf39bdf479bfdf4/users/104eb2f609e64a90bafffdb6029a15fe/roles/6c99f5d9f1f243eaa134d8540f21095d => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 09:49:11.584962 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff242df-0c45-4a74-ba7a-903c44f23a2b None None] REQUEST_METHOD: `POST` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:11.585419 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff242df-0c45-4a74-ba7a-903c44f23a2b None None] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:11.585767 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff242df-0c45-4a74-ba7a-903c44f23a2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:11.618693 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8362c27f-2b99-4cb0-9e14-0329ac2818b0 admin admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:49:11.622434 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.fernet_utils [None req-8362c27f-2b99-4cb0-9e14-0329ac2818b0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88114) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:49:11.710575 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.auth.core [None req-3ff242df-0c45-4a74-ba7a-903c44f23a2b None None] MFA Rules not processed for user `104eb2f609e64a90bafffdb6029a15fe`. Rule list: `[]` (Enabled: `True`). {{(pid=88113) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:49:11.713659 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.fernet_utils [None req-3ff242df-0c45-4a74-ba7a-903c44f23a2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:49:11.726141 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8362c27f-2b99-4cb0-9e14-0329ac2818b0 admin admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:49:11.738697 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c7fcea-9812-4933-87bb-8a35cf602dbc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:49:11.742764 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c7fcea-9812-4933-87bb-8a35cf602dbc admin admin] REQUEST_METHOD: `DELETE` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:11.742764 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c7fcea-9812-4933-87bb-8a35cf602dbc admin admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:11.742764 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c7fcea-9812-4933-87bb-8a35cf602dbc admin admin] PATH_INFO: `/v3/users/1e9acc8fe90c4c7ea63f86dfd468809c` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:11.746554 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9c7fcea-9812-4933-87bb-8a35cf602dbc admin admin] RBAC: Authorizing `identity:delete_user(user_id=1e9acc8fe90c4c7ea63f86dfd468809c)` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:49:11.753590 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9c7fcea-9812-4933-87bb-8a35cf602dbc admin admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:49:11.878433 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.notifications [None req-d9c7fcea-9812-4933-87bb-8a35cf602dbc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1e9acc8fe90c4c7ea63f86dfd468809c'} {{(pid=88114) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:49:11.904249 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.notifications [None req-d9c7fcea-9812-4933-87bb-8a35cf602dbc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1e9acc8fe90c4c7ea63f86dfd468809c'} {{(pid=88114) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:49:11.934752 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.notifications [None req-d9c7fcea-9812-4933-87bb-8a35cf602dbc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1e9acc8fe90c4c7ea63f86dfd468809c'} {{(pid=88114) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:49:11.937120 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.notifications [None req-d9c7fcea-9812-4933-87bb-8a35cf602dbc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1e9acc8fe90c4c7ea63f86dfd468809c'} {{(pid=88114) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:49:11.950590 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 1061/2120] 10.222.0.13 () {58 vars in 958 bytes} [Wed Aug 31 09:49:11 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 367 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:49:12.012569 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=83,keystone:DELETE=13,keystone:INSERT=4 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:49:12.015236 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.notifications [None req-d9c7fcea-9812-4933-87bb-8a35cf602dbc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1e9acc8fe90c4c7ea63f86dfd468809c'} {{(pid=88114) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:49:12.030862 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff242df-0c45-4a74-ba7a-903c44f23a2b None None] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:49:12.032777 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.fernet_utils [None req-3ff242df-0c45-4a74-ba7a-903c44f23a2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:49:12.054253 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 1060/2121] 10.222.0.13 () {58 vars in 1224 bytes} [Wed Aug 31 09:49:11 2022] DELETE /identity/v3/users/1e9acc8fe90c4c7ea63f86dfd468809c => generated 0 bytes in 436 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:49:12.059330 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83fb1bf-c47e-43ef-85dd-030b625f9fe6 None None] REQUEST_METHOD: `POST` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:12.059649 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83fb1bf-c47e-43ef-85dd-030b625f9fe6 None None] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:12.060024 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83fb1bf-c47e-43ef-85dd-030b625f9fe6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:12.088505 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.auth.core [None req-f83fb1bf-c47e-43ef-85dd-030b625f9fe6 None None] MFA Rules not processed for user `fd836cf590a145df9acf19cf3ae2ac56`. Rule list: `[]` (Enabled: `True`). {{(pid=88114) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:49:12.089544 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.fernet_utils [None req-f83fb1bf-c47e-43ef-85dd-030b625f9fe6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88114) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:49:12.180114 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff242df-0c45-4a74-ba7a-903c44f23a2b None None] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:49:12.191565 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 1061/2122] 10.222.0.13 () {58 vars in 958 bytes} [Wed Aug 31 09:49:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 09:49:12.207070 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f83fb1bf-c47e-43ef-85dd-030b625f9fe6 None None] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:49:12.210374 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.fernet_utils [None req-f83fb1bf-c47e-43ef-85dd-030b625f9fe6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88114) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:49:12.246875 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:49:12.281865 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce9086e2-48eb-4265-8f73-a9499e0f827a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d52b69c8e94441dacfcff944326f169', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '49548eb44d62415582af820ab66decee', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:49:12.282905 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9086e2-48eb-4265-8f73-a9499e0f827a service cinder] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:12.284090 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9086e2-48eb-4265-8f73-a9499e0f827a service cinder] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:12.284607 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9086e2-48eb-4265-8f73-a9499e0f827a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:12.287990 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.fernet_utils [None req-ce9086e2-48eb-4265-8f73-a9499e0f827a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:49:12.296955 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f83fb1bf-c47e-43ef-85dd-030b625f9fe6 None None] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:49:12.301952 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cba4d12-b691-4207-b013-eabb77f9d27a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9bdaacd597442c2bcde2c555f2cc140', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '49548eb44d62415582af820ab66decee', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:49:12.302585 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cba4d12-b691-4207-b013-eabb77f9d27a service neutron] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:12.303026 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cba4d12-b691-4207-b013-eabb77f9d27a service neutron] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:12.303309 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cba4d12-b691-4207-b013-eabb77f9d27a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:12.315813 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cba4d12-b691-4207-b013-eabb77f9d27a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:49:12.317446 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cba4d12-b691-4207-b013-eabb77f9d27a service neutron] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:49:12.328580 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 1062/2123] 10.222.0.13 () {60 vars in 1365 bytes} [Wed Aug 31 09:49:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 09:49:12.394317 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce9086e2-48eb-4265-8f73-a9499e0f827a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:49:12.399307 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce9086e2-48eb-4265-8f73-a9499e0f827a service cinder] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:49:12.464382 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 1062/2124] 10.222.0.13 () {60 vars in 1365 bytes} [Wed Aug 31 09:49:12 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 433 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:49:12.494055 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cba4d12-b691-4207-b013-eabb77f9d27a service neutron] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:49:12.496616 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.fernet_utils [None req-0cba4d12-b691-4207-b013-eabb77f9d27a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88114) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:49:12.517187 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cba4d12-b691-4207-b013-eabb77f9d27a service neutron] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:49:12.525277 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fbd4086-d46d-4211-bbc7-9fada26d9c5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:49:12.526490 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fbd4086-d46d-4211-bbc7-9fada26d9c5e admin admin] REQUEST_METHOD: `DELETE` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:12.527104 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fbd4086-d46d-4211-bbc7-9fada26d9c5e admin admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:12.530366 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fbd4086-d46d-4211-bbc7-9fada26d9c5e admin admin] PATH_INFO: `/v3/users/104eb2f609e64a90bafffdb6029a15fe` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:12.558041 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fbd4086-d46d-4211-bbc7-9fada26d9c5e admin admin] RBAC: Authorizing `identity:delete_user(user_id=104eb2f609e64a90bafffdb6029a15fe)` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:49:12.568348 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fbd4086-d46d-4211-bbc7-9fada26d9c5e admin admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:49:12.672505 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.notifications [None req-0fbd4086-d46d-4211-bbc7-9fada26d9c5e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '104eb2f609e64a90bafffdb6029a15fe'} {{(pid=88114) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:49:12.692097 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.notifications [None req-0fbd4086-d46d-4211-bbc7-9fada26d9c5e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '104eb2f609e64a90bafffdb6029a15fe'} {{(pid=88114) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:49:12.706759 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.notifications [None req-0fbd4086-d46d-4211-bbc7-9fada26d9c5e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '104eb2f609e64a90bafffdb6029a15fe'} {{(pid=88114) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:49:12.707917 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce9086e2-48eb-4265-8f73-a9499e0f827a service cinder] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:49:12.709720 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.fernet_utils [None req-ce9086e2-48eb-4265-8f73-a9499e0f827a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:49:12.710944 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.notifications [None req-0fbd4086-d46d-4211-bbc7-9fada26d9c5e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '104eb2f609e64a90bafffdb6029a15fe'} {{(pid=88114) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:49:12.723019 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce9086e2-48eb-4265-8f73-a9499e0f827a service cinder] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:49:12.728519 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a20408bc-2512-472f-877f-1f5d6eb93ff6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d52b69c8e94441dacfcff944326f169', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '49548eb44d62415582af820ab66decee', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:49:12.729271 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20408bc-2512-472f-877f-1f5d6eb93ff6 service cinder] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:12.729740 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20408bc-2512-472f-877f-1f5d6eb93ff6 service cinder] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:12.730070 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20408bc-2512-472f-877f-1f5d6eb93ff6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:12.732374 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.fernet_utils [None req-a20408bc-2512-472f-877f-1f5d6eb93ff6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:49:12.741370 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.notifications [None req-0fbd4086-d46d-4211-bbc7-9fada26d9c5e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '104eb2f609e64a90bafffdb6029a15fe'} {{(pid=88114) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:49:12.753491 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 1063/2125] 10.222.0.13 () {58 vars in 1224 bytes} [Wed Aug 31 09:49:12 2022] DELETE /identity/v3/users/104eb2f609e64a90bafffdb6029a15fe => generated 0 bytes in 257 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:49:12.755778 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c527f0-e7fe-414e-8e93-70e9a808c8a2 None None] REQUEST_METHOD: `POST` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:12.756541 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c527f0-e7fe-414e-8e93-70e9a808c8a2 None None] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:12.756774 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c527f0-e7fe-414e-8e93-70e9a808c8a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:12.780805 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.auth.core [None req-40c527f0-e7fe-414e-8e93-70e9a808c8a2 None None] MFA Rules not processed for user `fd836cf590a145df9acf19cf3ae2ac56`. Rule list: `[]` (Enabled: `True`). {{(pid=88114) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:49:12.782099 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.fernet_utils [None req-40c527f0-e7fe-414e-8e93-70e9a808c8a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88114) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:49:12.790186 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:49:12.853886 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a20408bc-2512-472f-877f-1f5d6eb93ff6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:49:12.856387 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a20408bc-2512-472f-877f-1f5d6eb93ff6 service cinder] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:49:12.869761 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 1063/2126] 10.222.0.13 () {60 vars in 1365 bytes} [Wed Aug 31 09:49:12 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 162 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 09:49:12.873420 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 1064/2127] 10.222.0.13 () {58 vars in 958 bytes} [Wed Aug 31 09:49:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:49:12.886137 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a20408bc-2512-472f-877f-1f5d6eb93ff6 service cinder] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:49:12.887823 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.fernet_utils [None req-a20408bc-2512-472f-877f-1f5d6eb93ff6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:49:12.916374 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a20408bc-2512-472f-877f-1f5d6eb93ff6 service cinder] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:49:12.951975 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c527f0-e7fe-414e-8e93-70e9a808c8a2 None None] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:49:12.953730 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.fernet_utils [None req-40c527f0-e7fe-414e-8e93-70e9a808c8a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88114) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:49:12.983975 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c527f0-e7fe-414e-8e93-70e9a808c8a2 None None] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:49:12.994625 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b68fdea4-a757-45a7-8907-0d7ea1d6e69e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:49:12.995737 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68fdea4-a757-45a7-8907-0d7ea1d6e69e admin admin] REQUEST_METHOD: `DELETE` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:12.996369 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68fdea4-a757-45a7-8907-0d7ea1d6e69e admin admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:12.996879 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68fdea4-a757-45a7-8907-0d7ea1d6e69e admin admin] PATH_INFO: `/v3/projects/1bb839c560c64bed81e75e12bec0d7aa` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:13.005369 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b68fdea4-a757-45a7-8907-0d7ea1d6e69e admin admin] RBAC: Authorizing `identity:delete_project(project_id=1bb839c560c64bed81e75e12bec0d7aa)` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:49:13.011588 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b68fdea4-a757-45a7-8907-0d7ea1d6e69e admin admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:49:13.035729 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e9c761-46d9-46a4-ac83-fb022f8382c6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9bdaacd597442c2bcde2c555f2cc140', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '49548eb44d62415582af820ab66decee', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:49:13.036466 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e9c761-46d9-46a4-ac83-fb022f8382c6 service neutron] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:13.036777 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e9c761-46d9-46a4-ac83-fb022f8382c6 service neutron] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:13.037115 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e9c761-46d9-46a4-ac83-fb022f8382c6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:13.053897 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53e9c761-46d9-46a4-ac83-fb022f8382c6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:49:13.056743 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53e9c761-46d9-46a4-ac83-fb022f8382c6 service neutron] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:49:13.090839 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 1064/2128] 10.222.0.13 () {60 vars in 1365 bytes} [Wed Aug 31 09:49:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 205 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:49:13.160924 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.notifications [None req-b68fdea4-a757-45a7-8907-0d7ea1d6e69e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1bb839c560c64bed81e75e12bec0d7aa', 'request_id': 'req-b68fdea4-a757-45a7-8907-0d7ea1d6e69e'} {{(pid=88114) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:49:13.172668 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.notifications [None req-b68fdea4-a757-45a7-8907-0d7ea1d6e69e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1bb839c560c64bed81e75e12bec0d7aa', 'request_id': 'req-b68fdea4-a757-45a7-8907-0d7ea1d6e69e'} {{(pid=88114) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:49:13.183655 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.notifications [None req-b68fdea4-a757-45a7-8907-0d7ea1d6e69e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1bb839c560c64bed81e75e12bec0d7aa', 'request_id': 'req-b68fdea4-a757-45a7-8907-0d7ea1d6e69e'} {{(pid=88114) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:49:13.214455 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.notifications [None req-b68fdea4-a757-45a7-8907-0d7ea1d6e69e admin admin] The token cache is being invalidate because project 1bb839c560c64bed81e75e12bec0d7aa was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88114) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 09:49:13.214883 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.notifications [None req-b68fdea4-a757-45a7-8907-0d7ea1d6e69e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88114) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:49:13.215662 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.notifications [None req-b68fdea4-a757-45a7-8907-0d7ea1d6e69e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88114) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:49:13.217624 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 1065/2129] 10.222.0.13 () {58 vars in 1233 bytes} [Wed Aug 31 09:49:12 2022] DELETE /identity/v3/projects/1bb839c560c64bed81e75e12bec0d7aa => generated 0 bytes in 267 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:49:13.224104 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e9c761-46d9-46a4-ac83-fb022f8382c6 service neutron] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:49:13.239904 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e9c761-46d9-46a4-ac83-fb022f8382c6 service neutron] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:49:13.321230 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=100 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:49:13.423210 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b1e279-4da5-4147-9608-75d916d4bc67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:49:13.424035 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b1e279-4da5-4147-9608-75d916d4bc67 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:13.424433 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b1e279-4da5-4147-9608-75d916d4bc67 admin admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:13.424881 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b1e279-4da5-4147-9608-75d916d4bc67 admin admin] PATH_INFO: `/v3/users/69878209c5e94774a21296a004dea3a7` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:13.426829 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64b1e279-4da5-4147-9608-75d916d4bc67 admin admin] RBAC: Authorizing `identity:delete_user(user_id=69878209c5e94774a21296a004dea3a7)` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:49:13.428863 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64b1e279-4da5-4147-9608-75d916d4bc67 admin admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:49:13.514073 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.notifications [None req-64b1e279-4da5-4147-9608-75d916d4bc67 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '69878209c5e94774a21296a004dea3a7'} {{(pid=88113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:49:13.534913 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.notifications [None req-64b1e279-4da5-4147-9608-75d916d4bc67 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '69878209c5e94774a21296a004dea3a7'} {{(pid=88113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:49:13.564300 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.notifications [None req-64b1e279-4da5-4147-9608-75d916d4bc67 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '69878209c5e94774a21296a004dea3a7'} {{(pid=88113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:49:13.570229 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.notifications [None req-64b1e279-4da5-4147-9608-75d916d4bc67 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '69878209c5e94774a21296a004dea3a7'} {{(pid=88113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:49:13.581148 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b68fdea4-a757-45a7-8907-0d7ea1d6e69e admin admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:49:13.586025 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.fernet_utils [None req-b68fdea4-a757-45a7-8907-0d7ea1d6e69e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88114) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:49:13.627990 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.notifications [None req-64b1e279-4da5-4147-9608-75d916d4bc67 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '69878209c5e94774a21296a004dea3a7'} {{(pid=88113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:49:13.643933 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 1065/2130] 10.222.0.13 () {58 vars in 1224 bytes} [Wed Aug 31 09:49:13 2022] DELETE /identity/v3/users/69878209c5e94774a21296a004dea3a7 => generated 0 bytes in 420 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 09:49:13.644164 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b68fdea4-a757-45a7-8907-0d7ea1d6e69e admin admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:49:13.774635 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba6f20be-2bf5-4c5b-aac0-9eccff1f5eb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:49:13.775439 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6f20be-2bf5-4c5b-aac0-9eccff1f5eb2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:13.775556 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6f20be-2bf5-4c5b-aac0-9eccff1f5eb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:13.775829 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6f20be-2bf5-4c5b-aac0-9eccff1f5eb2 admin admin] PATH_INFO: `/v3/projects/09a3b3c80f6f404fbbf39bdf479bfdf4` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:13.777087 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba6f20be-2bf5-4c5b-aac0-9eccff1f5eb2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=09a3b3c80f6f404fbbf39bdf479bfdf4)` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:49:13.779131 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba6f20be-2bf5-4c5b-aac0-9eccff1f5eb2 admin admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:49:13.860891 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:49:13.888347 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.notifications [None req-ba6f20be-2bf5-4c5b-aac0-9eccff1f5eb2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '09a3b3c80f6f404fbbf39bdf479bfdf4', 'request_id': 'req-ba6f20be-2bf5-4c5b-aac0-9eccff1f5eb2'} {{(pid=88114) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:49:13.901090 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.notifications [None req-ba6f20be-2bf5-4c5b-aac0-9eccff1f5eb2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '09a3b3c80f6f404fbbf39bdf479bfdf4', 'request_id': 'req-ba6f20be-2bf5-4c5b-aac0-9eccff1f5eb2'} {{(pid=88114) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:49:13.912265 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.notifications [None req-ba6f20be-2bf5-4c5b-aac0-9eccff1f5eb2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '09a3b3c80f6f404fbbf39bdf479bfdf4', 'request_id': 'req-ba6f20be-2bf5-4c5b-aac0-9eccff1f5eb2'} {{(pid=88114) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:49:13.942694 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.notifications [None req-ba6f20be-2bf5-4c5b-aac0-9eccff1f5eb2 admin admin] The token cache is being invalidate because project 09a3b3c80f6f404fbbf39bdf479bfdf4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88114) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 09:49:13.942694 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.notifications [None req-ba6f20be-2bf5-4c5b-aac0-9eccff1f5eb2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88114) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:49:13.942694 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.notifications [None req-ba6f20be-2bf5-4c5b-aac0-9eccff1f5eb2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88114) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:49:13.942694 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 1066/2131] 10.222.0.13 () {58 vars in 1233 bytes} [Wed Aug 31 09:49:13 2022] DELETE /identity/v3/projects/09a3b3c80f6f404fbbf39bdf479bfdf4 => generated 0 bytes in 359 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:49:13.950503 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0018e7cf-9805-4dc4-939b-ddaa00a7f208 None None] REQUEST_METHOD: `POST` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:13.950939 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0018e7cf-9805-4dc4-939b-ddaa00a7f208 None None] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:13.951290 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0018e7cf-9805-4dc4-939b-ddaa00a7f208 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:14.009318 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.auth.core [None req-0018e7cf-9805-4dc4-939b-ddaa00a7f208 None None] MFA Rules not processed for user `fd836cf590a145df9acf19cf3ae2ac56`. Rule list: `[]` (Enabled: `True`). {{(pid=88113) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:49:14.011135 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.fernet_utils [None req-0018e7cf-9805-4dc4-939b-ddaa00a7f208 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:49:14.139639 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba6f20be-2bf5-4c5b-aac0-9eccff1f5eb2 admin admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:49:14.141723 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.fernet_utils [None req-ba6f20be-2bf5-4c5b-aac0-9eccff1f5eb2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88114) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:49:14.153861 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 1066/2132] 10.222.0.13 () {58 vars in 958 bytes} [Wed Aug 31 09:49:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 208 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 09:49:14.162477 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed35e834-7ba1-40af-bfc3-8335c4f29bff None None] REQUEST_METHOD: `POST` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:14.162769 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed35e834-7ba1-40af-bfc3-8335c4f29bff None None] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:14.163036 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed35e834-7ba1-40af-bfc3-8335c4f29bff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:14.195167 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.auth.core [None req-ed35e834-7ba1-40af-bfc3-8335c4f29bff None None] MFA Rules not processed for user `fd836cf590a145df9acf19cf3ae2ac56`. Rule list: `[]` (Enabled: `True`). {{(pid=88113) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:49:14.195167 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.fernet_utils [None req-ed35e834-7ba1-40af-bfc3-8335c4f29bff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:49:14.204318 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 1067/2133] 10.222.0.13 () {58 vars in 958 bytes} [Wed Aug 31 09:49:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 09:49:14.205984 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed35e834-7ba1-40af-bfc3-8335c4f29bff None None] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:49:14.240425 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed35e834-7ba1-40af-bfc3-8335c4f29bff None None] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:49:14.247763 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d948f3b6-664b-423f-bf43-b76a63f2b020 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:49:14.248230 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d948f3b6-664b-423f-bf43-b76a63f2b020 admin admin] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:14.248582 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d948f3b6-664b-423f-bf43-b76a63f2b020 admin admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:14.248931 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d948f3b6-664b-423f-bf43-b76a63f2b020 admin admin] PATH_INFO: `/v3/domains` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:14.249776 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d948f3b6-664b-423f-bf43-b76a63f2b020 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88113) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:49:14.250155 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d948f3b6-664b-423f-bf43-b76a63f2b020 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:49:14.252893 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d948f3b6-664b-423f-bf43-b76a63f2b020 admin admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:49:14.277659 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 1068/2134] 10.222.0.13 () {58 vars in 1153 bytes} [Wed Aug 31 09:49:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:49:14.277659 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d948f3b6-664b-423f-bf43-b76a63f2b020 admin admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:49:14.303877 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d948f3b6-664b-423f-bf43-b76a63f2b020 admin admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:49:14.322442 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20490884-97a9-45fb-878a-184c09118154 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:49:14.323461 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20490884-97a9-45fb-878a-184c09118154 admin admin] REQUEST_METHOD: `POST` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:14.323886 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20490884-97a9-45fb-878a-184c09118154 admin admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:14.324317 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20490884-97a9-45fb-878a-184c09118154 admin admin] PATH_INFO: `/v3/projects` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:14.325257 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20490884-97a9-45fb-878a-184c09118154 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:49:14.327429 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20490884-97a9-45fb-878a-184c09118154 admin admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:49:14.331958 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba6f20be-2bf5-4c5b-aac0-9eccff1f5eb2 admin admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:49:14.339214 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e2e0c2d-06da-49f8-a882-eb642554e7c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:49:14.340005 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2e0c2d-06da-49f8-a882-eb642554e7c8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:14.340390 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2e0c2d-06da-49f8-a882-eb642554e7c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:14.340815 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2e0c2d-06da-49f8-a882-eb642554e7c8 admin admin] PATH_INFO: `/v3/projects/3d9546d7f106424f96461fc562e94a61` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:14.342129 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e2e0c2d-06da-49f8-a882-eb642554e7c8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3d9546d7f106424f96461fc562e94a61)` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:49:14.345327 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e2e0c2d-06da-49f8-a882-eb642554e7c8 admin admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:49:14.412230 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 1069/2135] 10.222.0.13 () {60 vars in 1153 bytes} [Wed Aug 31 09:49:14 2022] POST /identity/v3/projects => generated 369 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 09:49:14.413803 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20490884-97a9-45fb-878a-184c09118154 admin admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:49:14.429913 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:49:14.430507 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.notifications [None req-5e2e0c2d-06da-49f8-a882-eb642554e7c8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3d9546d7f106424f96461fc562e94a61', 'request_id': 'req-5e2e0c2d-06da-49f8-a882-eb642554e7c8'} {{(pid=88114) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:49:14.441993 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.notifications [None req-5e2e0c2d-06da-49f8-a882-eb642554e7c8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3d9546d7f106424f96461fc562e94a61', 'request_id': 'req-5e2e0c2d-06da-49f8-a882-eb642554e7c8'} {{(pid=88114) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:49:14.456839 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.notifications [None req-5e2e0c2d-06da-49f8-a882-eb642554e7c8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3d9546d7f106424f96461fc562e94a61', 'request_id': 'req-5e2e0c2d-06da-49f8-a882-eb642554e7c8'} {{(pid=88114) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:49:14.499662 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.notifications [None req-5e2e0c2d-06da-49f8-a882-eb642554e7c8 admin admin] The token cache is being invalidate because project 3d9546d7f106424f96461fc562e94a61 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88114) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 09:49:14.500192 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.notifications [None req-5e2e0c2d-06da-49f8-a882-eb642554e7c8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88114) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:49:14.501584 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.notifications [None req-5e2e0c2d-06da-49f8-a882-eb642554e7c8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88114) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:49:14.509520 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 1067/2136] 10.222.0.13 () {58 vars in 1233 bytes} [Wed Aug 31 09:49:14 2022] DELETE /identity/v3/projects/3d9546d7f106424f96461fc562e94a61 => generated 0 bytes in 365 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 09:49:14.512781 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d991689-865e-497c-9a42-9286e8e24e87 None None] REQUEST_METHOD: `POST` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:14.513245 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d991689-865e-497c-9a42-9286e8e24e87 None None] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:14.513738 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d991689-865e-497c-9a42-9286e8e24e87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:14.553286 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.auth.core [None req-4d991689-865e-497c-9a42-9286e8e24e87 None None] MFA Rules not processed for user `fd836cf590a145df9acf19cf3ae2ac56`. Rule list: `[]` (Enabled: `True`). {{(pid=88114) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:49:14.555138 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20490884-97a9-45fb-878a-184c09118154 admin admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:49:14.555421 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.fernet_utils [None req-4d991689-865e-497c-9a42-9286e8e24e87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88114) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:49:14.562789 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d06bca-78eb-4636-b6d5-ed99e85adce2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:49:14.563591 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d06bca-78eb-4636-b6d5-ed99e85adce2 admin admin] REQUEST_METHOD: `POST` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:14.563859 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d06bca-78eb-4636-b6d5-ed99e85adce2 admin admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:14.564171 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d06bca-78eb-4636-b6d5-ed99e85adce2 admin admin] PATH_INFO: `/v3/users` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:14.564890 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73d06bca-78eb-4636-b6d5-ed99e85adce2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:49:14.566994 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73d06bca-78eb-4636-b6d5-ed99e85adce2 admin admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:49:14.577719 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 1068/2137] 10.222.0.13 () {58 vars in 958 bytes} [Wed Aug 31 09:49:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 60 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:49:14.583597 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc3cf01-95e8-4d85-83c9-a65e924585ca None None] REQUEST_METHOD: `POST` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:14.585298 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc3cf01-95e8-4d85-83c9-a65e924585ca None None] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:14.586597 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc3cf01-95e8-4d85-83c9-a65e924585ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:14.597161 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 1070/2138] 10.222.0.13 () {60 vars in 1144 bytes} [Wed Aug 31 09:49:14 2022] POST /identity/v3/users => generated 285 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 09:49:14.603091 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d06bca-78eb-4636-b6d5-ed99e85adce2 admin admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:49:14.605377 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.fernet_utils [None req-73d06bca-78eb-4636-b6d5-ed99e85adce2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:49:14.620090 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d06bca-78eb-4636-b6d5-ed99e85adce2 admin admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:49:14.632255 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a19b37a-59ae-4014-a507-15a08aa0f20f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:49:14.632897 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a19b37a-59ae-4014-a507-15a08aa0f20f admin admin] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:14.633177 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a19b37a-59ae-4014-a507-15a08aa0f20f admin admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:14.633537 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a19b37a-59ae-4014-a507-15a08aa0f20f admin admin] PATH_INFO: `/v3/roles` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:14.634201 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a19b37a-59ae-4014-a507-15a08aa0f20f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:49:14.635880 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a19b37a-59ae-4014-a507-15a08aa0f20f admin admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:49:14.636846 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.auth.core [None req-dfc3cf01-95e8-4d85-83c9-a65e924585ca None None] MFA Rules not processed for user `fd836cf590a145df9acf19cf3ae2ac56`. Rule list: `[]` (Enabled: `True`). {{(pid=88114) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:49:14.638326 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.fernet_utils [None req-dfc3cf01-95e8-4d85-83c9-a65e924585ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88114) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:49:14.648705 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 1069/2139] 10.222.0.13 () {58 vars in 958 bytes} [Wed Aug 31 09:49:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 70 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:49:14.649698 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 1071/2140] 10.222.0.13 () {58 vars in 1122 bytes} [Wed Aug 31 09:49:14 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 09:49:14.653828 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc3cf01-95e8-4d85-83c9-a65e924585ca None None] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:49:14.658124 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a19b37a-59ae-4014-a507-15a08aa0f20f admin admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:49:14.666358 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc3cf01-95e8-4d85-83c9-a65e924585ca None None] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:49:14.673439 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc51988d-1f2c-4f0b-b74d-1e3719d85d28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:49:14.674091 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc51988d-1f2c-4f0b-b74d-1e3719d85d28 admin admin] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:14.674471 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc51988d-1f2c-4f0b-b74d-1e3719d85d28 admin admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:14.674895 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a19b37a-59ae-4014-a507-15a08aa0f20f admin admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:49:14.675070 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc51988d-1f2c-4f0b-b74d-1e3719d85d28 admin admin] PATH_INFO: `/v3/domains` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:14.675670 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc51988d-1f2c-4f0b-b74d-1e3719d85d28 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88114) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:49:14.676817 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc51988d-1f2c-4f0b-b74d-1e3719d85d28 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:49:14.679484 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc51988d-1f2c-4f0b-b74d-1e3719d85d28 admin admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:49:14.683519 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12780848-bd37-4282-9048-850c5c6a2283 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:49:14.684572 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12780848-bd37-4282-9048-850c5c6a2283 admin admin] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:14.684957 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12780848-bd37-4282-9048-850c5c6a2283 admin admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:14.685250 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12780848-bd37-4282-9048-850c5c6a2283 admin admin] PATH_INFO: `/v3/roles` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:14.686162 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12780848-bd37-4282-9048-850c5c6a2283 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:49:14.688122 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12780848-bd37-4282-9048-850c5c6a2283 admin admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:49:14.698204 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 1070/2141] 10.222.0.13 () {58 vars in 1153 bytes} [Wed Aug 31 09:49:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 09:49:14.703055 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc51988d-1f2c-4f0b-b74d-1e3719d85d28 admin admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:49:14.717507 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 1072/2142] 10.222.0.13 () {58 vars in 1122 bytes} [Wed Aug 31 09:49:14 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 09:49:14.719426 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12780848-bd37-4282-9048-850c5c6a2283 admin admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:49:14.728510 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc51988d-1f2c-4f0b-b74d-1e3719d85d28 admin admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:49:14.731687 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12780848-bd37-4282-9048-850c5c6a2283 admin admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:49:14.738549 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb0ee5dd-69db-4596-b676-1c35bbd1fc64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:49:14.739237 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0ee5dd-69db-4596-b676-1c35bbd1fc64 admin admin] REQUEST_METHOD: `PUT` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:14.739511 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0ee5dd-69db-4596-b676-1c35bbd1fc64 admin admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:14.739776 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0ee5dd-69db-4596-b676-1c35bbd1fc64 admin admin] PATH_INFO: `/v3/projects/722c32420cf54ef2b9357d78ecbe3460/users/1a2cb7562b3d4332a5ac181e49236fc3/roles/6c99f5d9f1f243eaa134d8540f21095d` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:14.741291 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d1c0452-328a-46d0-adea-78703a16433d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:49:14.742891 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1c0452-328a-46d0-adea-78703a16433d admin admin] REQUEST_METHOD: `POST` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:14.744841 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1c0452-328a-46d0-adea-78703a16433d admin admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:14.745565 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1c0452-328a-46d0-adea-78703a16433d admin admin] PATH_INFO: `/v3/projects` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:14.747090 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d1c0452-328a-46d0-adea-78703a16433d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:49:14.754362 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d1c0452-328a-46d0-adea-78703a16433d admin admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:49:14.760124 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb0ee5dd-69db-4596-b676-1c35bbd1fc64 admin admin] RBAC: Authorizing `identity:create_grant(project_id=722c32420cf54ef2b9357d78ecbe3460, user_id=1a2cb7562b3d4332a5ac181e49236fc3, role_id=6c99f5d9f1f243eaa134d8540f21095d)` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:49:14.763126 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb0ee5dd-69db-4596-b676-1c35bbd1fc64 admin admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:49:14.787032 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 1073/2143] 10.222.0.13 () {60 vars in 1483 bytes} [Wed Aug 31 09:49:14 2022] PUT /identity/v3/projects/722c32420cf54ef2b9357d78ecbe3460/users/1a2cb7562b3d4332a5ac181e49236fc3/roles/6c99f5d9f1f243eaa134d8540f21095d => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 09:49:14.805799 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45dd09ae-09b0-4b03-9e77-d0fc09bc4b07 None None] REQUEST_METHOD: `POST` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:14.806720 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45dd09ae-09b0-4b03-9e77-d0fc09bc4b07 None None] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:14.807232 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45dd09ae-09b0-4b03-9e77-d0fc09bc4b07 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:14.815602 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 1071/2144] 10.222.0.13 () {60 vars in 1153 bytes} [Wed Aug 31 09:49:14 2022] POST /identity/v3/projects => generated 387 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 09:49:14.831330 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d1c0452-328a-46d0-adea-78703a16433d admin admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:49:14.856395 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=86,keystone:DELETE=11,keystone:INSERT=3 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:49:14.876757 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.auth.core [None req-45dd09ae-09b0-4b03-9e77-d0fc09bc4b07 None None] MFA Rules not processed for user `1a2cb7562b3d4332a5ac181e49236fc3`. Rule list: `[]` (Enabled: `True`). {{(pid=88113) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:49:14.880315 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.fernet_utils [None req-45dd09ae-09b0-4b03-9e77-d0fc09bc4b07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:49:14.919469 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d1c0452-328a-46d0-adea-78703a16433d admin admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:49:14.925306 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e549f4b0-3a52-4fb6-af38-a9f55fc03fd2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:49:14.926055 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e549f4b0-3a52-4fb6-af38-a9f55fc03fd2 admin admin] REQUEST_METHOD: `POST` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:14.926404 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e549f4b0-3a52-4fb6-af38-a9f55fc03fd2 admin admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:14.926742 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e549f4b0-3a52-4fb6-af38-a9f55fc03fd2 admin admin] PATH_INFO: `/v3/users` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:14.927395 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e549f4b0-3a52-4fb6-af38-a9f55fc03fd2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:49:14.929345 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e549f4b0-3a52-4fb6-af38-a9f55fc03fd2 admin admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:49:14.964693 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 1072/2145] 10.222.0.13 () {60 vars in 1144 bytes} [Wed Aug 31 09:49:14 2022] POST /identity/v3/users => generated 294 bytes in 142 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 09:49:14.966227 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e549f4b0-3a52-4fb6-af38-a9f55fc03fd2 admin admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:49:14.972473 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 1074/2146] 10.222.0.13 () {58 vars in 958 bytes} [Wed Aug 31 09:49:14 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 09:49:14.980868 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e549f4b0-3a52-4fb6-af38-a9f55fc03fd2 admin admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:49:14.981054 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45dd09ae-09b0-4b03-9e77-d0fc09bc4b07 None None] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:49:14.982471 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.fernet_utils [None req-45dd09ae-09b0-4b03-9e77-d0fc09bc4b07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:49:14.985947 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64fcf63c-c243-4029-8bd0-26975d3bef5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:49:14.987300 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64fcf63c-c243-4029-8bd0-26975d3bef5b admin admin] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:14.987540 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64fcf63c-c243-4029-8bd0-26975d3bef5b admin admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:14.987652 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64fcf63c-c243-4029-8bd0-26975d3bef5b admin admin] PATH_INFO: `/v3/roles` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:14.988267 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64fcf63c-c243-4029-8bd0-26975d3bef5b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:49:14.989920 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64fcf63c-c243-4029-8bd0-26975d3bef5b admin admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:49:15.001212 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 1073/2147] 10.222.0.13 () {58 vars in 1122 bytes} [Wed Aug 31 09:49:14 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 09:49:15.006907 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64fcf63c-c243-4029-8bd0-26975d3bef5b admin admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:49:15.018833 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64fcf63c-c243-4029-8bd0-26975d3bef5b admin admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:49:15.024765 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf65a074-8b84-44df-83e2-1deeae1a177a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:49:15.025468 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf65a074-8b84-44df-83e2-1deeae1a177a admin admin] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:15.025746 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf65a074-8b84-44df-83e2-1deeae1a177a admin admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:15.026015 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf65a074-8b84-44df-83e2-1deeae1a177a admin admin] PATH_INFO: `/v3/roles` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:15.026662 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf65a074-8b84-44df-83e2-1deeae1a177a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:49:15.029140 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf65a074-8b84-44df-83e2-1deeae1a177a admin admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:49:15.044209 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 1074/2148] 10.222.0.13 () {58 vars in 1122 bytes} [Wed Aug 31 09:49:15 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 09:49:15.044942 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:49:15.048668 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf65a074-8b84-44df-83e2-1deeae1a177a admin admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:49:15.060962 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf65a074-8b84-44df-83e2-1deeae1a177a admin admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:49:15.066726 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e510404-8c1c-4b7f-834c-7982f2836877 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:49:15.067430 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e510404-8c1c-4b7f-834c-7982f2836877 admin admin] REQUEST_METHOD: `PUT` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:15.067713 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e510404-8c1c-4b7f-834c-7982f2836877 admin admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:15.068016 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e510404-8c1c-4b7f-834c-7982f2836877 admin admin] PATH_INFO: `/v3/projects/85a73a65ee8a472f8d2a005a15c6d2ab/users/fa4c596675a14f4a89070ee31e83c928/roles/6c99f5d9f1f243eaa134d8540f21095d` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:15.086776 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45dd09ae-09b0-4b03-9e77-d0fc09bc4b07 None None] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:49:15.086990 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e510404-8c1c-4b7f-834c-7982f2836877 admin admin] RBAC: Authorizing `identity:create_grant(project_id=85a73a65ee8a472f8d2a005a15c6d2ab, user_id=fa4c596675a14f4a89070ee31e83c928, role_id=6c99f5d9f1f243eaa134d8540f21095d)` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:49:15.089973 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e510404-8c1c-4b7f-834c-7982f2836877 admin admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:49:15.092434 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ed4261-e11b-49fb-961a-5001f16ad948 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be216d156b54452cbbf978c7734343aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '49548eb44d62415582af820ab66decee', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:49:15.093536 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ed4261-e11b-49fb-961a-5001f16ad948 service glance] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:15.094027 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ed4261-e11b-49fb-961a-5001f16ad948 service glance] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:15.094490 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ed4261-e11b-49fb-961a-5001f16ad948 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:15.105473 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 1075/2149] 10.222.0.13 () {60 vars in 1483 bytes} [Wed Aug 31 09:49:15 2022] PUT /identity/v3/projects/85a73a65ee8a472f8d2a005a15c6d2ab/users/fa4c596675a14f4a89070ee31e83c928/roles/6c99f5d9f1f243eaa134d8540f21095d => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 09:49:15.113404 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79ed4261-e11b-49fb-961a-5001f16ad948 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:49:15.114891 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79ed4261-e11b-49fb-961a-5001f16ad948 service glance] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:49:15.124008 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d503f6-886f-448a-803b-ace58920937e None None] REQUEST_METHOD: `POST` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:15.124289 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d503f6-886f-448a-803b-ace58920937e None None] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:15.124566 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d503f6-886f-448a-803b-ace58920937e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:15.129690 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 1075/2150] 10.222.0.13 () {60 vars in 1365 bytes} [Wed Aug 31 09:49:14 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 149 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 09:49:15.157092 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.auth.core [None req-01d503f6-886f-448a-803b-ace58920937e None None] MFA Rules not processed for user `fa4c596675a14f4a89070ee31e83c928`. Rule list: `[]` (Enabled: `True`). {{(pid=88114) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:49:15.158401 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.fernet_utils [None req-01d503f6-886f-448a-803b-ace58920937e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88114) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:49:15.260778 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 1076/2151] 10.222.0.13 () {58 vars in 958 bytes} [Wed Aug 31 09:49:15 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 09:49:15.265935 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ed4261-e11b-49fb-961a-5001f16ad948 service glance] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:49:15.282438 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ed4261-e11b-49fb-961a-5001f16ad948 service glance] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:49:15.376935 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-262236e6-e998-48bd-bb3b-0d352058ed86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:49:15.377780 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262236e6-e998-48bd-bb3b-0d352058ed86 admin admin] REQUEST_METHOD: `POST` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:15.378199 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262236e6-e998-48bd-bb3b-0d352058ed86 admin admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:15.378540 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262236e6-e998-48bd-bb3b-0d352058ed86 admin admin] PATH_INFO: `/v3/projects` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:15.379239 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-262236e6-e998-48bd-bb3b-0d352058ed86 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:49:15.381227 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-262236e6-e998-48bd-bb3b-0d352058ed86 admin admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:49:15.418759 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 1076/2152] 10.222.0.13 () {60 vars in 1153 bytes} [Wed Aug 31 09:49:15 2022] POST /identity/v3/projects => generated 387 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 09:49:15.423530 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d503f6-886f-448a-803b-ace58920937e None None] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:49:15.439709 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d503f6-886f-448a-803b-ace58920937e None None] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:49:15.496431 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:49:15.529954 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b64186ac-99e1-46cf-9f0f-579876754721 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:49:15.530750 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64186ac-99e1-46cf-9f0f-579876754721 admin admin] REQUEST_METHOD: `POST` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:15.531023 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64186ac-99e1-46cf-9f0f-579876754721 admin admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:15.531425 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64186ac-99e1-46cf-9f0f-579876754721 admin admin] PATH_INFO: `/v3/users` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:15.532081 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b64186ac-99e1-46cf-9f0f-579876754721 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:49:15.535118 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b64186ac-99e1-46cf-9f0f-579876754721 admin admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:49:15.564071 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 1077/2153] 10.222.0.13 () {60 vars in 1144 bytes} [Wed Aug 31 09:49:15 2022] POST /identity/v3/users => generated 300 bytes in 141 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 09:49:15.568224 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-262236e6-e998-48bd-bb3b-0d352058ed86 admin admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:49:15.579974 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-262236e6-e998-48bd-bb3b-0d352058ed86 admin admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:49:15.587046 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f13468d-4c2d-427e-a390-91077b848648 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:49:15.587727 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f13468d-4c2d-427e-a390-91077b848648 admin admin] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:15.590016 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f13468d-4c2d-427e-a390-91077b848648 admin admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:15.590016 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f13468d-4c2d-427e-a390-91077b848648 admin admin] PATH_INFO: `/v3/roles` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:15.590016 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f13468d-4c2d-427e-a390-91077b848648 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:49:15.590677 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f13468d-4c2d-427e-a390-91077b848648 admin admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:49:15.613136 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 1077/2154] 10.222.0.13 () {58 vars in 1122 bytes} [Wed Aug 31 09:49:15 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 09:49:15.618043 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b64186ac-99e1-46cf-9f0f-579876754721 admin admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:49:15.647879 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b64186ac-99e1-46cf-9f0f-579876754721 admin admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:49:15.666964 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c29f7b9-f942-4642-9473-dbf1095e462c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:49:15.667709 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c29f7b9-f942-4642-9473-dbf1095e462c admin admin] REQUEST_METHOD: `PUT` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:15.668000 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c29f7b9-f942-4642-9473-dbf1095e462c admin admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:15.668342 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c29f7b9-f942-4642-9473-dbf1095e462c admin admin] PATH_INFO: `/v3/projects/5d7862f095544960a113dfc8c5e70f06/users/1a72028b15c9448ca2c142e61016772e/roles/5d4ee63a3add47fc9a6b8b16d39e0a1f` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:15.720165 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c29f7b9-f942-4642-9473-dbf1095e462c admin admin] RBAC: Authorizing `identity:create_grant(project_id=5d7862f095544960a113dfc8c5e70f06, user_id=1a72028b15c9448ca2c142e61016772e, role_id=5d4ee63a3add47fc9a6b8b16d39e0a1f)` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:49:15.726684 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c29f7b9-f942-4642-9473-dbf1095e462c admin admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:49:15.772058 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 1078/2155] 10.222.0.13 () {60 vars in 1483 bytes} [Wed Aug 31 09:49:15 2022] PUT /identity/v3/projects/5d7862f095544960a113dfc8c5e70f06/users/1a72028b15c9448ca2c142e61016772e/roles/5d4ee63a3add47fc9a6b8b16d39e0a1f => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:49:15.773447 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f13468d-4c2d-427e-a390-91077b848648 admin admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:49:15.792167 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f13468d-4c2d-427e-a390-91077b848648 admin admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:49:15.942136 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d7e525-f282-49a9-9165-ee6619ee0692 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:49:15.942940 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d7e525-f282-49a9-9165-ee6619ee0692 admin admin] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:15.943270 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d7e525-f282-49a9-9165-ee6619ee0692 admin admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:15.944279 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d7e525-f282-49a9-9165-ee6619ee0692 admin admin] PATH_INFO: `/v3/roles` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:15.944406 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35d7e525-f282-49a9-9165-ee6619ee0692 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:49:15.946333 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35d7e525-f282-49a9-9165-ee6619ee0692 admin admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:49:15.965494 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 1078/2156] 10.222.0.13 () {58 vars in 1122 bytes} [Wed Aug 31 09:49:15 2022] GET /identity/v3/roles => generated 1420 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 09:49:15.969147 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c29f7b9-f942-4642-9473-dbf1095e462c admin admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:49:15.994718 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c29f7b9-f942-4642-9473-dbf1095e462c admin admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:49:16.004262 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-890401d9-10aa-41be-8e8c-98d3f9717506 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:49:16.005473 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890401d9-10aa-41be-8e8c-98d3f9717506 admin admin] REQUEST_METHOD: `PUT` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:16.005940 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890401d9-10aa-41be-8e8c-98d3f9717506 admin admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:16.006388 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890401d9-10aa-41be-8e8c-98d3f9717506 admin admin] PATH_INFO: `/v3/projects/5d7862f095544960a113dfc8c5e70f06/users/1a72028b15c9448ca2c142e61016772e/roles/5d4ee63a3add47fc9a6b8b16d39e0a1f` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:16.010495 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-890401d9-10aa-41be-8e8c-98d3f9717506 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5d7862f095544960a113dfc8c5e70f06, user_id=1a72028b15c9448ca2c142e61016772e, role_id=5d4ee63a3add47fc9a6b8b16d39e0a1f)` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:49:16.016034 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-890401d9-10aa-41be-8e8c-98d3f9717506 admin admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:49:16.046267 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 1079/2157] 10.222.0.13 () {60 vars in 1483 bytes} [Wed Aug 31 09:49:15 2022] PUT /identity/v3/projects/5d7862f095544960a113dfc8c5e70f06/users/1a72028b15c9448ca2c142e61016772e/roles/5d4ee63a3add47fc9a6b8b16d39e0a1f => generated 0 bytes in 78 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:49:16.059794 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019bc618-6e31-4613-9970-290fdd394372 None None] REQUEST_METHOD: `POST` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:16.060633 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019bc618-6e31-4613-9970-290fdd394372 None None] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:16.060975 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019bc618-6e31-4613-9970-290fdd394372 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:16.106629 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.auth.core [None req-019bc618-6e31-4613-9970-290fdd394372 None None] MFA Rules not processed for user `1a72028b15c9448ca2c142e61016772e`. Rule list: `[]` (Enabled: `True`). {{(pid=88113) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:49:16.107887 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.fernet_utils [None req-019bc618-6e31-4613-9970-290fdd394372 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:49:16.149461 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:49:16.223260 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 1079/2158] 10.222.0.13 () {58 vars in 958 bytes} [Wed Aug 31 09:49:16 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:49:16.238974 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-890401d9-10aa-41be-8e8c-98d3f9717506 admin admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:49:16.242562 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.fernet_utils [None req-890401d9-10aa-41be-8e8c-98d3f9717506 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88114) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:49:16.317854 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-019bc618-6e31-4613-9970-290fdd394372 None None] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:49:16.321664 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.fernet_utils [None req-019bc618-6e31-4613-9970-290fdd394372 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:49:16.430062 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-890401d9-10aa-41be-8e8c-98d3f9717506 admin admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:49:16.437933 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8da4bff-3a90-46b2-a925-6140c56056c2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d52b69c8e94441dacfcff944326f169', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '49548eb44d62415582af820ab66decee', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:49:16.439469 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8da4bff-3a90-46b2-a925-6140c56056c2 service cinder] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:16.442104 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8da4bff-3a90-46b2-a925-6140c56056c2 service cinder] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:16.442904 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8da4bff-3a90-46b2-a925-6140c56056c2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:16.463131 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8da4bff-3a90-46b2-a925-6140c56056c2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:49:16.466268 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8da4bff-3a90-46b2-a925-6140c56056c2 service cinder] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:49:16.491397 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 1080/2159] 10.222.0.13 () {60 vars in 1365 bytes} [Wed Aug 31 09:49:16 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 253 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:49:16.504273 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-019bc618-6e31-4613-9970-290fdd394372 None None] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:49:16.521067 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-505302d6-ec92-4c1b-a3f1-0629a834ace9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:49:16.521884 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505302d6-ec92-4c1b-a3f1-0629a834ace9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:16.523252 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505302d6-ec92-4c1b-a3f1-0629a834ace9 admin admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:16.524695 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505302d6-ec92-4c1b-a3f1-0629a834ace9 admin admin] PATH_INFO: `/v3/users/53f4390bd6b7419b892997cb01755d16` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:16.528932 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-505302d6-ec92-4c1b-a3f1-0629a834ace9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=53f4390bd6b7419b892997cb01755d16)` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:49:16.531785 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-505302d6-ec92-4c1b-a3f1-0629a834ace9 admin admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:49:16.649359 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.notifications [None req-505302d6-ec92-4c1b-a3f1-0629a834ace9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '53f4390bd6b7419b892997cb01755d16'} {{(pid=88113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:49:16.664543 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.notifications [None req-505302d6-ec92-4c1b-a3f1-0629a834ace9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '53f4390bd6b7419b892997cb01755d16'} {{(pid=88113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:49:16.700199 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.notifications [None req-505302d6-ec92-4c1b-a3f1-0629a834ace9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '53f4390bd6b7419b892997cb01755d16'} {{(pid=88113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:49:16.701590 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.notifications [None req-505302d6-ec92-4c1b-a3f1-0629a834ace9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '53f4390bd6b7419b892997cb01755d16'} {{(pid=88113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:49:16.730350 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.notifications [None req-505302d6-ec92-4c1b-a3f1-0629a834ace9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '53f4390bd6b7419b892997cb01755d16'} {{(pid=88113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:49:16.748210 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 1080/2160] 10.222.0.13 () {58 vars in 1224 bytes} [Wed Aug 31 09:49:16 2022] DELETE /identity/v3/users/53f4390bd6b7419b892997cb01755d16 => generated 0 bytes in 431 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:49:16.753459 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18a804c-ff89-4722-8abb-d528fde7838a None None] REQUEST_METHOD: `POST` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:16.754050 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18a804c-ff89-4722-8abb-d528fde7838a None None] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:16.754260 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18a804c-ff89-4722-8abb-d528fde7838a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:16.794094 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.auth.core [None req-f18a804c-ff89-4722-8abb-d528fde7838a None None] MFA Rules not processed for user `fd836cf590a145df9acf19cf3ae2ac56`. Rule list: `[]` (Enabled: `True`). {{(pid=88114) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:49:16.796294 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.fernet_utils [None req-f18a804c-ff89-4722-8abb-d528fde7838a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88114) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:49:16.811556 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-505302d6-ec92-4c1b-a3f1-0629a834ace9 admin admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:49:16.815083 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.fernet_utils [None req-505302d6-ec92-4c1b-a3f1-0629a834ace9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:49:16.959594 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:49:16.960101 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 1081/2161] 10.222.0.13 () {58 vars in 958 bytes} [Wed Aug 31 09:49:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:49:16.977600 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f18a804c-ff89-4722-8abb-d528fde7838a None None] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:49:16.981540 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.fernet_utils [None req-f18a804c-ff89-4722-8abb-d528fde7838a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88114) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:49:16.996507 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-505302d6-ec92-4c1b-a3f1-0629a834ace9 admin admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:49:17.002558 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-627e7a63-1a68-4864-a5a8-c68b4253c696 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:49:17.003620 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627e7a63-1a68-4864-a5a8-c68b4253c696 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:17.003865 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627e7a63-1a68-4864-a5a8-c68b4253c696 admin admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:17.004153 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627e7a63-1a68-4864-a5a8-c68b4253c696 admin admin] PATH_INFO: `/v3/users/fa4c596675a14f4a89070ee31e83c928` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:17.012005 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-627e7a63-1a68-4864-a5a8-c68b4253c696 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fa4c596675a14f4a89070ee31e83c928)` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:49:17.017991 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-627e7a63-1a68-4864-a5a8-c68b4253c696 admin admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:49:17.128111 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:49:17.146880 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.notifications [None req-627e7a63-1a68-4864-a5a8-c68b4253c696 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fa4c596675a14f4a89070ee31e83c928'} {{(pid=88113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:49:17.165211 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.notifications [None req-627e7a63-1a68-4864-a5a8-c68b4253c696 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fa4c596675a14f4a89070ee31e83c928'} {{(pid=88113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:49:17.166294 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f18a804c-ff89-4722-8abb-d528fde7838a None None] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:49:17.176997 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d858a4bd-fc94-4799-a1e1-062904b448b7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9bdaacd597442c2bcde2c555f2cc140', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '49548eb44d62415582af820ab66decee', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:49:17.178360 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d858a4bd-fc94-4799-a1e1-062904b448b7 service neutron] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:17.179029 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d858a4bd-fc94-4799-a1e1-062904b448b7 service neutron] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:17.179763 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d858a4bd-fc94-4799-a1e1-062904b448b7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:17.183239 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.notifications [None req-627e7a63-1a68-4864-a5a8-c68b4253c696 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fa4c596675a14f4a89070ee31e83c928'} {{(pid=88113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:49:17.184187 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.notifications [None req-627e7a63-1a68-4864-a5a8-c68b4253c696 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fa4c596675a14f4a89070ee31e83c928'} {{(pid=88113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:49:17.196216 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.notifications [None req-627e7a63-1a68-4864-a5a8-c68b4253c696 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fa4c596675a14f4a89070ee31e83c928'} {{(pid=88113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:49:17.197331 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d858a4bd-fc94-4799-a1e1-062904b448b7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:49:17.199357 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d858a4bd-fc94-4799-a1e1-062904b448b7 service neutron] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:49:17.210033 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 1081/2162] 10.222.0.13 () {58 vars in 1224 bytes} [Wed Aug 31 09:49:16 2022] DELETE /identity/v3/users/fa4c596675a14f4a89070ee31e83c928 => generated 0 bytes in 400 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:49:17.216514 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402224db-fd40-4668-b6ab-8b9af4d422f9 None None] REQUEST_METHOD: `POST` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:17.217053 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402224db-fd40-4668-b6ab-8b9af4d422f9 None None] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:17.218604 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402224db-fd40-4668-b6ab-8b9af4d422f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:17.228703 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 1082/2163] 10.222.0.13 () {60 vars in 1365 bytes} [Wed Aug 31 09:49:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 252 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 09:49:17.246627 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.auth.core [None req-402224db-fd40-4668-b6ab-8b9af4d422f9 None None] MFA Rules not processed for user `fd836cf590a145df9acf19cf3ae2ac56`. Rule list: `[]` (Enabled: `True`). {{(pid=88113) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:49:17.248078 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.fernet_utils [None req-402224db-fd40-4668-b6ab-8b9af4d422f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:49:17.355000 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 1082/2164] 10.222.0.13 () {58 vars in 958 bytes} [Wed Aug 31 09:49:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:49:17.383643 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d858a4bd-fc94-4799-a1e1-062904b448b7 service neutron] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:49:17.385331 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.fernet_utils [None req-d858a4bd-fc94-4799-a1e1-062904b448b7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88114) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:49:17.401787 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d858a4bd-fc94-4799-a1e1-062904b448b7 service neutron] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:49:17.582224 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd48cd3-68f8-47d2-bf6e-bace7982ce95 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9bdaacd597442c2bcde2c555f2cc140', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '49548eb44d62415582af820ab66decee', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:49:17.583120 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd48cd3-68f8-47d2-bf6e-bace7982ce95 service neutron] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:17.583676 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd48cd3-68f8-47d2-bf6e-bace7982ce95 service neutron] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:17.584422 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd48cd3-68f8-47d2-bf6e-bace7982ce95 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:17.603812 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bd48cd3-68f8-47d2-bf6e-bace7982ce95 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:49:17.607056 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bd48cd3-68f8-47d2-bf6e-bace7982ce95 service neutron] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:49:17.609532 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-402224db-fd40-4668-b6ab-8b9af4d422f9 None None] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:49:17.611917 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.fernet_utils [None req-402224db-fd40-4668-b6ab-8b9af4d422f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:49:17.651743 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 1083/2165] 10.222.0.13 () {60 vars in 1365 bytes} [Wed Aug 31 09:49:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 268 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:49:17.656844 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-402224db-fd40-4668-b6ab-8b9af4d422f9 None None] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:49:17.663125 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b651c585-bac9-4af9-99ff-d0d6c90ea294 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:49:17.663879 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b651c585-bac9-4af9-99ff-d0d6c90ea294 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:17.665301 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b651c585-bac9-4af9-99ff-d0d6c90ea294 admin admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:17.665626 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b651c585-bac9-4af9-99ff-d0d6c90ea294 admin admin] PATH_INFO: `/v3/projects/2eb58e59f570457b955ec9fde3af6944` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:17.667276 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b651c585-bac9-4af9-99ff-d0d6c90ea294 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2eb58e59f570457b955ec9fde3af6944)` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:49:17.670864 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b651c585-bac9-4af9-99ff-d0d6c90ea294 admin admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:49:17.766459 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.notifications [None req-b651c585-bac9-4af9-99ff-d0d6c90ea294 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2eb58e59f570457b955ec9fde3af6944', 'request_id': 'req-b651c585-bac9-4af9-99ff-d0d6c90ea294'} {{(pid=88113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:49:17.772730 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=83,keystone:DELETE=16,keystone:INSERT=1 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:49:17.774690 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.notifications [None req-b651c585-bac9-4af9-99ff-d0d6c90ea294 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2eb58e59f570457b955ec9fde3af6944', 'request_id': 'req-b651c585-bac9-4af9-99ff-d0d6c90ea294'} {{(pid=88113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:49:17.787091 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.notifications [None req-b651c585-bac9-4af9-99ff-d0d6c90ea294 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2eb58e59f570457b955ec9fde3af6944', 'request_id': 'req-b651c585-bac9-4af9-99ff-d0d6c90ea294'} {{(pid=88113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:49:17.812851 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.notifications [None req-b651c585-bac9-4af9-99ff-d0d6c90ea294 admin admin] The token cache is being invalidate because project 2eb58e59f570457b955ec9fde3af6944 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88113) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 09:49:17.813368 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.notifications [None req-b651c585-bac9-4af9-99ff-d0d6c90ea294 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:49:17.815598 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.notifications [None req-b651c585-bac9-4af9-99ff-d0d6c90ea294 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:49:17.818580 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 1083/2166] 10.222.0.13 () {58 vars in 1233 bytes} [Wed Aug 31 09:49:17 2022] DELETE /identity/v3/projects/2eb58e59f570457b955ec9fde3af6944 => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 09:49:17.822926 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd48cd3-68f8-47d2-bf6e-bace7982ce95 service neutron] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:49:17.838098 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd48cd3-68f8-47d2-bf6e-bace7982ce95 service neutron] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:49:17.933406 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-785f0a56-e5f9-4f62-9c42-7df62a7f81ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:49:17.934070 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785f0a56-e5f9-4f62-9c42-7df62a7f81ad admin admin] REQUEST_METHOD: `DELETE` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:17.934347 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785f0a56-e5f9-4f62-9c42-7df62a7f81ad admin admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:17.934623 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785f0a56-e5f9-4f62-9c42-7df62a7f81ad admin admin] PATH_INFO: `/v3/users/473970514dab4dbb9a046737ecc73b7e` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:17.935867 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-785f0a56-e5f9-4f62-9c42-7df62a7f81ad admin admin] RBAC: Authorizing `identity:delete_user(user_id=473970514dab4dbb9a046737ecc73b7e)` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:49:17.938043 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-785f0a56-e5f9-4f62-9c42-7df62a7f81ad admin admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:49:18.024871 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.notifications [None req-785f0a56-e5f9-4f62-9c42-7df62a7f81ad admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '473970514dab4dbb9a046737ecc73b7e'} {{(pid=88114) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:49:18.034515 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.notifications [None req-785f0a56-e5f9-4f62-9c42-7df62a7f81ad admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '473970514dab4dbb9a046737ecc73b7e'} {{(pid=88114) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:49:18.046389 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.notifications [None req-785f0a56-e5f9-4f62-9c42-7df62a7f81ad admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '473970514dab4dbb9a046737ecc73b7e'} {{(pid=88114) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:49:18.047392 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.notifications [None req-785f0a56-e5f9-4f62-9c42-7df62a7f81ad admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '473970514dab4dbb9a046737ecc73b7e'} {{(pid=88114) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:49:18.065222 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.notifications [None req-785f0a56-e5f9-4f62-9c42-7df62a7f81ad admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '473970514dab4dbb9a046737ecc73b7e'} {{(pid=88114) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:49:18.068602 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:49:18.075730 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 1084/2167] 10.222.0.13 () {58 vars in 1224 bytes} [Wed Aug 31 09:49:17 2022] DELETE /identity/v3/users/473970514dab4dbb9a046737ecc73b7e => generated 0 bytes in 253 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 09:49:18.534049 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b651c585-bac9-4af9-99ff-d0d6c90ea294 admin admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:49:18.538404 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.fernet_utils [None req-b651c585-bac9-4af9-99ff-d0d6c90ea294 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:49:18.598214 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b651c585-bac9-4af9-99ff-d0d6c90ea294 admin admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:49:18.665227 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-785f0a56-e5f9-4f62-9c42-7df62a7f81ad admin admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:49:18.668085 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.fernet_utils [None req-785f0a56-e5f9-4f62-9c42-7df62a7f81ad admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88114) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:49:18.764562 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef92816-3b0c-4c59-9c27-56001538a2e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:49:18.765617 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef92816-3b0c-4c59-9c27-56001538a2e8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:18.766089 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef92816-3b0c-4c59-9c27-56001538a2e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:18.766689 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef92816-3b0c-4c59-9c27-56001538a2e8 admin admin] PATH_INFO: `/v3/projects/047132299e3a41c4b1e610a19666b929` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:18.769166 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bef92816-3b0c-4c59-9c27-56001538a2e8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=047132299e3a41c4b1e610a19666b929)` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:49:18.772499 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bef92816-3b0c-4c59-9c27-56001538a2e8 admin admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:49:18.850189 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-785f0a56-e5f9-4f62-9c42-7df62a7f81ad admin admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:49:18.856441 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c1531f4-f81b-4164-b132-dbab115465a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:49:18.857584 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1531f4-f81b-4164-b132-dbab115465a8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:18.858108 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1531f4-f81b-4164-b132-dbab115465a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:18.858645 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1531f4-f81b-4164-b132-dbab115465a8 admin admin] PATH_INFO: `/v3/projects/85a73a65ee8a472f8d2a005a15c6d2ab` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:18.862051 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c1531f4-f81b-4164-b132-dbab115465a8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=85a73a65ee8a472f8d2a005a15c6d2ab)` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:49:18.865571 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c1531f4-f81b-4164-b132-dbab115465a8 admin admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:49:18.943736 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.notifications [None req-bef92816-3b0c-4c59-9c27-56001538a2e8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '047132299e3a41c4b1e610a19666b929', 'request_id': 'req-bef92816-3b0c-4c59-9c27-56001538a2e8'} {{(pid=88113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:49:18.963765 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.notifications [None req-bef92816-3b0c-4c59-9c27-56001538a2e8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '047132299e3a41c4b1e610a19666b929', 'request_id': 'req-bef92816-3b0c-4c59-9c27-56001538a2e8'} {{(pid=88113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:49:18.990806 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.notifications [None req-bef92816-3b0c-4c59-9c27-56001538a2e8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '047132299e3a41c4b1e610a19666b929', 'request_id': 'req-bef92816-3b0c-4c59-9c27-56001538a2e8'} {{(pid=88113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:49:19.018673 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.notifications [None req-bef92816-3b0c-4c59-9c27-56001538a2e8 admin admin] The token cache is being invalidate because project 047132299e3a41c4b1e610a19666b929 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88113) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 09:49:19.019085 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.notifications [None req-bef92816-3b0c-4c59-9c27-56001538a2e8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:49:19.022664 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.notifications [None req-bef92816-3b0c-4c59-9c27-56001538a2e8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:49:19.023387 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 1084/2168] 10.222.0.13 () {58 vars in 1233 bytes} [Wed Aug 31 09:49:18 2022] DELETE /identity/v3/projects/047132299e3a41c4b1e610a19666b929 => generated 0 bytes in 490 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:49:19.027337 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef92816-3b0c-4c59-9c27-56001538a2e8 admin admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:49:19.028838 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.fernet_utils [None req-bef92816-3b0c-4c59-9c27-56001538a2e8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:49:19.043500 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.notifications [None req-5c1531f4-f81b-4164-b132-dbab115465a8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '85a73a65ee8a472f8d2a005a15c6d2ab', 'request_id': 'req-5c1531f4-f81b-4164-b132-dbab115465a8'} {{(pid=88114) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:49:19.055663 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.notifications [None req-5c1531f4-f81b-4164-b132-dbab115465a8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '85a73a65ee8a472f8d2a005a15c6d2ab', 'request_id': 'req-5c1531f4-f81b-4164-b132-dbab115465a8'} {{(pid=88114) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:49:19.076335 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.notifications [None req-5c1531f4-f81b-4164-b132-dbab115465a8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '85a73a65ee8a472f8d2a005a15c6d2ab', 'request_id': 'req-5c1531f4-f81b-4164-b132-dbab115465a8'} {{(pid=88114) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:49:19.108578 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.notifications [None req-5c1531f4-f81b-4164-b132-dbab115465a8 admin admin] The token cache is being invalidate because project 85a73a65ee8a472f8d2a005a15c6d2ab was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88114) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 09:49:19.109285 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.notifications [None req-5c1531f4-f81b-4164-b132-dbab115465a8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88114) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:49:19.110317 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.notifications [None req-5c1531f4-f81b-4164-b132-dbab115465a8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88114) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:49:19.113658 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 1085/2169] 10.222.0.13 () {58 vars in 1233 bytes} [Wed Aug 31 09:49:18 2022] DELETE /identity/v3/projects/85a73a65ee8a472f8d2a005a15c6d2ab => generated 0 bytes in 449 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:49:19.133739 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c1531f4-f81b-4164-b132-dbab115465a8 admin admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:49:19.140632 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.fernet_utils [None req-5c1531f4-f81b-4164-b132-dbab115465a8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88114) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:49:19.227942 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef92816-3b0c-4c59-9c27-56001538a2e8 admin admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:49:19.304053 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=90,keystone:INSERT=2,keystone:DELETE=8 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:49:19.406563 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c1531f4-f81b-4164-b132-dbab115465a8 admin admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:49:19.431337 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-400f15d7-1614-4bab-b1f6-adbcaef0eeee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:49:19.433347 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400f15d7-1614-4bab-b1f6-adbcaef0eeee admin admin] REQUEST_METHOD: `DELETE` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:19.434838 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400f15d7-1614-4bab-b1f6-adbcaef0eeee admin admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:19.438313 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400f15d7-1614-4bab-b1f6-adbcaef0eeee admin admin] PATH_INFO: `/v3/users/1a72028b15c9448ca2c142e61016772e` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:19.442182 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-400f15d7-1614-4bab-b1f6-adbcaef0eeee admin admin] RBAC: Authorizing `identity:delete_user(user_id=1a72028b15c9448ca2c142e61016772e)` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:49:19.456499 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-400f15d7-1614-4bab-b1f6-adbcaef0eeee admin admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:49:19.495842 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20dcea5b-3aad-440b-97ba-545bd11d91ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:49:19.502269 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20dcea5b-3aad-440b-97ba-545bd11d91ee admin admin] REQUEST_METHOD: `DELETE` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:19.504663 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20dcea5b-3aad-440b-97ba-545bd11d91ee admin admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:19.505421 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20dcea5b-3aad-440b-97ba-545bd11d91ee admin admin] PATH_INFO: `/v3/users/712beab5eea84a2a815f5bc4914c8558` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:19.507896 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20dcea5b-3aad-440b-97ba-545bd11d91ee admin admin] RBAC: Authorizing `identity:delete_user(user_id=712beab5eea84a2a815f5bc4914c8558)` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:49:19.518965 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20dcea5b-3aad-440b-97ba-545bd11d91ee admin admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:49:19.555424 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:49:19.609901 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.notifications [None req-400f15d7-1614-4bab-b1f6-adbcaef0eeee admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1a72028b15c9448ca2c142e61016772e'} {{(pid=88114) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:49:19.628189 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.notifications [None req-400f15d7-1614-4bab-b1f6-adbcaef0eeee admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1a72028b15c9448ca2c142e61016772e'} {{(pid=88114) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:49:19.639150 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.notifications [None req-20dcea5b-3aad-440b-97ba-545bd11d91ee admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '712beab5eea84a2a815f5bc4914c8558'} {{(pid=88113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:49:19.648008 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.notifications [None req-400f15d7-1614-4bab-b1f6-adbcaef0eeee admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1a72028b15c9448ca2c142e61016772e'} {{(pid=88114) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:49:19.650213 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.notifications [None req-400f15d7-1614-4bab-b1f6-adbcaef0eeee admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1a72028b15c9448ca2c142e61016772e'} {{(pid=88114) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:49:19.663814 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.notifications [None req-20dcea5b-3aad-440b-97ba-545bd11d91ee admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '712beab5eea84a2a815f5bc4914c8558'} {{(pid=88113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:49:19.678353 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.notifications [None req-400f15d7-1614-4bab-b1f6-adbcaef0eeee admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1a72028b15c9448ca2c142e61016772e'} {{(pid=88114) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:49:19.692884 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 1086/2170] 10.222.0.13 () {58 vars in 1224 bytes} [Wed Aug 31 09:49:19 2022] DELETE /identity/v3/users/1a72028b15c9448ca2c142e61016772e => generated 0 bytes in 575 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:49:19.700826 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.notifications [None req-20dcea5b-3aad-440b-97ba-545bd11d91ee admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '712beab5eea84a2a815f5bc4914c8558'} {{(pid=88113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:49:19.702585 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.notifications [None req-20dcea5b-3aad-440b-97ba-545bd11d91ee admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '712beab5eea84a2a815f5bc4914c8558'} {{(pid=88113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:49:19.714253 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.notifications [None req-20dcea5b-3aad-440b-97ba-545bd11d91ee admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '712beab5eea84a2a815f5bc4914c8558'} {{(pid=88113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:49:19.741707 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 1085/2171] 10.222.0.13 () {58 vars in 1224 bytes} [Wed Aug 31 09:49:19 2022] DELETE /identity/v3/users/712beab5eea84a2a815f5bc4914c8558 => generated 0 bytes in 712 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:49:20.263034 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-400f15d7-1614-4bab-b1f6-adbcaef0eeee admin admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:49:20.266917 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.fernet_utils [None req-400f15d7-1614-4bab-b1f6-adbcaef0eeee admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88114) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:49:20.387403 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-400f15d7-1614-4bab-b1f6-adbcaef0eeee admin admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:49:20.412109 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20dcea5b-3aad-440b-97ba-545bd11d91ee admin admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:49:20.430544 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.fernet_utils [None req-20dcea5b-3aad-440b-97ba-545bd11d91ee admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:49:20.512732 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6054d840-a439-4c9a-95a5-036ad69233d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:49:20.513526 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6054d840-a439-4c9a-95a5-036ad69233d2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:20.513739 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6054d840-a439-4c9a-95a5-036ad69233d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:20.514012 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6054d840-a439-4c9a-95a5-036ad69233d2 admin admin] PATH_INFO: `/v3/projects/49b47df28ff94283948257611a2fe330` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:20.515581 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6054d840-a439-4c9a-95a5-036ad69233d2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=49b47df28ff94283948257611a2fe330)` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:49:20.517824 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6054d840-a439-4c9a-95a5-036ad69233d2 admin admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:49:20.585071 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.notifications [None req-6054d840-a439-4c9a-95a5-036ad69233d2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '49b47df28ff94283948257611a2fe330', 'request_id': 'req-6054d840-a439-4c9a-95a5-036ad69233d2'} {{(pid=88114) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:49:20.594081 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20dcea5b-3aad-440b-97ba-545bd11d91ee admin admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:49:20.603187 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.notifications [None req-6054d840-a439-4c9a-95a5-036ad69233d2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '49b47df28ff94283948257611a2fe330', 'request_id': 'req-6054d840-a439-4c9a-95a5-036ad69233d2'} {{(pid=88114) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:49:20.609777 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:49:20.616069 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.notifications [None req-6054d840-a439-4c9a-95a5-036ad69233d2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '49b47df28ff94283948257611a2fe330', 'request_id': 'req-6054d840-a439-4c9a-95a5-036ad69233d2'} {{(pid=88114) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:49:20.640795 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.notifications [None req-6054d840-a439-4c9a-95a5-036ad69233d2 admin admin] The token cache is being invalidate because project 49b47df28ff94283948257611a2fe330 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88114) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 09:49:20.641220 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.notifications [None req-6054d840-a439-4c9a-95a5-036ad69233d2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88114) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:49:20.642255 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.notifications [None req-6054d840-a439-4c9a-95a5-036ad69233d2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88114) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:49:20.645881 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 1087/2172] 10.222.0.13 () {58 vars in 1233 bytes} [Wed Aug 31 09:49:20 2022] DELETE /identity/v3/projects/49b47df28ff94283948257611a2fe330 => generated 0 bytes in 383 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:49:20.651928 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce9b57a-9e36-428c-9901-652673720da9 None None] REQUEST_METHOD: `POST` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:20.652213 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce9b57a-9e36-428c-9901-652673720da9 None None] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:20.652493 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce9b57a-9e36-428c-9901-652673720da9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:20.680808 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.auth.core [None req-cce9b57a-9e36-428c-9901-652673720da9 None None] MFA Rules not processed for user `fd836cf590a145df9acf19cf3ae2ac56`. Rule list: `[]` (Enabled: `True`). {{(pid=88114) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:49:20.682917 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.fernet_utils [None req-cce9b57a-9e36-428c-9901-652673720da9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88114) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:49:20.697891 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:49:20.734166 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30dff964-31be-488f-9a90-a2a6e62e9232 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:49:20.737247 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30dff964-31be-488f-9a90-a2a6e62e9232 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:20.737791 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30dff964-31be-488f-9a90-a2a6e62e9232 admin admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:20.738289 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30dff964-31be-488f-9a90-a2a6e62e9232 admin admin] PATH_INFO: `/v3/projects/5d7862f095544960a113dfc8c5e70f06` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:20.740498 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30dff964-31be-488f-9a90-a2a6e62e9232 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5d7862f095544960a113dfc8c5e70f06)` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:49:20.746050 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30dff964-31be-488f-9a90-a2a6e62e9232 admin admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:49:20.793006 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 1088/2173] 10.222.0.13 () {58 vars in 958 bytes} [Wed Aug 31 09:49:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:49:20.808120 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498c2396-9a84-4fd8-861c-0a964dba1e24 None None] REQUEST_METHOD: `POST` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:20.808403 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498c2396-9a84-4fd8-861c-0a964dba1e24 None None] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:20.808702 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498c2396-9a84-4fd8-861c-0a964dba1e24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:20.851823 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.auth.core [None req-498c2396-9a84-4fd8-861c-0a964dba1e24 None None] MFA Rules not processed for user `fd836cf590a145df9acf19cf3ae2ac56`. Rule list: `[]` (Enabled: `True`). {{(pid=88114) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:49:20.853031 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.fernet_utils [None req-498c2396-9a84-4fd8-861c-0a964dba1e24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88114) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:49:20.871574 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.notifications [None req-30dff964-31be-488f-9a90-a2a6e62e9232 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5d7862f095544960a113dfc8c5e70f06', 'request_id': 'req-30dff964-31be-488f-9a90-a2a6e62e9232'} {{(pid=88113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:49:20.882489 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.notifications [None req-30dff964-31be-488f-9a90-a2a6e62e9232 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5d7862f095544960a113dfc8c5e70f06', 'request_id': 'req-30dff964-31be-488f-9a90-a2a6e62e9232'} {{(pid=88113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:49:20.896820 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.notifications [None req-30dff964-31be-488f-9a90-a2a6e62e9232 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5d7862f095544960a113dfc8c5e70f06', 'request_id': 'req-30dff964-31be-488f-9a90-a2a6e62e9232'} {{(pid=88113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:49:20.926877 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.notifications [None req-30dff964-31be-488f-9a90-a2a6e62e9232 admin admin] The token cache is being invalidate because project 5d7862f095544960a113dfc8c5e70f06 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88113) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 09:49:20.927820 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.notifications [None req-30dff964-31be-488f-9a90-a2a6e62e9232 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:49:20.930062 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.notifications [None req-30dff964-31be-488f-9a90-a2a6e62e9232 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:49:20.935478 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 1086/2174] 10.222.0.13 () {58 vars in 1233 bytes} [Wed Aug 31 09:49:20 2022] DELETE /identity/v3/projects/5d7862f095544960a113dfc8c5e70f06 => generated 0 bytes in 525 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:49:20.942697 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b98c2d-1ca5-4fd8-ac71-2637ae1e4452 None None] REQUEST_METHOD: `POST` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:20.942980 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b98c2d-1ca5-4fd8-ac71-2637ae1e4452 None None] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:20.943259 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b98c2d-1ca5-4fd8-ac71-2637ae1e4452 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:20.977353 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.auth.core [None req-f8b98c2d-1ca5-4fd8-ac71-2637ae1e4452 None None] MFA Rules not processed for user `fd836cf590a145df9acf19cf3ae2ac56`. Rule list: `[]` (Enabled: `True`). {{(pid=88113) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:49:20.979244 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.fernet_utils [None req-f8b98c2d-1ca5-4fd8-ac71-2637ae1e4452 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:49:21.010826 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 1089/2175] 10.222.0.13 () {58 vars in 958 bytes} [Wed Aug 31 09:49:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:49:21.016916 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-498c2396-9a84-4fd8-861c-0a964dba1e24 None None] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:49:21.046182 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-498c2396-9a84-4fd8-861c-0a964dba1e24 None None] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:49:21.057181 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6773a79-2c4e-4a2b-b979-fced6fe645ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:49:21.057852 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6773a79-2c4e-4a2b-b979-fced6fe645ab admin admin] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:21.058131 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6773a79-2c4e-4a2b-b979-fced6fe645ab admin admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:21.058404 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6773a79-2c4e-4a2b-b979-fced6fe645ab admin admin] PATH_INFO: `/v3/domains` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:21.059080 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6773a79-2c4e-4a2b-b979-fced6fe645ab admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88114) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:49:21.059381 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6773a79-2c4e-4a2b-b979-fced6fe645ab admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:49:21.061583 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6773a79-2c4e-4a2b-b979-fced6fe645ab admin admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:49:21.092867 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 1090/2176] 10.222.0.13 () {58 vars in 1153 bytes} [Wed Aug 31 09:49:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 09:49:21.098621 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6773a79-2c4e-4a2b-b979-fced6fe645ab admin admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:49:21.105523 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 1087/2177] 10.222.0.13 () {58 vars in 958 bytes} [Wed Aug 31 09:49:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:49:21.108035 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f5b576-576d-4b23-bbf5-51aba86e5847 None None] REQUEST_METHOD: `POST` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:21.108489 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f5b576-576d-4b23-bbf5-51aba86e5847 None None] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:21.108941 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f5b576-576d-4b23-bbf5-51aba86e5847 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:21.144527 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6773a79-2c4e-4a2b-b979-fced6fe645ab admin admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:49:21.152581 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e62e652-3a50-4b1e-9d0a-7474f3785537 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:49:21.153588 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e62e652-3a50-4b1e-9d0a-7474f3785537 admin admin] REQUEST_METHOD: `POST` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:21.154155 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e62e652-3a50-4b1e-9d0a-7474f3785537 admin admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:21.154744 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e62e652-3a50-4b1e-9d0a-7474f3785537 admin admin] PATH_INFO: `/v3/projects` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:21.155930 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e62e652-3a50-4b1e-9d0a-7474f3785537 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:49:21.156529 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.auth.core [None req-b8f5b576-576d-4b23-bbf5-51aba86e5847 None None] MFA Rules not processed for user `fd836cf590a145df9acf19cf3ae2ac56`. Rule list: `[]` (Enabled: `True`). {{(pid=88113) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:49:21.160325 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.fernet_utils [None req-b8f5b576-576d-4b23-bbf5-51aba86e5847 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:49:21.161711 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e62e652-3a50-4b1e-9d0a-7474f3785537 admin admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:49:21.178749 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 1088/2178] 10.222.0.13 () {58 vars in 958 bytes} [Wed Aug 31 09:49:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 72 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:49:21.183383 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f5b576-576d-4b23-bbf5-51aba86e5847 None None] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:49:21.201279 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f5b576-576d-4b23-bbf5-51aba86e5847 None None] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:49:21.210114 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff13d25b-05e0-466d-8725-a43f38659803 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:49:21.211206 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff13d25b-05e0-466d-8725-a43f38659803 admin admin] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:21.211732 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff13d25b-05e0-466d-8725-a43f38659803 admin admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:21.212216 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff13d25b-05e0-466d-8725-a43f38659803 admin admin] PATH_INFO: `/v3/domains` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:21.213766 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff13d25b-05e0-466d-8725-a43f38659803 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88113) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:49:21.214393 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff13d25b-05e0-466d-8725-a43f38659803 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:49:21.217586 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff13d25b-05e0-466d-8725-a43f38659803 admin admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:49:21.244587 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 1089/2179] 10.222.0.13 () {58 vars in 1153 bytes} [Wed Aug 31 09:49:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:49:21.246393 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff13d25b-05e0-466d-8725-a43f38659803 admin admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:49:21.253578 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 1091/2180] 10.222.0.13 () {60 vars in 1153 bytes} [Wed Aug 31 09:49:21 2022] POST /identity/v3/projects => generated 391 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 09:49:21.258574 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e62e652-3a50-4b1e-9d0a-7474f3785537 admin admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:49:21.270643 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff13d25b-05e0-466d-8725-a43f38659803 admin admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:49:21.352600 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:DELETE=2,keystone:SELECT=96,keystone:INSERT=2 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:49:21.367925 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24355397-7162-4764-a6ce-f8bc3abe6b78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:49:21.368649 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24355397-7162-4764-a6ce-f8bc3abe6b78 admin admin] REQUEST_METHOD: `POST` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:21.369113 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24355397-7162-4764-a6ce-f8bc3abe6b78 admin admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:21.369302 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24355397-7162-4764-a6ce-f8bc3abe6b78 admin admin] PATH_INFO: `/v3/projects` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:21.369669 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e62e652-3a50-4b1e-9d0a-7474f3785537 admin admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:49:21.370376 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24355397-7162-4764-a6ce-f8bc3abe6b78 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:49:21.372202 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24355397-7162-4764-a6ce-f8bc3abe6b78 admin admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:49:21.376379 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b6d9811-affc-4857-9be0-ee457b39843e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:49:21.377662 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6d9811-affc-4857-9be0-ee457b39843e admin admin] REQUEST_METHOD: `POST` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:21.377932 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6d9811-affc-4857-9be0-ee457b39843e admin admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:21.378193 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6d9811-affc-4857-9be0-ee457b39843e admin admin] PATH_INFO: `/v3/users` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:21.378393 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:49:21.379206 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b6d9811-affc-4857-9be0-ee457b39843e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:49:21.380838 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b6d9811-affc-4857-9be0-ee457b39843e admin admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:49:21.413696 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 1090/2181] 10.222.0.13 () {60 vars in 1153 bytes} [Wed Aug 31 09:49:21 2022] POST /identity/v3/projects => generated 375 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 09:49:21.414582 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24355397-7162-4764-a6ce-f8bc3abe6b78 admin admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:49:21.424316 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 1092/2182] 10.222.0.13 () {60 vars in 1144 bytes} [Wed Aug 31 09:49:21 2022] POST /identity/v3/users => generated 296 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 09:49:21.425815 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b6d9811-affc-4857-9be0-ee457b39843e admin admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:49:21.437114 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b6d9811-affc-4857-9be0-ee457b39843e admin admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:49:21.495685 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24355397-7162-4764-a6ce-f8bc3abe6b78 admin admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:49:21.504089 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b050a1-1cdb-48cf-8dba-b790e7195c23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:49:21.504726 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b050a1-1cdb-48cf-8dba-b790e7195c23 admin admin] REQUEST_METHOD: `POST` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:21.505012 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b050a1-1cdb-48cf-8dba-b790e7195c23 admin admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:21.505286 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b050a1-1cdb-48cf-8dba-b790e7195c23 admin admin] PATH_INFO: `/v3/users` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:21.506032 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4b050a1-1cdb-48cf-8dba-b790e7195c23 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:49:21.508018 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4b050a1-1cdb-48cf-8dba-b790e7195c23 admin admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:49:21.511371 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb043aff-c503-46c3-b15c-cc530a480872 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:49:21.512225 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb043aff-c503-46c3-b15c-cc530a480872 admin admin] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:21.512655 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb043aff-c503-46c3-b15c-cc530a480872 admin admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:21.513078 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb043aff-c503-46c3-b15c-cc530a480872 admin admin] PATH_INFO: `/v3/roles` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:21.514088 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb043aff-c503-46c3-b15c-cc530a480872 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:49:21.516276 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb043aff-c503-46c3-b15c-cc530a480872 admin admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:49:21.530874 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 1093/2183] 10.222.0.13 () {58 vars in 1122 bytes} [Wed Aug 31 09:49:21 2022] GET /identity/v3/roles => generated 1420 bytes in 106 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 09:49:21.536916 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb043aff-c503-46c3-b15c-cc530a480872 admin admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:49:21.541257 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 1091/2184] 10.222.0.13 () {60 vars in 1144 bytes} [Wed Aug 31 09:49:21 2022] POST /identity/v3/users => generated 288 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 09:49:21.549653 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b050a1-1cdb-48cf-8dba-b790e7195c23 admin admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:49:21.551035 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb043aff-c503-46c3-b15c-cc530a480872 admin admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:49:21.556964 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b82332b-9b4b-4497-bfc9-bdddec79fbdf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:49:21.557903 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b82332b-9b4b-4497-bfc9-bdddec79fbdf admin admin] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:21.558420 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b82332b-9b4b-4497-bfc9-bdddec79fbdf admin admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:21.558889 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b82332b-9b4b-4497-bfc9-bdddec79fbdf admin admin] PATH_INFO: `/v3/roles` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:21.559706 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b82332b-9b4b-4497-bfc9-bdddec79fbdf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:49:21.560461 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b050a1-1cdb-48cf-8dba-b790e7195c23 admin admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:49:21.562230 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b82332b-9b4b-4497-bfc9-bdddec79fbdf admin admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:49:21.566417 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe938d4-f6b2-42ba-8f1c-bc219b6c1714 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:49:21.567070 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe938d4-f6b2-42ba-8f1c-bc219b6c1714 admin admin] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:21.567342 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe938d4-f6b2-42ba-8f1c-bc219b6c1714 admin admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:21.567618 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe938d4-f6b2-42ba-8f1c-bc219b6c1714 admin admin] PATH_INFO: `/v3/roles` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:21.568282 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fe938d4-f6b2-42ba-8f1c-bc219b6c1714 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:49:21.569992 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fe938d4-f6b2-42ba-8f1c-bc219b6c1714 admin admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:49:21.582304 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 1092/2185] 10.222.0.13 () {58 vars in 1122 bytes} [Wed Aug 31 09:49:21 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 09:49:21.585546 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 1094/2186] 10.222.0.13 () {58 vars in 1122 bytes} [Wed Aug 31 09:49:21 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 09:49:21.586188 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe938d4-f6b2-42ba-8f1c-bc219b6c1714 admin admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:49:21.587114 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b82332b-9b4b-4497-bfc9-bdddec79fbdf admin admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:49:21.597279 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe938d4-f6b2-42ba-8f1c-bc219b6c1714 admin admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:49:21.599643 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b82332b-9b4b-4497-bfc9-bdddec79fbdf admin admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:49:21.602460 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f7fb42-41c9-487b-9fee-8fe8387def64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:49:21.603287 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f7fb42-41c9-487b-9fee-8fe8387def64 admin admin] REQUEST_METHOD: `PUT` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:21.603674 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f7fb42-41c9-487b-9fee-8fe8387def64 admin admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:21.604053 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f7fb42-41c9-487b-9fee-8fe8387def64 admin admin] PATH_INFO: `/v3/projects/c60fad1266414b68b00b15a1e6ef973b/users/e751c150f99f422aaba4200d68647b5a/roles/6c99f5d9f1f243eaa134d8540f21095d` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:21.604730 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-075fa132-4d91-41ec-8a06-e59f359585f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:49:21.605804 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075fa132-4d91-41ec-8a06-e59f359585f5 admin admin] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:21.606150 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075fa132-4d91-41ec-8a06-e59f359585f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:21.606430 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075fa132-4d91-41ec-8a06-e59f359585f5 admin admin] PATH_INFO: `/v3/roles` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:21.607540 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-075fa132-4d91-41ec-8a06-e59f359585f5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:49:21.609269 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-075fa132-4d91-41ec-8a06-e59f359585f5 admin admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:49:21.623774 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 1095/2187] 10.222.0.13 () {58 vars in 1122 bytes} [Wed Aug 31 09:49:21 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 09:49:21.630407 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-075fa132-4d91-41ec-8a06-e59f359585f5 admin admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:49:21.635175 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93f7fb42-41c9-487b-9fee-8fe8387def64 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c60fad1266414b68b00b15a1e6ef973b, user_id=e751c150f99f422aaba4200d68647b5a, role_id=6c99f5d9f1f243eaa134d8540f21095d)` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:49:21.640319 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93f7fb42-41c9-487b-9fee-8fe8387def64 admin admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:49:21.641931 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-075fa132-4d91-41ec-8a06-e59f359585f5 admin admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:49:21.648948 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb343b0a-acd9-4667-9d11-e75f205e4b58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:49:21.650034 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb343b0a-acd9-4667-9d11-e75f205e4b58 admin admin] REQUEST_METHOD: `PUT` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:21.650265 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb343b0a-acd9-4667-9d11-e75f205e4b58 admin admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:21.650640 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb343b0a-acd9-4667-9d11-e75f205e4b58 admin admin] PATH_INFO: `/v3/projects/340cdbf71a7a4b82b5250c53611908d7/users/0649160ba37c4f4eba80092d84879f7d/roles/6c99f5d9f1f243eaa134d8540f21095d` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:21.669430 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 1093/2188] 10.222.0.13 () {60 vars in 1483 bytes} [Wed Aug 31 09:49:21 2022] PUT /identity/v3/projects/c60fad1266414b68b00b15a1e6ef973b/users/e751c150f99f422aaba4200d68647b5a/roles/6c99f5d9f1f243eaa134d8540f21095d => generated 0 bytes in 79 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:49:21.681729 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4808958f-18b5-4c4f-9545-7ad71a37a16c None None] REQUEST_METHOD: `POST` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:21.682686 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb343b0a-acd9-4667-9d11-e75f205e4b58 admin admin] RBAC: Authorizing `identity:create_grant(project_id=340cdbf71a7a4b82b5250c53611908d7, user_id=0649160ba37c4f4eba80092d84879f7d, role_id=6c99f5d9f1f243eaa134d8540f21095d)` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:49:21.683976 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4808958f-18b5-4c4f-9545-7ad71a37a16c None None] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:21.684535 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4808958f-18b5-4c4f-9545-7ad71a37a16c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:21.687576 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb343b0a-acd9-4667-9d11-e75f205e4b58 admin admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:49:21.729731 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 1096/2189] 10.222.0.13 () {60 vars in 1483 bytes} [Wed Aug 31 09:49:21 2022] PUT /identity/v3/projects/340cdbf71a7a4b82b5250c53611908d7/users/0649160ba37c4f4eba80092d84879f7d/roles/6c99f5d9f1f243eaa134d8540f21095d => generated 0 bytes in 93 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:49:21.742672 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3465f11f-f4fd-4956-b421-31cbc90a8651 None None] REQUEST_METHOD: `POST` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:21.743269 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3465f11f-f4fd-4956-b421-31cbc90a8651 None None] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:21.743814 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3465f11f-f4fd-4956-b421-31cbc90a8651 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:21.749144 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.auth.core [None req-4808958f-18b5-4c4f-9545-7ad71a37a16c None None] MFA Rules not processed for user `e751c150f99f422aaba4200d68647b5a`. Rule list: `[]` (Enabled: `True`). {{(pid=88113) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:49:21.753408 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.fernet_utils [None req-4808958f-18b5-4c4f-9545-7ad71a37a16c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:49:21.786568 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.auth.core [None req-3465f11f-f4fd-4956-b421-31cbc90a8651 None None] MFA Rules not processed for user `0649160ba37c4f4eba80092d84879f7d`. Rule list: `[]` (Enabled: `True`). {{(pid=88114) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:49:21.787802 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.fernet_utils [None req-3465f11f-f4fd-4956-b421-31cbc90a8651 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88114) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:49:21.917464 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 1094/2190] 10.222.0.13 () {58 vars in 958 bytes} [Wed Aug 31 09:49:21 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 237 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:49:21.919359 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4808958f-18b5-4c4f-9545-7ad71a37a16c None None] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:49:21.953698 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 1097/2191] 10.222.0.13 () {58 vars in 958 bytes} [Wed Aug 31 09:49:21 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 213 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:49:21.976531 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3465f11f-f4fd-4956-b421-31cbc90a8651 None None] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:49:21.979089 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.fernet_utils [None req-3465f11f-f4fd-4956-b421-31cbc90a8651 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88114) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:49:21.982955 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:49:22.068890 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:49:22.116488 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4808958f-18b5-4c4f-9545-7ad71a37a16c None None] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:49:22.138949 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-418e1912-0b92-4051-92d6-b29058144373 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:49:22.140399 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3465f11f-f4fd-4956-b421-31cbc90a8651 None None] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:49:22.144169 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418e1912-0b92-4051-92d6-b29058144373 admin admin] REQUEST_METHOD: `POST` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:22.145000 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418e1912-0b92-4051-92d6-b29058144373 admin admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:22.145525 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418e1912-0b92-4051-92d6-b29058144373 admin admin] PATH_INFO: `/v3/projects` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:22.146829 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-418e1912-0b92-4051-92d6-b29058144373 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:49:22.147979 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6714b72e-6004-4680-8251-83d4f932e66b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d52b69c8e94441dacfcff944326f169', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '49548eb44d62415582af820ab66decee', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:49:22.149257 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6714b72e-6004-4680-8251-83d4f932e66b service cinder] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:22.149771 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6714b72e-6004-4680-8251-83d4f932e66b service cinder] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:22.150400 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6714b72e-6004-4680-8251-83d4f932e66b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:22.154178 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-418e1912-0b92-4051-92d6-b29058144373 admin admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:49:22.175319 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6714b72e-6004-4680-8251-83d4f932e66b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:49:22.186215 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6714b72e-6004-4680-8251-83d4f932e66b service cinder] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:49:22.214713 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 1098/2192] 10.222.0.13 () {60 vars in 1365 bytes} [Wed Aug 31 09:49:21 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 240 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:49:22.275854 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 1095/2193] 10.222.0.13 () {60 vars in 1153 bytes} [Wed Aug 31 09:49:21 2022] POST /identity/v3/projects => generated 391 bytes in 357 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 09:49:22.284298 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6714b72e-6004-4680-8251-83d4f932e66b service cinder] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:49:22.310642 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6714b72e-6004-4680-8251-83d4f932e66b service cinder] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:49:22.633281 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d1a84f1-16b8-4dd9-9d0e-3b3537b89fa4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:49:22.640414 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1a84f1-16b8-4dd9-9d0e-3b3537b89fa4 admin admin] REQUEST_METHOD: `POST` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:22.643031 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1a84f1-16b8-4dd9-9d0e-3b3537b89fa4 admin admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:22.643778 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1a84f1-16b8-4dd9-9d0e-3b3537b89fa4 admin admin] PATH_INFO: `/v3/users` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:22.650877 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d1a84f1-16b8-4dd9-9d0e-3b3537b89fa4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:49:22.657690 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d1a84f1-16b8-4dd9-9d0e-3b3537b89fa4 admin admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:49:22.741275 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 1099/2194] 10.222.0.13 () {60 vars in 1144 bytes} [Wed Aug 31 09:49:22 2022] POST /identity/v3/users => generated 302 bytes in 451 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 09:49:22.741888 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-418e1912-0b92-4051-92d6-b29058144373 admin admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:49:22.756088 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-418e1912-0b92-4051-92d6-b29058144373 admin admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:49:22.769776 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48799dc6-42a2-472d-a5e0-ef70b29d8538 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:49:22.770382 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48799dc6-42a2-472d-a5e0-ef70b29d8538 admin admin] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:22.770672 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48799dc6-42a2-472d-a5e0-ef70b29d8538 admin admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:22.771048 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48799dc6-42a2-472d-a5e0-ef70b29d8538 admin admin] PATH_INFO: `/v3/roles` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:22.771772 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48799dc6-42a2-472d-a5e0-ef70b29d8538 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:49:22.775912 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48799dc6-42a2-472d-a5e0-ef70b29d8538 admin admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:49:22.802473 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 1096/2195] 10.222.0.13 () {58 vars in 1122 bytes} [Wed Aug 31 09:49:22 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 09:49:22.815658 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d1a84f1-16b8-4dd9-9d0e-3b3537b89fa4 admin admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:49:22.832012 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d1a84f1-16b8-4dd9-9d0e-3b3537b89fa4 admin admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:49:22.875544 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0bb30fe-c9c7-4e34-a7bb-0dd47bd7ceb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:49:22.878214 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0bb30fe-c9c7-4e34-a7bb-0dd47bd7ceb5 admin admin] REQUEST_METHOD: `PUT` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:22.878616 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0bb30fe-c9c7-4e34-a7bb-0dd47bd7ceb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:22.879049 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0bb30fe-c9c7-4e34-a7bb-0dd47bd7ceb5 admin admin] PATH_INFO: `/v3/projects/1527f99179cf464091fc156a34f4208b/users/9e4606581ca145afb624f75dfe4a2125/roles/5d4ee63a3add47fc9a6b8b16d39e0a1f` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:22.954080 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0bb30fe-c9c7-4e34-a7bb-0dd47bd7ceb5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1527f99179cf464091fc156a34f4208b, user_id=9e4606581ca145afb624f75dfe4a2125, role_id=5d4ee63a3add47fc9a6b8b16d39e0a1f)` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:49:22.957288 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0bb30fe-c9c7-4e34-a7bb-0dd47bd7ceb5 admin admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:49:22.997758 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 1100/2196] 10.222.0.13 () {60 vars in 1483 bytes} [Wed Aug 31 09:49:22 2022] PUT /identity/v3/projects/1527f99179cf464091fc156a34f4208b/users/9e4606581ca145afb624f75dfe4a2125/roles/5d4ee63a3add47fc9a6b8b16d39e0a1f => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:49:23.000264 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48799dc6-42a2-472d-a5e0-ef70b29d8538 admin admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:49:23.013790 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48799dc6-42a2-472d-a5e0-ef70b29d8538 admin admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:49:23.193179 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4be6bcb-5fea-4e8a-b278-a6f899376411 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:49:23.194632 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4be6bcb-5fea-4e8a-b278-a6f899376411 admin admin] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:23.195463 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4be6bcb-5fea-4e8a-b278-a6f899376411 admin admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:23.196599 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4be6bcb-5fea-4e8a-b278-a6f899376411 admin admin] PATH_INFO: `/v3/roles` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:23.198928 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4be6bcb-5fea-4e8a-b278-a6f899376411 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:49:23.204233 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4be6bcb-5fea-4e8a-b278-a6f899376411 admin admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:49:23.252520 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 1097/2197] 10.222.0.13 () {58 vars in 1122 bytes} [Wed Aug 31 09:49:22 2022] GET /identity/v3/roles => generated 1420 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 09:49:23.260964 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0bb30fe-c9c7-4e34-a7bb-0dd47bd7ceb5 admin admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:49:23.298919 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0bb30fe-c9c7-4e34-a7bb-0dd47bd7ceb5 admin admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:49:23.337458 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b1e6ea-3f54-435e-9c40-4feed1e0fd73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:49:23.338637 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b1e6ea-3f54-435e-9c40-4feed1e0fd73 admin admin] REQUEST_METHOD: `PUT` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:23.339362 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b1e6ea-3f54-435e-9c40-4feed1e0fd73 admin admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:23.339918 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b1e6ea-3f54-435e-9c40-4feed1e0fd73 admin admin] PATH_INFO: `/v3/projects/1527f99179cf464091fc156a34f4208b/users/9e4606581ca145afb624f75dfe4a2125/roles/5d4ee63a3add47fc9a6b8b16d39e0a1f` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:23.370541 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40b1e6ea-3f54-435e-9c40-4feed1e0fd73 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1527f99179cf464091fc156a34f4208b, user_id=9e4606581ca145afb624f75dfe4a2125, role_id=5d4ee63a3add47fc9a6b8b16d39e0a1f)` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:49:23.380716 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40b1e6ea-3f54-435e-9c40-4feed1e0fd73 admin admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:49:23.418474 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 1101/2198] 10.222.0.13 () {60 vars in 1483 bytes} [Wed Aug 31 09:49:23 2022] PUT /identity/v3/projects/1527f99179cf464091fc156a34f4208b/users/9e4606581ca145afb624f75dfe4a2125/roles/5d4ee63a3add47fc9a6b8b16d39e0a1f => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:49:23.430137 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30259769-a178-4632-a2d4-4c55e07339d2 None None] REQUEST_METHOD: `POST` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:23.431018 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30259769-a178-4632-a2d4-4c55e07339d2 None None] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:23.431684 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30259769-a178-4632-a2d4-4c55e07339d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:23.491353 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.auth.core [None req-30259769-a178-4632-a2d4-4c55e07339d2 None None] MFA Rules not processed for user `9e4606581ca145afb624f75dfe4a2125`. Rule list: `[]` (Enabled: `True`). {{(pid=88113) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:49:23.492932 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.fernet_utils [None req-30259769-a178-4632-a2d4-4c55e07339d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:49:23.628009 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 1098/2199] 10.222.0.13 () {58 vars in 958 bytes} [Wed Aug 31 09:49:23 2022] POST /identity/v3/auth/tokens => generated 3335 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:49:23.644443 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b1e6ea-3f54-435e-9c40-4feed1e0fd73 admin admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:49:23.739601 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798bce60-20e2-443a-8704-3b923542e38a None None] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:23.740076 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798bce60-20e2-443a-8704-3b923542e38a None None] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:23.741607 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798bce60-20e2-443a-8704-3b923542e38a None None] PATH_INFO: `/` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:23.742906 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 1099/2200] 10.222.0.13 () {54 vars in 921 bytes} [Wed Aug 31 09:49:23 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:49:23.754491 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b1e6ea-3f54-435e-9c40-4feed1e0fd73 admin admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:49:23.759518 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40aeb058-4627-4bf3-a8a9-3992d2012a60 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d52b69c8e94441dacfcff944326f169', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '49548eb44d62415582af820ab66decee', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:49:23.760162 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40aeb058-4627-4bf3-a8a9-3992d2012a60 service cinder] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:23.760566 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40aeb058-4627-4bf3-a8a9-3992d2012a60 service cinder] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:23.760937 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40aeb058-4627-4bf3-a8a9-3992d2012a60 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:23.774475 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40aeb058-4627-4bf3-a8a9-3992d2012a60 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:49:23.777408 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40aeb058-4627-4bf3-a8a9-3992d2012a60 service cinder] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:49:23.784194 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5440a427-8b37-4e36-982a-cdc291bf22eb None None] REQUEST_METHOD: `POST` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:23.785545 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5440a427-8b37-4e36-982a-cdc291bf22eb None None] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:23.786529 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5440a427-8b37-4e36-982a-cdc291bf22eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:23.800799 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 1102/2201] 10.222.0.13 () {60 vars in 1365 bytes} [Wed Aug 31 09:49:23 2022] GET /identity/v3/auth/tokens => generated 3335 bytes in 159 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:49:23.853840 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.auth.core [None req-5440a427-8b37-4e36-982a-cdc291bf22eb None None] MFA Rules not processed for user `1ff3aff5ef444910be6dbb989d0e3194`. Rule list: `[]` (Enabled: `True`). {{(pid=88113) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:49:23.855685 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.fernet_utils [None req-5440a427-8b37-4e36-982a-cdc291bf22eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:49:23.868940 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40aeb058-4627-4bf3-a8a9-3992d2012a60 service cinder] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:49:23.882040 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.fernet_utils [None req-40aeb058-4627-4bf3-a8a9-3992d2012a60 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88114) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:49:23.883013 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:49:23.954219 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:49:23.959666 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 1100/2202] 10.222.0.13 () {58 vars in 1018 bytes} [Wed Aug 31 09:49:23 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 09:49:23.961046 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5440a427-8b37-4e36-982a-cdc291bf22eb None None] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:49:23.962853 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.fernet_utils [None req-5440a427-8b37-4e36-982a-cdc291bf22eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:49:23.988926 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5440a427-8b37-4e36-982a-cdc291bf22eb None None] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:49:23.993008 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a7eb5a0-f50e-456e-ac91-77401ccc3afb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d52b69c8e94441dacfcff944326f169', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '49548eb44d62415582af820ab66decee', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:49:23.993688 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7eb5a0-f50e-456e-ac91-77401ccc3afb service cinder] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:23.993968 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7eb5a0-f50e-456e-ac91-77401ccc3afb service cinder] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:23.994238 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7eb5a0-f50e-456e-ac91-77401ccc3afb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:24.038298 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40aeb058-4627-4bf3-a8a9-3992d2012a60 service cinder] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:49:24.046072 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4617b256-cc8b-4814-9348-91cb4be128b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:49:24.046929 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4617b256-cc8b-4814-9348-91cb4be128b2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:24.047220 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4617b256-cc8b-4814-9348-91cb4be128b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:24.047468 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4617b256-cc8b-4814-9348-91cb4be128b2 admin admin] PATH_INFO: `/v3/users/ae46122578a143a78f07f7f3a80ae079` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:24.049460 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4617b256-cc8b-4814-9348-91cb4be128b2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ae46122578a143a78f07f7f3a80ae079)` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:49:24.051024 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4617b256-cc8b-4814-9348-91cb4be128b2 admin admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:49:24.072810 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a7eb5a0-f50e-456e-ac91-77401ccc3afb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:49:24.074541 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a7eb5a0-f50e-456e-ac91-77401ccc3afb service cinder] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:49:24.096222 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 1101/2203] 10.222.0.13 () {60 vars in 1365 bytes} [Wed Aug 31 09:49:23 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:49:24.098733 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0cfa6a-7c17-4492-8308-5d4844889fa4 None None] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:24.099027 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0cfa6a-7c17-4492-8308-5d4844889fa4 None None] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:24.099382 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0cfa6a-7c17-4492-8308-5d4844889fa4 None None] PATH_INFO: `/` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:24.101532 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 1102/2204] 10.222.0.13 () {54 vars in 961 bytes} [Wed Aug 31 09:49:24 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:49:24.116852 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a378a467-849f-48ac-8a65-18e1cfbbebc4 None None] REQUEST_METHOD: `POST` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:24.117845 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a378a467-849f-48ac-8a65-18e1cfbbebc4 None None] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:24.119177 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a378a467-849f-48ac-8a65-18e1cfbbebc4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:24.131760 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.notifications [None req-4617b256-cc8b-4814-9348-91cb4be128b2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ae46122578a143a78f07f7f3a80ae079'} {{(pid=88114) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:49:24.143581 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.notifications [None req-4617b256-cc8b-4814-9348-91cb4be128b2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ae46122578a143a78f07f7f3a80ae079'} {{(pid=88114) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:49:24.154152 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.notifications [None req-4617b256-cc8b-4814-9348-91cb4be128b2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ae46122578a143a78f07f7f3a80ae079'} {{(pid=88114) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:49:24.155014 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.notifications [None req-4617b256-cc8b-4814-9348-91cb4be128b2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ae46122578a143a78f07f7f3a80ae079'} {{(pid=88114) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:49:24.159615 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.auth.core [None req-a378a467-849f-48ac-8a65-18e1cfbbebc4 None None] MFA Rules not processed for user `1ff3aff5ef444910be6dbb989d0e3194`. Rule list: `[]` (Enabled: `True`). {{(pid=88113) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:49:24.162102 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.fernet_utils [None req-a378a467-849f-48ac-8a65-18e1cfbbebc4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:49:24.172577 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.notifications [None req-4617b256-cc8b-4814-9348-91cb4be128b2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ae46122578a143a78f07f7f3a80ae079'} {{(pid=88114) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:49:24.178759 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 1103/2205] 10.222.0.13 () {58 vars in 1058 bytes} [Wed Aug 31 09:49:24 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 64 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 09:49:24.186555 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a378a467-849f-48ac-8a65-18e1cfbbebc4 None None] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:49:24.191288 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 1103/2206] 10.222.0.13 () {58 vars in 1224 bytes} [Wed Aug 31 09:49:23 2022] DELETE /identity/v3/users/ae46122578a143a78f07f7f3a80ae079 => generated 0 bytes in 324 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 09:49:24.204784 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a378a467-849f-48ac-8a65-18e1cfbbebc4 None None] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:49:24.334077 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea63152-4b20-4d67-bb71-9efc48d73fc7 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ff3aff5ef444910be6dbb989d0e3194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '49548eb44d62415582af820ab66decee', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:49:24.335177 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea63152-4b20-4d67-bb71-9efc48d73fc7 service placement] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:24.335671 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea63152-4b20-4d67-bb71-9efc48d73fc7 service placement] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:24.336157 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea63152-4b20-4d67-bb71-9efc48d73fc7 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:24.339641 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.fernet_utils [None req-cea63152-4b20-4d67-bb71-9efc48d73fc7 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:49:24.365263 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cea63152-4b20-4d67-bb71-9efc48d73fc7 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:49:24.368431 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cea63152-4b20-4d67-bb71-9efc48d73fc7 service placement] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:49:24.392423 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 1104/2207] 10.222.0.13 () {60 vars in 1365 bytes} [Wed Aug 31 09:49:24 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 206 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 09:49:24.531757 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4617b256-cc8b-4814-9348-91cb4be128b2 admin admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:49:24.534427 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.fernet_utils [None req-4617b256-cc8b-4814-9348-91cb4be128b2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88114) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:49:24.733651 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4617b256-cc8b-4814-9348-91cb4be128b2 admin admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:49:24.746665 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e34f1d-c2a9-44b7-bc0b-38f302df80e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:49:24.747814 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e34f1d-c2a9-44b7-bc0b-38f302df80e0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:24.748358 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e34f1d-c2a9-44b7-bc0b-38f302df80e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:24.748984 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e34f1d-c2a9-44b7-bc0b-38f302df80e0 admin admin] PATH_INFO: `/v3/projects/eb404a7dd529496888531e7ef00fd352` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:24.751574 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1e34f1d-c2a9-44b7-bc0b-38f302df80e0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=eb404a7dd529496888531e7ef00fd352)` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:49:24.755904 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1e34f1d-c2a9-44b7-bc0b-38f302df80e0 admin admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:49:24.812619 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:49:24.822300 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.notifications [None req-b1e34f1d-c2a9-44b7-bc0b-38f302df80e0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'eb404a7dd529496888531e7ef00fd352', 'request_id': 'req-b1e34f1d-c2a9-44b7-bc0b-38f302df80e0'} {{(pid=88114) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:49:24.831235 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.notifications [None req-b1e34f1d-c2a9-44b7-bc0b-38f302df80e0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'eb404a7dd529496888531e7ef00fd352', 'request_id': 'req-b1e34f1d-c2a9-44b7-bc0b-38f302df80e0'} {{(pid=88114) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:49:24.837748 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.notifications [None req-b1e34f1d-c2a9-44b7-bc0b-38f302df80e0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'eb404a7dd529496888531e7ef00fd352', 'request_id': 'req-b1e34f1d-c2a9-44b7-bc0b-38f302df80e0'} {{(pid=88114) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:49:24.854276 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.notifications [None req-b1e34f1d-c2a9-44b7-bc0b-38f302df80e0 admin admin] The token cache is being invalidate because project eb404a7dd529496888531e7ef00fd352 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88114) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 09:49:24.854572 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.notifications [None req-b1e34f1d-c2a9-44b7-bc0b-38f302df80e0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88114) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:49:24.855297 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.notifications [None req-b1e34f1d-c2a9-44b7-bc0b-38f302df80e0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88114) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:49:24.856874 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 1104/2208] 10.222.0.13 () {58 vars in 1233 bytes} [Wed Aug 31 09:49:24 2022] DELETE /identity/v3/projects/eb404a7dd529496888531e7ef00fd352 => generated 0 bytes in 326 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:49:24.862784 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440b8eee-2ceb-4969-becb-e2c081dd2cb6 None None] REQUEST_METHOD: `POST` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:24.863511 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440b8eee-2ceb-4969-becb-e2c081dd2cb6 None None] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:24.863511 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440b8eee-2ceb-4969-becb-e2c081dd2cb6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:24.887095 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.auth.core [None req-440b8eee-2ceb-4969-becb-e2c081dd2cb6 None None] MFA Rules not processed for user `fd836cf590a145df9acf19cf3ae2ac56`. Rule list: `[]` (Enabled: `True`). {{(pid=88113) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:49:24.888230 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.fernet_utils [None req-440b8eee-2ceb-4969-becb-e2c081dd2cb6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:49:24.934578 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=100 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:49:24.966726 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 1105/2209] 10.222.0.13 () {58 vars in 958 bytes} [Wed Aug 31 09:49:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 09:49:24.977309 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be94364d-45e2-40ed-bec2-c264bef6e725 None None] REQUEST_METHOD: `POST` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:24.977989 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be94364d-45e2-40ed-bec2-c264bef6e725 None None] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:24.978410 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be94364d-45e2-40ed-bec2-c264bef6e725 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:25.002873 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.auth.core [None req-be94364d-45e2-40ed-bec2-c264bef6e725 None None] MFA Rules not processed for user `fd836cf590a145df9acf19cf3ae2ac56`. Rule list: `[]` (Enabled: `True`). {{(pid=88114) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:49:25.004108 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.fernet_utils [None req-be94364d-45e2-40ed-bec2-c264bef6e725 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88114) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:49:25.020405 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 1105/2210] 10.222.0.13 () {58 vars in 958 bytes} [Wed Aug 31 09:49:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:49:25.024916 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-440b8eee-2ceb-4969-becb-e2c081dd2cb6 None None] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:49:25.040746 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-440b8eee-2ceb-4969-becb-e2c081dd2cb6 None None] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:49:25.046603 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e664eb-e11b-4aef-ab99-4354d6a36203 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:49:25.047242 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e664eb-e11b-4aef-ab99-4354d6a36203 admin admin] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:25.047528 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e664eb-e11b-4aef-ab99-4354d6a36203 admin admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:25.047803 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e664eb-e11b-4aef-ab99-4354d6a36203 admin admin] PATH_INFO: `/v3/domains` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:25.048480 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8e664eb-e11b-4aef-ab99-4354d6a36203 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88113) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:49:25.048854 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8e664eb-e11b-4aef-ab99-4354d6a36203 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:49:25.050572 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8e664eb-e11b-4aef-ab99-4354d6a36203 admin admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:49:25.066379 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 1106/2211] 10.222.0.13 () {58 vars in 1153 bytes} [Wed Aug 31 09:49:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 09:49:25.071435 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be94364d-45e2-40ed-bec2-c264bef6e725 None None] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:49:25.086016 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be94364d-45e2-40ed-bec2-c264bef6e725 None None] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:49:25.094470 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5744caf0-8461-43dd-8ea5-1d9759d68917 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:49:25.095561 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5744caf0-8461-43dd-8ea5-1d9759d68917 admin admin] REQUEST_METHOD: `POST` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:25.096063 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5744caf0-8461-43dd-8ea5-1d9759d68917 admin admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:25.096560 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5744caf0-8461-43dd-8ea5-1d9759d68917 admin admin] PATH_INFO: `/v3/projects` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:25.097732 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5744caf0-8461-43dd-8ea5-1d9759d68917 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:49:25.101513 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5744caf0-8461-43dd-8ea5-1d9759d68917 admin admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:49:25.144205 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 1106/2212] 10.222.0.13 () {60 vars in 1153 bytes} [Wed Aug 31 09:49:25 2022] POST /identity/v3/projects => generated 383 bytes in 73 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 09:49:25.148617 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e664eb-e11b-4aef-ab99-4354d6a36203 admin admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:49:25.160770 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e664eb-e11b-4aef-ab99-4354d6a36203 admin admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:49:25.241039 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d5a85a-3316-4643-b1b6-6e1c534aefce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:49:25.241840 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d5a85a-3316-4643-b1b6-6e1c534aefce admin admin] REQUEST_METHOD: `POST` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:25.242195 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d5a85a-3316-4643-b1b6-6e1c534aefce admin admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:25.242550 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d5a85a-3316-4643-b1b6-6e1c534aefce admin admin] PATH_INFO: `/v3/users` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:25.243319 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2d5a85a-3316-4643-b1b6-6e1c534aefce admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:49:25.245855 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2d5a85a-3316-4643-b1b6-6e1c534aefce admin admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:49:25.279576 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 1107/2213] 10.222.0.13 () {60 vars in 1144 bytes} [Wed Aug 31 09:49:25 2022] POST /identity/v3/users => generated 292 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 09:49:25.280895 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5744caf0-8461-43dd-8ea5-1d9759d68917 admin admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:49:25.294235 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5744caf0-8461-43dd-8ea5-1d9759d68917 admin admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:49:25.300439 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d95fdcf5-7380-4381-832d-6d94d6ad3a45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:49:25.301471 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95fdcf5-7380-4381-832d-6d94d6ad3a45 admin admin] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:25.301970 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95fdcf5-7380-4381-832d-6d94d6ad3a45 admin admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:25.302318 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95fdcf5-7380-4381-832d-6d94d6ad3a45 admin admin] PATH_INFO: `/v3/roles` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:25.303185 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d95fdcf5-7380-4381-832d-6d94d6ad3a45 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:49:25.305486 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d95fdcf5-7380-4381-832d-6d94d6ad3a45 admin admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:49:25.321140 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 1107/2214] 10.222.0.13 () {58 vars in 1122 bytes} [Wed Aug 31 09:49:25 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 09:49:25.326748 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d5a85a-3316-4643-b1b6-6e1c534aefce admin admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:49:25.343421 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d5a85a-3316-4643-b1b6-6e1c534aefce admin admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:49:25.351378 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c5dba25-7773-43db-9d54-f2e65fb90a78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:49:25.351976 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5dba25-7773-43db-9d54-f2e65fb90a78 admin admin] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:25.352262 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5dba25-7773-43db-9d54-f2e65fb90a78 admin admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:25.352534 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5dba25-7773-43db-9d54-f2e65fb90a78 admin admin] PATH_INFO: `/v3/roles` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:25.353338 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c5dba25-7773-43db-9d54-f2e65fb90a78 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:49:25.355025 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c5dba25-7773-43db-9d54-f2e65fb90a78 admin admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:49:25.377472 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 1108/2215] 10.222.0.13 () {58 vars in 1122 bytes} [Wed Aug 31 09:49:25 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 09:49:25.381186 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d95fdcf5-7380-4381-832d-6d94d6ad3a45 admin admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:49:25.394482 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d95fdcf5-7380-4381-832d-6d94d6ad3a45 admin admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:49:25.400480 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b0b1b99-2328-4f6c-ab26-2d54d70b0b20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:49:25.401171 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0b1b99-2328-4f6c-ab26-2d54d70b0b20 admin admin] REQUEST_METHOD: `PUT` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:25.401505 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0b1b99-2328-4f6c-ab26-2d54d70b0b20 admin admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:25.401759 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0b1b99-2328-4f6c-ab26-2d54d70b0b20 admin admin] PATH_INFO: `/v3/projects/5ed9143757584694a5139ada8dbd5570/users/25b4233e5f0e41bf99b3fb3f5f6229c9/roles/6c99f5d9f1f243eaa134d8540f21095d` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:25.425975 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b0b1b99-2328-4f6c-ab26-2d54d70b0b20 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5ed9143757584694a5139ada8dbd5570, user_id=25b4233e5f0e41bf99b3fb3f5f6229c9, role_id=6c99f5d9f1f243eaa134d8540f21095d)` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:49:25.429352 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b0b1b99-2328-4f6c-ab26-2d54d70b0b20 admin admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:49:25.446490 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 1108/2216] 10.222.0.13 () {60 vars in 1483 bytes} [Wed Aug 31 09:49:25 2022] PUT /identity/v3/projects/5ed9143757584694a5139ada8dbd5570/users/25b4233e5f0e41bf99b3fb3f5f6229c9/roles/6c99f5d9f1f243eaa134d8540f21095d => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:49:25.451843 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a902ef-2440-4992-b5ce-a21db4f55505 None None] REQUEST_METHOD: `POST` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:25.452287 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a902ef-2440-4992-b5ce-a21db4f55505 None None] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:25.452678 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a902ef-2440-4992-b5ce-a21db4f55505 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:25.480828 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.auth.core [None req-43a902ef-2440-4992-b5ce-a21db4f55505 None None] MFA Rules not processed for user `fd836cf590a145df9acf19cf3ae2ac56`. Rule list: `[]` (Enabled: `True`). {{(pid=88113) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:49:25.482345 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.fernet_utils [None req-43a902ef-2440-4992-b5ce-a21db4f55505 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:49:25.580402 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 1109/2217] 10.222.0.13 () {58 vars in 958 bytes} [Wed Aug 31 09:49:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:49:25.588651 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b0b1b99-2328-4f6c-ab26-2d54d70b0b20 admin admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:49:25.590918 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.fernet_utils [None req-6b0b1b99-2328-4f6c-ab26-2d54d70b0b20 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88114) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:49:25.689016 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b0b1b99-2328-4f6c-ab26-2d54d70b0b20 admin admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:49:25.693657 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc8f362e-d265-4b9e-b302-14ac9569599e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9bdaacd597442c2bcde2c555f2cc140', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '49548eb44d62415582af820ab66decee', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:49:25.694284 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8f362e-d265-4b9e-b302-14ac9569599e service neutron] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:25.694557 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8f362e-d265-4b9e-b302-14ac9569599e service neutron] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:25.694822 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8f362e-d265-4b9e-b302-14ac9569599e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:25.706359 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc8f362e-d265-4b9e-b302-14ac9569599e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:49:25.708761 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc8f362e-d265-4b9e-b302-14ac9569599e service neutron] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:49:25.723266 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 1109/2218] 10.222.0.13 () {60 vars in 1365 bytes} [Wed Aug 31 09:49:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 09:49:25.781100 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b62d53-bac0-42e4-a533-de93cee1c1b4 None None] REQUEST_METHOD: `POST` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:25.782388 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b62d53-bac0-42e4-a533-de93cee1c1b4 None None] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:25.782895 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b62d53-bac0-42e4-a533-de93cee1c1b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:25.819018 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.auth.core [None req-c7b62d53-bac0-42e4-a533-de93cee1c1b4 None None] MFA Rules not processed for user `c475984de9e84ff18fb96c588a07c80b`. Rule list: `[]` (Enabled: `True`). {{(pid=88113) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:49:25.820868 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.fernet_utils [None req-c7b62d53-bac0-42e4-a533-de93cee1c1b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:49:25.886400 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:49:25.903238 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 1110/2219] 10.222.0.13 () {58 vars in 958 bytes} [Wed Aug 31 09:49:25 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:49:32.848565 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8293689-b854-4101-9e00-0427b4555273 None None] REQUEST_METHOD: `POST` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:32.850976 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8293689-b854-4101-9e00-0427b4555273 None None] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:32.851481 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8293689-b854-4101-9e00-0427b4555273 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:32.919291 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.auth.core [None req-a8293689-b854-4101-9e00-0427b4555273 None None] MFA Rules not processed for user `25b4233e5f0e41bf99b3fb3f5f6229c9`. Rule list: `[]` (Enabled: `True`). {{(pid=88114) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:49:32.921005 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.fernet_utils [None req-a8293689-b854-4101-9e00-0427b4555273 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88114) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:49:33.030651 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=93,keystone:DELETE=4,keystone:INSERT=3 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:49:33.040852 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 1110/2220] 10.222.0.13 () {58 vars in 958 bytes} [Wed Aug 31 09:49:32 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:49:33.059262 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b62d53-bac0-42e4-a533-de93cee1c1b4 None None] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:49:33.060833 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.fernet_utils [None req-c7b62d53-bac0-42e4-a533-de93cee1c1b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:49:33.172531 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b62d53-bac0-42e4-a533-de93cee1c1b4 None None] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:49:33.179326 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f68cebb7-e9d1-4013-a8da-26515b7adbf7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc076493239d4ccdbdc37c0369c49529', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '49548eb44d62415582af820ab66decee', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:49:33.179970 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68cebb7-e9d1-4013-a8da-26515b7adbf7 service nova] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:33.180257 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68cebb7-e9d1-4013-a8da-26515b7adbf7 service nova] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:33.180534 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68cebb7-e9d1-4013-a8da-26515b7adbf7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:33.190819 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f68cebb7-e9d1-4013-a8da-26515b7adbf7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:49:33.192503 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f68cebb7-e9d1-4013-a8da-26515b7adbf7 service nova] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:49:33.203371 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 1111/2221] 10.222.0.13 () {60 vars in 1365 bytes} [Wed Aug 31 09:49:33 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 09:49:36.100304 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8293689-b854-4101-9e00-0427b4555273 None None] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:49:36.105031 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.fernet_utils [None req-a8293689-b854-4101-9e00-0427b4555273 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88114) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:49:36.139660 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8293689-b854-4101-9e00-0427b4555273 None None] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:49:36.165791 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf07ab3-9111-4f78-9ff2-8c80bd7a78fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:49:36.171359 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf07ab3-9111-4f78-9ff2-8c80bd7a78fb admin admin] REQUEST_METHOD: `DELETE` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:36.180550 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf07ab3-9111-4f78-9ff2-8c80bd7a78fb admin admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:36.190313 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf07ab3-9111-4f78-9ff2-8c80bd7a78fb admin admin] PATH_INFO: `/v3/users/e751c150f99f422aaba4200d68647b5a` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:36.202397 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bf07ab3-9111-4f78-9ff2-8c80bd7a78fb admin admin] RBAC: Authorizing `identity:delete_user(user_id=e751c150f99f422aaba4200d68647b5a)` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:49:36.207187 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bf07ab3-9111-4f78-9ff2-8c80bd7a78fb admin admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:49:36.326115 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.notifications [None req-7bf07ab3-9111-4f78-9ff2-8c80bd7a78fb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e751c150f99f422aaba4200d68647b5a'} {{(pid=88114) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:49:36.335146 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.notifications [None req-7bf07ab3-9111-4f78-9ff2-8c80bd7a78fb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e751c150f99f422aaba4200d68647b5a'} {{(pid=88114) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:49:36.346757 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.notifications [None req-7bf07ab3-9111-4f78-9ff2-8c80bd7a78fb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e751c150f99f422aaba4200d68647b5a'} {{(pid=88114) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:49:36.347406 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.notifications [None req-7bf07ab3-9111-4f78-9ff2-8c80bd7a78fb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e751c150f99f422aaba4200d68647b5a'} {{(pid=88114) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:49:36.364887 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.notifications [None req-7bf07ab3-9111-4f78-9ff2-8c80bd7a78fb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e751c150f99f422aaba4200d68647b5a'} {{(pid=88114) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:49:36.378222 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 1111/2222] 10.222.0.13 () {58 vars in 1224 bytes} [Wed Aug 31 09:49:36 2022] DELETE /identity/v3/users/e751c150f99f422aaba4200d68647b5a => generated 0 bytes in 274 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:49:36.384901 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3489cc6-7196-4c99-8c12-c4cd24c4ca77 None None] REQUEST_METHOD: `POST` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:36.385810 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3489cc6-7196-4c99-8c12-c4cd24c4ca77 None None] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:36.385810 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3489cc6-7196-4c99-8c12-c4cd24c4ca77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:36.396741 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9a50e2-4ef7-4c24-95bf-e71494e07bc5 None None] REQUEST_METHOD: `POST` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:36.397228 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9a50e2-4ef7-4c24-95bf-e71494e07bc5 None None] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:36.397803 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9a50e2-4ef7-4c24-95bf-e71494e07bc5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:36.411410 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.auth.core [None req-b3489cc6-7196-4c99-8c12-c4cd24c4ca77 None None] MFA Rules not processed for user `fd836cf590a145df9acf19cf3ae2ac56`. Rule list: `[]` (Enabled: `True`). {{(pid=88113) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:49:36.414059 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.fernet_utils [None req-b3489cc6-7196-4c99-8c12-c4cd24c4ca77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:49:36.435764 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.auth.core [None req-bc9a50e2-4ef7-4c24-95bf-e71494e07bc5 None None] MFA Rules not processed for user `25b4233e5f0e41bf99b3fb3f5f6229c9`. Rule list: `[]` (Enabled: `True`). {{(pid=88114) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:49:36.444141 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.fernet_utils [None req-bc9a50e2-4ef7-4c24-95bf-e71494e07bc5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88114) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:49:36.591621 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 1112/2223] 10.222.0.13 () {58 vars in 958 bytes} [Wed Aug 31 09:49:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 208 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:49:36.605761 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3489cc6-7196-4c99-8c12-c4cd24c4ca77 None None] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:49:36.607537 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.fernet_utils [None req-b3489cc6-7196-4c99-8c12-c4cd24c4ca77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:49:36.654207 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 1112/2224] 10.222.0.13 () {58 vars in 958 bytes} [Wed Aug 31 09:49:36 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 259 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:49:36.744273 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3489cc6-7196-4c99-8c12-c4cd24c4ca77 None None] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:49:36.753092 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eacec7c-d060-4cc0-9eaa-eb31bb54b5ee service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9bdaacd597442c2bcde2c555f2cc140', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '49548eb44d62415582af820ab66decee', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:49:36.753955 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eacec7c-d060-4cc0-9eaa-eb31bb54b5ee service neutron] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:36.754432 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eacec7c-d060-4cc0-9eaa-eb31bb54b5ee service neutron] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:36.754810 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eacec7c-d060-4cc0-9eaa-eb31bb54b5ee service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:36.767972 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eacec7c-d060-4cc0-9eaa-eb31bb54b5ee service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:49:36.770177 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eacec7c-d060-4cc0-9eaa-eb31bb54b5ee service neutron] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:49:36.785123 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 1113/2225] 10.222.0.13 () {60 vars in 1365 bytes} [Wed Aug 31 09:49:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 180 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:49:37.529788 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc9a50e2-4ef7-4c24-95bf-e71494e07bc5 None None] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:49:37.532582 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.fernet_utils [None req-bc9a50e2-4ef7-4c24-95bf-e71494e07bc5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88114) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:49:37.578192 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc9a50e2-4ef7-4c24-95bf-e71494e07bc5 None None] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:49:37.586604 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b136478d-cf9a-496d-b6f0-c7c018887ff1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:49:37.587829 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b136478d-cf9a-496d-b6f0-c7c018887ff1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:37.588218 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b136478d-cf9a-496d-b6f0-c7c018887ff1 admin admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:37.590217 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b136478d-cf9a-496d-b6f0-c7c018887ff1 admin admin] PATH_INFO: `/v3/projects/c60fad1266414b68b00b15a1e6ef973b` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:37.593254 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b136478d-cf9a-496d-b6f0-c7c018887ff1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c60fad1266414b68b00b15a1e6ef973b)` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:49:37.596923 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b136478d-cf9a-496d-b6f0-c7c018887ff1 admin admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:49:37.700180 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.notifications [None req-b136478d-cf9a-496d-b6f0-c7c018887ff1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c60fad1266414b68b00b15a1e6ef973b', 'request_id': 'req-b136478d-cf9a-496d-b6f0-c7c018887ff1'} {{(pid=88114) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:49:37.717708 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.notifications [None req-b136478d-cf9a-496d-b6f0-c7c018887ff1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c60fad1266414b68b00b15a1e6ef973b', 'request_id': 'req-b136478d-cf9a-496d-b6f0-c7c018887ff1'} {{(pid=88114) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:49:37.727902 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.notifications [None req-b136478d-cf9a-496d-b6f0-c7c018887ff1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c60fad1266414b68b00b15a1e6ef973b', 'request_id': 'req-b136478d-cf9a-496d-b6f0-c7c018887ff1'} {{(pid=88114) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:49:37.749828 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:49:37.762087 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.notifications [None req-b136478d-cf9a-496d-b6f0-c7c018887ff1 admin admin] The token cache is being invalidate because project c60fad1266414b68b00b15a1e6ef973b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88114) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 09:49:37.762378 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.notifications [None req-b136478d-cf9a-496d-b6f0-c7c018887ff1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88114) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:49:37.768362 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.notifications [None req-b136478d-cf9a-496d-b6f0-c7c018887ff1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88114) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:49:37.778514 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 1113/2226] 10.222.0.13 () {58 vars in 1233 bytes} [Wed Aug 31 09:49:37 2022] DELETE /identity/v3/projects/c60fad1266414b68b00b15a1e6ef973b => generated 0 bytes in 246 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:49:37.779184 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eacec7c-d060-4cc0-9eaa-eb31bb54b5ee service neutron] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:49:37.796029 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eacec7c-d060-4cc0-9eaa-eb31bb54b5ee service neutron] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:49:37.881640 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=100 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:49:37.943278 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd67af2-7984-46de-a80f-070a266e628b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:49:37.944313 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd67af2-7984-46de-a80f-070a266e628b admin admin] REQUEST_METHOD: `DELETE` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:37.944779 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd67af2-7984-46de-a80f-070a266e628b admin admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:37.945331 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd67af2-7984-46de-a80f-070a266e628b admin admin] PATH_INFO: `/v3/users/9e4606581ca145afb624f75dfe4a2125` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:37.947404 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcd67af2-7984-46de-a80f-070a266e628b admin admin] RBAC: Authorizing `identity:delete_user(user_id=9e4606581ca145afb624f75dfe4a2125)` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:49:37.951129 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcd67af2-7984-46de-a80f-070a266e628b admin admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:49:38.045972 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.notifications [None req-bcd67af2-7984-46de-a80f-070a266e628b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9e4606581ca145afb624f75dfe4a2125'} {{(pid=88113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:49:38.055175 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.notifications [None req-bcd67af2-7984-46de-a80f-070a266e628b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9e4606581ca145afb624f75dfe4a2125'} {{(pid=88113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:49:38.071387 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.notifications [None req-bcd67af2-7984-46de-a80f-070a266e628b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9e4606581ca145afb624f75dfe4a2125'} {{(pid=88113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:49:38.072147 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.notifications [None req-bcd67af2-7984-46de-a80f-070a266e628b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9e4606581ca145afb624f75dfe4a2125'} {{(pid=88113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:49:38.083812 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.notifications [None req-bcd67af2-7984-46de-a80f-070a266e628b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9e4606581ca145afb624f75dfe4a2125'} {{(pid=88113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:49:38.093465 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 1114/2227] 10.222.0.13 () {58 vars in 1224 bytes} [Wed Aug 31 09:49:37 2022] DELETE /identity/v3/users/9e4606581ca145afb624f75dfe4a2125 => generated 0 bytes in 314 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:49:38.520638 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b136478d-cf9a-496d-b6f0-c7c018887ff1 admin admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:49:38.522700 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.fernet_utils [None req-b136478d-cf9a-496d-b6f0-c7c018887ff1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88114) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:49:38.561123 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b136478d-cf9a-496d-b6f0-c7c018887ff1 admin admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:49:38.778793 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f5ff3f-ce1e-4e16-ab71-202bb2f3042c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:49:38.780001 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f5ff3f-ce1e-4e16-ab71-202bb2f3042c admin admin] REQUEST_METHOD: `DELETE` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:38.780359 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f5ff3f-ce1e-4e16-ab71-202bb2f3042c admin admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:38.780927 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f5ff3f-ce1e-4e16-ab71-202bb2f3042c admin admin] PATH_INFO: `/v3/projects/1527f99179cf464091fc156a34f4208b` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:38.783934 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4f5ff3f-ce1e-4e16-ab71-202bb2f3042c admin admin] RBAC: Authorizing `identity:delete_project(project_id=1527f99179cf464091fc156a34f4208b)` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:49:38.788267 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4f5ff3f-ce1e-4e16-ab71-202bb2f3042c admin admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:49:38.903817 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.notifications [None req-a4f5ff3f-ce1e-4e16-ab71-202bb2f3042c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1527f99179cf464091fc156a34f4208b', 'request_id': 'req-a4f5ff3f-ce1e-4e16-ab71-202bb2f3042c'} {{(pid=88114) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:49:38.914366 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.notifications [None req-a4f5ff3f-ce1e-4e16-ab71-202bb2f3042c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1527f99179cf464091fc156a34f4208b', 'request_id': 'req-a4f5ff3f-ce1e-4e16-ab71-202bb2f3042c'} {{(pid=88114) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:49:38.923924 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.notifications [None req-a4f5ff3f-ce1e-4e16-ab71-202bb2f3042c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1527f99179cf464091fc156a34f4208b', 'request_id': 'req-a4f5ff3f-ce1e-4e16-ab71-202bb2f3042c'} {{(pid=88114) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:49:38.948779 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.notifications [None req-a4f5ff3f-ce1e-4e16-ab71-202bb2f3042c admin admin] The token cache is being invalidate because project 1527f99179cf464091fc156a34f4208b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88114) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 09:49:38.949126 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.notifications [None req-a4f5ff3f-ce1e-4e16-ab71-202bb2f3042c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88114) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:49:38.950069 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.notifications [None req-a4f5ff3f-ce1e-4e16-ab71-202bb2f3042c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88114) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:49:38.953045 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 1114/2228] 10.222.0.13 () {58 vars in 1233 bytes} [Wed Aug 31 09:49:38 2022] DELETE /identity/v3/projects/1527f99179cf464091fc156a34f4208b => generated 0 bytes in 433 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:49:38.961038 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328695fa-370f-4118-a5d5-0add413f1212 None None] REQUEST_METHOD: `POST` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:38.962435 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328695fa-370f-4118-a5d5-0add413f1212 None None] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:38.962435 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328695fa-370f-4118-a5d5-0add413f1212 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:39.019456 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.auth.core [None req-328695fa-370f-4118-a5d5-0add413f1212 None None] MFA Rules not processed for user `fd836cf590a145df9acf19cf3ae2ac56`. Rule list: `[]` (Enabled: `True`). {{(pid=88113) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:49:39.025133 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.fernet_utils [None req-328695fa-370f-4118-a5d5-0add413f1212 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:49:39.204151 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10efdba5-122a-419c-85d2-cf314d45d8b5 None None] REQUEST_METHOD: `POST` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:39.204703 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10efdba5-122a-419c-85d2-cf314d45d8b5 None None] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:39.205188 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10efdba5-122a-419c-85d2-cf314d45d8b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:39.205510 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 1115/2229] 10.222.0.13 () {58 vars in 958 bytes} [Wed Aug 31 09:49:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 243 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:49:39.244282 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.auth.core [None req-10efdba5-122a-419c-85d2-cf314d45d8b5 None None] MFA Rules not processed for user `fd836cf590a145df9acf19cf3ae2ac56`. Rule list: `[]` (Enabled: `True`). {{(pid=88114) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:49:39.246155 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.fernet_utils [None req-10efdba5-122a-419c-85d2-cf314d45d8b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88114) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:49:39.265452 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 1115/2230] 10.222.0.13 () {58 vars in 958 bytes} [Wed Aug 31 09:49:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 60 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:49:39.268870 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-328695fa-370f-4118-a5d5-0add413f1212 None None] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:49:39.291069 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-328695fa-370f-4118-a5d5-0add413f1212 None None] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:49:39.299296 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a1354a-2e66-4df0-b938-be5107bc66d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:49:39.300885 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a1354a-2e66-4df0-b938-be5107bc66d5 admin admin] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:39.301711 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a1354a-2e66-4df0-b938-be5107bc66d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:39.302162 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a1354a-2e66-4df0-b938-be5107bc66d5 admin admin] PATH_INFO: `/v3/domains` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:39.302955 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3a1354a-2e66-4df0-b938-be5107bc66d5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88113) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:49:39.303753 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3a1354a-2e66-4df0-b938-be5107bc66d5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:49:39.306447 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3a1354a-2e66-4df0-b938-be5107bc66d5 admin admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:49:39.325656 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 1116/2231] 10.222.0.13 () {58 vars in 1153 bytes} [Wed Aug 31 09:49:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 09:49:39.327170 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10efdba5-122a-419c-85d2-cf314d45d8b5 None None] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:49:39.337725 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10efdba5-122a-419c-85d2-cf314d45d8b5 None None] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:49:39.344072 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ca98f6-226b-4398-8e64-b746ca21c11c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:49:39.345860 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ca98f6-226b-4398-8e64-b746ca21c11c admin admin] REQUEST_METHOD: `POST` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:39.346904 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ca98f6-226b-4398-8e64-b746ca21c11c admin admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:39.347329 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ca98f6-226b-4398-8e64-b746ca21c11c admin admin] PATH_INFO: `/v3/projects` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:39.348180 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0ca98f6-226b-4398-8e64-b746ca21c11c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:49:39.350194 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0ca98f6-226b-4398-8e64-b746ca21c11c admin admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:49:39.383763 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 1116/2232] 10.222.0.13 () {60 vars in 1153 bytes} [Wed Aug 31 09:49:39 2022] POST /identity/v3/projects => generated 385 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 09:49:39.389369 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a1354a-2e66-4df0-b938-be5107bc66d5 admin admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:49:39.400624 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a1354a-2e66-4df0-b938-be5107bc66d5 admin admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:49:39.442057 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:49:39.480720 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e092891c-101b-4200-be12-b6caf8c35c94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:49:39.481367 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e092891c-101b-4200-be12-b6caf8c35c94 admin admin] REQUEST_METHOD: `POST` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:39.481676 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e092891c-101b-4200-be12-b6caf8c35c94 admin admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:39.481947 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e092891c-101b-4200-be12-b6caf8c35c94 admin admin] PATH_INFO: `/v3/users` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:39.482590 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e092891c-101b-4200-be12-b6caf8c35c94 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:49:39.484571 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e092891c-101b-4200-be12-b6caf8c35c94 admin admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:49:39.516734 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 1117/2233] 10.222.0.13 () {60 vars in 1144 bytes} [Wed Aug 31 09:49:39 2022] POST /identity/v3/users => generated 293 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 09:49:39.522039 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ca98f6-226b-4398-8e64-b746ca21c11c admin admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:49:39.535083 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ca98f6-226b-4398-8e64-b746ca21c11c admin admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:49:39.540521 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd58fcc3-21ff-4ea2-852d-e9fdcc26f8ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:49:39.541340 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd58fcc3-21ff-4ea2-852d-e9fdcc26f8ae admin admin] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:39.541643 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd58fcc3-21ff-4ea2-852d-e9fdcc26f8ae admin admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:39.541984 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd58fcc3-21ff-4ea2-852d-e9fdcc26f8ae admin admin] PATH_INFO: `/v3/roles` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:39.542720 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd58fcc3-21ff-4ea2-852d-e9fdcc26f8ae admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:49:39.544422 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd58fcc3-21ff-4ea2-852d-e9fdcc26f8ae admin admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:49:39.559066 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 1117/2234] 10.222.0.13 () {58 vars in 1122 bytes} [Wed Aug 31 09:49:39 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 09:49:39.564243 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e092891c-101b-4200-be12-b6caf8c35c94 admin admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:49:39.576811 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e092891c-101b-4200-be12-b6caf8c35c94 admin admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:49:39.582654 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-768294ce-0cc0-47aa-b217-419fba38e83f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:49:39.583279 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768294ce-0cc0-47aa-b217-419fba38e83f admin admin] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:39.583553 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768294ce-0cc0-47aa-b217-419fba38e83f admin admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:39.583819 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768294ce-0cc0-47aa-b217-419fba38e83f admin admin] PATH_INFO: `/v3/roles` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:39.584473 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-768294ce-0cc0-47aa-b217-419fba38e83f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:49:39.586146 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-768294ce-0cc0-47aa-b217-419fba38e83f admin admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:49:39.600065 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 1118/2235] 10.222.0.13 () {58 vars in 1122 bytes} [Wed Aug 31 09:49:39 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 09:49:39.604250 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd58fcc3-21ff-4ea2-852d-e9fdcc26f8ae admin admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:49:39.615683 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd58fcc3-21ff-4ea2-852d-e9fdcc26f8ae admin admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:49:39.620600 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6616e592-800e-414c-ad10-593c209317a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:49:39.621271 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6616e592-800e-414c-ad10-593c209317a1 admin admin] REQUEST_METHOD: `PUT` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:39.621579 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6616e592-800e-414c-ad10-593c209317a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:39.621852 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6616e592-800e-414c-ad10-593c209317a1 admin admin] PATH_INFO: `/v3/projects/50cbd5f9e1eb440aa10404dd6817c08f/users/dc73e8ee0b45484e86827a798f1ba018/roles/6c99f5d9f1f243eaa134d8540f21095d` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:39.644631 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6616e592-800e-414c-ad10-593c209317a1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=50cbd5f9e1eb440aa10404dd6817c08f, user_id=dc73e8ee0b45484e86827a798f1ba018, role_id=6c99f5d9f1f243eaa134d8540f21095d)` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:49:39.646879 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6616e592-800e-414c-ad10-593c209317a1 admin admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:49:39.661598 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 1118/2236] 10.222.0.13 () {60 vars in 1483 bytes} [Wed Aug 31 09:49:39 2022] PUT /identity/v3/projects/50cbd5f9e1eb440aa10404dd6817c08f/users/dc73e8ee0b45484e86827a798f1ba018/roles/6c99f5d9f1f243eaa134d8540f21095d => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 09:49:39.675103 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae8f1d1-e3f1-4c2c-8146-eff352099e9e None None] REQUEST_METHOD: `POST` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:39.675386 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae8f1d1-e3f1-4c2c-8146-eff352099e9e None None] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:39.675660 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae8f1d1-e3f1-4c2c-8146-eff352099e9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:39.707639 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.auth.core [None req-5ae8f1d1-e3f1-4c2c-8146-eff352099e9e None None] MFA Rules not processed for user `dc73e8ee0b45484e86827a798f1ba018`. Rule list: `[]` (Enabled: `True`). {{(pid=88113) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:49:39.708964 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.fernet_utils [None req-5ae8f1d1-e3f1-4c2c-8146-eff352099e9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:49:39.789485 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 1119/2237] 10.222.0.13 () {58 vars in 958 bytes} [Wed Aug 31 09:49:39 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:49:39.794374 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6616e592-800e-414c-ad10-593c209317a1 admin admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:49:39.829484 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:49:39.881177 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6616e592-800e-414c-ad10-593c209317a1 admin admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:49:39.886645 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc6b1349-a1ad-4dc9-b81e-571719c613ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:49:39.887283 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6b1349-a1ad-4dc9-b81e-571719c613ce admin admin] REQUEST_METHOD: `POST` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:39.887560 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6b1349-a1ad-4dc9-b81e-571719c613ce admin admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:39.887828 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6b1349-a1ad-4dc9-b81e-571719c613ce admin admin] PATH_INFO: `/v3/projects` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:39.888487 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc6b1349-a1ad-4dc9-b81e-571719c613ce admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:49:39.890522 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc6b1349-a1ad-4dc9-b81e-571719c613ce admin admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:49:39.934347 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 1119/2238] 10.222.0.13 () {60 vars in 1153 bytes} [Wed Aug 31 09:49:39 2022] POST /identity/v3/projects => generated 385 bytes in 141 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 09:49:39.938487 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae8f1d1-e3f1-4c2c-8146-eff352099e9e None None] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:49:39.952356 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae8f1d1-e3f1-4c2c-8146-eff352099e9e None None] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:49:40.025892 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d057c09-ab10-407b-93c8-6fc3ab4375de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:49:40.026464 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d057c09-ab10-407b-93c8-6fc3ab4375de admin admin] REQUEST_METHOD: `POST` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:40.026775 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d057c09-ab10-407b-93c8-6fc3ab4375de admin admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:40.027151 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d057c09-ab10-407b-93c8-6fc3ab4375de admin admin] PATH_INFO: `/v3/users` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:40.027930 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d057c09-ab10-407b-93c8-6fc3ab4375de admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:49:40.030527 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d057c09-ab10-407b-93c8-6fc3ab4375de admin admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:49:40.056428 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 1120/2239] 10.222.0.13 () {60 vars in 1144 bytes} [Wed Aug 31 09:49:39 2022] POST /identity/v3/users => generated 299 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 09:49:40.061551 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc6b1349-a1ad-4dc9-b81e-571719c613ce admin admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:49:40.081870 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc6b1349-a1ad-4dc9-b81e-571719c613ce admin admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:49:40.089017 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af8c586-9286-4ef4-809a-22dbac2c79cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:49:40.090019 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af8c586-9286-4ef4-809a-22dbac2c79cf admin admin] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:40.090456 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af8c586-9286-4ef4-809a-22dbac2c79cf admin admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:40.090849 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af8c586-9286-4ef4-809a-22dbac2c79cf admin admin] PATH_INFO: `/v3/roles` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:40.091715 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1af8c586-9286-4ef4-809a-22dbac2c79cf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:49:40.094696 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1af8c586-9286-4ef4-809a-22dbac2c79cf admin admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:49:40.109599 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 1120/2240] 10.222.0.13 () {58 vars in 1122 bytes} [Wed Aug 31 09:49:40 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 09:49:40.114627 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d057c09-ab10-407b-93c8-6fc3ab4375de admin admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:49:40.133291 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d057c09-ab10-407b-93c8-6fc3ab4375de admin admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:49:40.140704 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed94226b-0d7f-43f7-a9ec-a6b0435e0278 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:49:40.141435 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed94226b-0d7f-43f7-a9ec-a6b0435e0278 admin admin] REQUEST_METHOD: `PUT` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:40.141787 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed94226b-0d7f-43f7-a9ec-a6b0435e0278 admin admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:40.142745 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed94226b-0d7f-43f7-a9ec-a6b0435e0278 admin admin] PATH_INFO: `/v3/projects/5ac090dbae29480d994b2dbd38dbb153/users/451783bbe50f4173b08322c9ee9855ce/roles/5d4ee63a3add47fc9a6b8b16d39e0a1f` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:40.162929 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed94226b-0d7f-43f7-a9ec-a6b0435e0278 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5ac090dbae29480d994b2dbd38dbb153, user_id=451783bbe50f4173b08322c9ee9855ce, role_id=5d4ee63a3add47fc9a6b8b16d39e0a1f)` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:49:40.165836 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed94226b-0d7f-43f7-a9ec-a6b0435e0278 admin admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:49:40.182465 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 1121/2241] 10.222.0.13 () {60 vars in 1483 bytes} [Wed Aug 31 09:49:40 2022] PUT /identity/v3/projects/5ac090dbae29480d994b2dbd38dbb153/users/451783bbe50f4173b08322c9ee9855ce/roles/5d4ee63a3add47fc9a6b8b16d39e0a1f => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:49:40.186875 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af8c586-9286-4ef4-809a-22dbac2c79cf admin admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:49:40.197971 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af8c586-9286-4ef4-809a-22dbac2c79cf admin admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:49:40.264009 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb5dc48-2978-4434-89ca-f21568a767a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:49:40.264649 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb5dc48-2978-4434-89ca-f21568a767a6 admin admin] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:40.264930 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb5dc48-2978-4434-89ca-f21568a767a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:40.265205 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb5dc48-2978-4434-89ca-f21568a767a6 admin admin] PATH_INFO: `/v3/roles` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:40.265911 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbb5dc48-2978-4434-89ca-f21568a767a6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:49:40.267600 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbb5dc48-2978-4434-89ca-f21568a767a6 admin admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:49:40.285518 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 1121/2242] 10.222.0.13 () {58 vars in 1122 bytes} [Wed Aug 31 09:49:40 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 09:49:40.286996 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed94226b-0d7f-43f7-a9ec-a6b0435e0278 admin admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:49:40.292969 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:49:40.301225 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed94226b-0d7f-43f7-a9ec-a6b0435e0278 admin admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:49:40.306720 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c16629d5-4f54-41d1-ad47-4008bad285b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:49:40.307327 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16629d5-4f54-41d1-ad47-4008bad285b8 admin admin] REQUEST_METHOD: `PUT` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:40.307601 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16629d5-4f54-41d1-ad47-4008bad285b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:40.307872 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16629d5-4f54-41d1-ad47-4008bad285b8 admin admin] PATH_INFO: `/v3/projects/5ac090dbae29480d994b2dbd38dbb153/users/451783bbe50f4173b08322c9ee9855ce/roles/5d4ee63a3add47fc9a6b8b16d39e0a1f` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:40.309841 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c16629d5-4f54-41d1-ad47-4008bad285b8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5ac090dbae29480d994b2dbd38dbb153, user_id=451783bbe50f4173b08322c9ee9855ce, role_id=5d4ee63a3add47fc9a6b8b16d39e0a1f)` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:49:40.312582 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c16629d5-4f54-41d1-ad47-4008bad285b8 admin admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:49:40.324877 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 1122/2243] 10.222.0.13 () {60 vars in 1483 bytes} [Wed Aug 31 09:49:40 2022] PUT /identity/v3/projects/5ac090dbae29480d994b2dbd38dbb153/users/451783bbe50f4173b08322c9ee9855ce/roles/5d4ee63a3add47fc9a6b8b16d39e0a1f => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:49:40.337167 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202e4287-7081-46b9-9d5f-ae520d7ef7b4 None None] REQUEST_METHOD: `POST` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:40.337546 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202e4287-7081-46b9-9d5f-ae520d7ef7b4 None None] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:40.337750 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202e4287-7081-46b9-9d5f-ae520d7ef7b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:40.370067 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.auth.core [None req-202e4287-7081-46b9-9d5f-ae520d7ef7b4 None None] MFA Rules not processed for user `451783bbe50f4173b08322c9ee9855ce`. Rule list: `[]` (Enabled: `True`). {{(pid=88114) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:49:40.371332 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.fernet_utils [None req-202e4287-7081-46b9-9d5f-ae520d7ef7b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88114) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:49:40.458450 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 1122/2244] 10.222.0.13 () {58 vars in 958 bytes} [Wed Aug 31 09:49:40 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:49:40.469429 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c16629d5-4f54-41d1-ad47-4008bad285b8 admin admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:49:40.473005 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.fernet_utils [None req-c16629d5-4f54-41d1-ad47-4008bad285b8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:49:40.589703 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c16629d5-4f54-41d1-ad47-4008bad285b8 admin admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:49:40.595714 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87300d2a-6807-4005-892f-42842463f140 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d52b69c8e94441dacfcff944326f169', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '49548eb44d62415582af820ab66decee', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:49:40.596900 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87300d2a-6807-4005-892f-42842463f140 service cinder] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:40.597519 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87300d2a-6807-4005-892f-42842463f140 service cinder] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:40.598099 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87300d2a-6807-4005-892f-42842463f140 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:40.728932 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87300d2a-6807-4005-892f-42842463f140 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:49:40.730575 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87300d2a-6807-4005-892f-42842463f140 service cinder] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:49:40.752067 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 1123/2245] 10.222.0.13 () {60 vars in 1365 bytes} [Wed Aug 31 09:49:40 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 283 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:49:49.296053 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-202e4287-7081-46b9-9d5f-ae520d7ef7b4 None None] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:49:49.297887 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.fernet_utils [None req-202e4287-7081-46b9-9d5f-ae520d7ef7b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88114) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:49:49.313005 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-202e4287-7081-46b9-9d5f-ae520d7ef7b4 None None] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:49:49.317359 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb73234a-bdfb-46b1-860c-4fd23d47527e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d52b69c8e94441dacfcff944326f169', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '49548eb44d62415582af820ab66decee', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:49:49.318031 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb73234a-bdfb-46b1-860c-4fd23d47527e service cinder] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:49:49.318303 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb73234a-bdfb-46b1-860c-4fd23d47527e service cinder] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:49:49.318571 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb73234a-bdfb-46b1-860c-4fd23d47527e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:49:49.328579 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb73234a-bdfb-46b1-860c-4fd23d47527e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:49:49.330535 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb73234a-bdfb-46b1-860c-4fd23d47527e service cinder] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:49:49.341686 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 1123/2246] 10.222.0.13 () {60 vars in 1365 bytes} [Wed Aug 31 09:49:49 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:49:50.739618 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=52,keystone:INSERT=1 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:49:59.336589 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=96,keystone:INSERT=1 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:50:01.318005 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87300d2a-6807-4005-892f-42842463f140 service cinder] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:50:01.392590 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87300d2a-6807-4005-892f-42842463f140 service cinder] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:50:01.397321 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d37cd4a-9836-4218-b8ad-98c80914d209 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:50:01.397989 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d37cd4a-9836-4218-b8ad-98c80914d209 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:50:01.398265 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d37cd4a-9836-4218-b8ad-98c80914d209 admin admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:50:01.398540 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d37cd4a-9836-4218-b8ad-98c80914d209 admin admin] PATH_INFO: `/v3/users/dc73e8ee0b45484e86827a798f1ba018` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:50:01.399680 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d37cd4a-9836-4218-b8ad-98c80914d209 admin admin] RBAC: Authorizing `identity:delete_user(user_id=dc73e8ee0b45484e86827a798f1ba018)` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:50:01.401776 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d37cd4a-9836-4218-b8ad-98c80914d209 admin admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:50:01.459682 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.notifications [None req-5d37cd4a-9836-4218-b8ad-98c80914d209 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dc73e8ee0b45484e86827a798f1ba018'} {{(pid=88113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:50:01.468011 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.notifications [None req-5d37cd4a-9836-4218-b8ad-98c80914d209 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dc73e8ee0b45484e86827a798f1ba018'} {{(pid=88113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:50:01.486423 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.notifications [None req-5d37cd4a-9836-4218-b8ad-98c80914d209 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dc73e8ee0b45484e86827a798f1ba018'} {{(pid=88113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:50:01.487565 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.notifications [None req-5d37cd4a-9836-4218-b8ad-98c80914d209 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dc73e8ee0b45484e86827a798f1ba018'} {{(pid=88113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:50:01.506213 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.notifications [None req-5d37cd4a-9836-4218-b8ad-98c80914d209 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dc73e8ee0b45484e86827a798f1ba018'} {{(pid=88113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:50:01.514534 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 1124/2247] 10.222.0.13 () {58 vars in 1224 bytes} [Wed Aug 31 09:50:01 2022] DELETE /identity/v3/users/dc73e8ee0b45484e86827a798f1ba018 => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 09:50:01.522492 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124922b9-83d6-4f7d-af34-1f8b5f047dd1 None None] REQUEST_METHOD: `POST` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:50:01.523304 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124922b9-83d6-4f7d-af34-1f8b5f047dd1 None None] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:50:01.523304 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124922b9-83d6-4f7d-af34-1f8b5f047dd1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:50:01.550306 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.auth.core [None req-124922b9-83d6-4f7d-af34-1f8b5f047dd1 None None] MFA Rules not processed for user `fd836cf590a145df9acf19cf3ae2ac56`. Rule list: `[]` (Enabled: `True`). {{(pid=88114) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:50:01.551550 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.fernet_utils [None req-124922b9-83d6-4f7d-af34-1f8b5f047dd1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88114) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:50:01.617509 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 1124/2248] 10.222.0.13 () {58 vars in 958 bytes} [Wed Aug 31 09:50:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 09:50:01.628532 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d37cd4a-9836-4218-b8ad-98c80914d209 admin admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:50:01.630354 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.fernet_utils [None req-5d37cd4a-9836-4218-b8ad-98c80914d209 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:50:01.705745 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d37cd4a-9836-4218-b8ad-98c80914d209 admin admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:50:01.711040 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11af3397-f4af-4278-a524-004e8eb65c19 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9bdaacd597442c2bcde2c555f2cc140', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '49548eb44d62415582af820ab66decee', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:50:01.711833 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11af3397-f4af-4278-a524-004e8eb65c19 service neutron] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:50:01.712012 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11af3397-f4af-4278-a524-004e8eb65c19 service neutron] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:50:01.712345 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11af3397-f4af-4278-a524-004e8eb65c19 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:50:01.726492 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11af3397-f4af-4278-a524-004e8eb65c19 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:50:01.727982 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11af3397-f4af-4278-a524-004e8eb65c19 service neutron] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:50:01.747256 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 1125/2249] 10.222.0.13 () {60 vars in 1365 bytes} [Wed Aug 31 09:50:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:50:02.257933 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-124922b9-83d6-4f7d-af34-1f8b5f047dd1 None None] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:50:02.261244 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.fernet_utils [None req-124922b9-83d6-4f7d-af34-1f8b5f047dd1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88114) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:50:02.278078 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-124922b9-83d6-4f7d-af34-1f8b5f047dd1 None None] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:50:02.283939 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a34a78-44ad-40c9-8321-4d4074974bfd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:50:02.284601 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a34a78-44ad-40c9-8321-4d4074974bfd admin admin] REQUEST_METHOD: `DELETE` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:50:02.284883 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a34a78-44ad-40c9-8321-4d4074974bfd admin admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:50:02.285160 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a34a78-44ad-40c9-8321-4d4074974bfd admin admin] PATH_INFO: `/v3/projects/50cbd5f9e1eb440aa10404dd6817c08f` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:50:02.286468 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29a34a78-44ad-40c9-8321-4d4074974bfd admin admin] RBAC: Authorizing `identity:delete_project(project_id=50cbd5f9e1eb440aa10404dd6817c08f)` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:50:02.288597 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29a34a78-44ad-40c9-8321-4d4074974bfd admin admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:50:02.359223 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.notifications [None req-29a34a78-44ad-40c9-8321-4d4074974bfd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '50cbd5f9e1eb440aa10404dd6817c08f', 'request_id': 'req-29a34a78-44ad-40c9-8321-4d4074974bfd'} {{(pid=88114) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:50:02.371195 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.notifications [None req-29a34a78-44ad-40c9-8321-4d4074974bfd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '50cbd5f9e1eb440aa10404dd6817c08f', 'request_id': 'req-29a34a78-44ad-40c9-8321-4d4074974bfd'} {{(pid=88114) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:50:02.382631 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.notifications [None req-29a34a78-44ad-40c9-8321-4d4074974bfd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '50cbd5f9e1eb440aa10404dd6817c08f', 'request_id': 'req-29a34a78-44ad-40c9-8321-4d4074974bfd'} {{(pid=88114) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:50:02.406194 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.notifications [None req-29a34a78-44ad-40c9-8321-4d4074974bfd admin admin] The token cache is being invalidate because project 50cbd5f9e1eb440aa10404dd6817c08f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88114) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 09:50:02.408472 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.notifications [None req-29a34a78-44ad-40c9-8321-4d4074974bfd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88114) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:50:02.409888 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.notifications [None req-29a34a78-44ad-40c9-8321-4d4074974bfd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88114) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:50:02.413217 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 1125/2250] 10.222.0.13 () {58 vars in 1233 bytes} [Wed Aug 31 09:50:02 2022] DELETE /identity/v3/projects/50cbd5f9e1eb440aa10404dd6817c08f => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:50:02.419160 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11af3397-f4af-4278-a524-004e8eb65c19 service neutron] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:50:02.433736 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11af3397-f4af-4278-a524-004e8eb65c19 service neutron] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:50:02.464262 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:50:02.533937 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a64d4f-9497-4a46-8abe-9feff30b4174 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:50:02.535039 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a64d4f-9497-4a46-8abe-9feff30b4174 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:50:02.535511 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a64d4f-9497-4a46-8abe-9feff30b4174 admin admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:50:02.535936 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a64d4f-9497-4a46-8abe-9feff30b4174 admin admin] PATH_INFO: `/v3/users/451783bbe50f4173b08322c9ee9855ce` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:50:02.538658 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7a64d4f-9497-4a46-8abe-9feff30b4174 admin admin] RBAC: Authorizing `identity:delete_user(user_id=451783bbe50f4173b08322c9ee9855ce)` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:50:02.542484 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7a64d4f-9497-4a46-8abe-9feff30b4174 admin admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:50:02.616735 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.notifications [None req-c7a64d4f-9497-4a46-8abe-9feff30b4174 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '451783bbe50f4173b08322c9ee9855ce'} {{(pid=88113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:50:02.630157 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.notifications [None req-c7a64d4f-9497-4a46-8abe-9feff30b4174 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '451783bbe50f4173b08322c9ee9855ce'} {{(pid=88113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:50:02.648767 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.notifications [None req-c7a64d4f-9497-4a46-8abe-9feff30b4174 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '451783bbe50f4173b08322c9ee9855ce'} {{(pid=88113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:50:02.649649 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.notifications [None req-c7a64d4f-9497-4a46-8abe-9feff30b4174 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '451783bbe50f4173b08322c9ee9855ce'} {{(pid=88113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:50:02.658522 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.notifications [None req-c7a64d4f-9497-4a46-8abe-9feff30b4174 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '451783bbe50f4173b08322c9ee9855ce'} {{(pid=88113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:50:02.665617 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 1126/2251] 10.222.0.13 () {58 vars in 1224 bytes} [Wed Aug 31 09:50:02 2022] DELETE /identity/v3/users/451783bbe50f4173b08322c9ee9855ce => generated 0 bytes in 247 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:50:03.077436 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a34a78-44ad-40c9-8321-4d4074974bfd admin admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:50:03.082416 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.fernet_utils [None req-29a34a78-44ad-40c9-8321-4d4074974bfd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88114) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:50:03.096717 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a34a78-44ad-40c9-8321-4d4074974bfd admin admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:50:03.182481 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2651009-945d-4438-a49a-a59259a96a78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:50:03.183297 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2651009-945d-4438-a49a-a59259a96a78 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:50:03.183599 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2651009-945d-4438-a49a-a59259a96a78 admin admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:50:03.183981 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2651009-945d-4438-a49a-a59259a96a78 admin admin] PATH_INFO: `/v3/projects/5ac090dbae29480d994b2dbd38dbb153` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:50:03.185626 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2651009-945d-4438-a49a-a59259a96a78 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5ac090dbae29480d994b2dbd38dbb153)` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:50:03.190027 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2651009-945d-4438-a49a-a59259a96a78 admin admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:50:03.240446 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:50:03.283559 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.notifications [None req-a2651009-945d-4438-a49a-a59259a96a78 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5ac090dbae29480d994b2dbd38dbb153', 'request_id': 'req-a2651009-945d-4438-a49a-a59259a96a78'} {{(pid=88114) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:50:03.294263 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.notifications [None req-a2651009-945d-4438-a49a-a59259a96a78 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5ac090dbae29480d994b2dbd38dbb153', 'request_id': 'req-a2651009-945d-4438-a49a-a59259a96a78'} {{(pid=88114) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:50:03.304503 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.notifications [None req-a2651009-945d-4438-a49a-a59259a96a78 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5ac090dbae29480d994b2dbd38dbb153', 'request_id': 'req-a2651009-945d-4438-a49a-a59259a96a78'} {{(pid=88114) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:50:03.330794 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.notifications [None req-a2651009-945d-4438-a49a-a59259a96a78 admin admin] The token cache is being invalidate because project 5ac090dbae29480d994b2dbd38dbb153 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88114) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 09:50:03.331231 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.notifications [None req-a2651009-945d-4438-a49a-a59259a96a78 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88114) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:50:03.332129 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.notifications [None req-a2651009-945d-4438-a49a-a59259a96a78 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88114) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:50:03.334677 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 1126/2252] 10.222.0.13 () {58 vars in 1233 bytes} [Wed Aug 31 09:50:03 2022] DELETE /identity/v3/projects/5ac090dbae29480d994b2dbd38dbb153 => generated 0 bytes in 258 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 09:50:03.342833 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e47dc5-4f58-4f1e-9bfe-2eda36a26bee None None] REQUEST_METHOD: `POST` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:50:03.343137 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e47dc5-4f58-4f1e-9bfe-2eda36a26bee None None] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:50:03.343622 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e47dc5-4f58-4f1e-9bfe-2eda36a26bee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:50:03.402618 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.auth.core [None req-c9e47dc5-4f58-4f1e-9bfe-2eda36a26bee None None] MFA Rules not processed for user `fd836cf590a145df9acf19cf3ae2ac56`. Rule list: `[]` (Enabled: `True`). {{(pid=88113) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:50:03.404987 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.fernet_utils [None req-c9e47dc5-4f58-4f1e-9bfe-2eda36a26bee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:50:03.505442 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 1127/2253] 10.222.0.13 () {58 vars in 958 bytes} [Wed Aug 31 09:50:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:50:03.518864 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f59b189-1843-4d56-922c-564219121785 None None] REQUEST_METHOD: `POST` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:50:03.519281 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f59b189-1843-4d56-922c-564219121785 None None] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:50:03.519672 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f59b189-1843-4d56-922c-564219121785 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:50:03.542722 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.auth.core [None req-7f59b189-1843-4d56-922c-564219121785 None None] MFA Rules not processed for user `fd836cf590a145df9acf19cf3ae2ac56`. Rule list: `[]` (Enabled: `True`). {{(pid=88114) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:50:03.544010 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.fernet_utils [None req-7f59b189-1843-4d56-922c-564219121785 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88114) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:50:03.553431 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 1127/2254] 10.222.0.13 () {58 vars in 958 bytes} [Wed Aug 31 09:50:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 09:50:03.557961 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e47dc5-4f58-4f1e-9bfe-2eda36a26bee None None] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:50:03.570425 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e47dc5-4f58-4f1e-9bfe-2eda36a26bee None None] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:50:03.579662 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05af427b-bb8e-4ba4-93ac-7f6c5564afbb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:50:03.580277 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05af427b-bb8e-4ba4-93ac-7f6c5564afbb admin admin] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:50:03.580603 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05af427b-bb8e-4ba4-93ac-7f6c5564afbb admin admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:50:03.580924 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05af427b-bb8e-4ba4-93ac-7f6c5564afbb admin admin] PATH_INFO: `/v3/domains` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:50:03.581705 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05af427b-bb8e-4ba4-93ac-7f6c5564afbb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88113) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:50:03.582100 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05af427b-bb8e-4ba4-93ac-7f6c5564afbb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:50:03.583864 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05af427b-bb8e-4ba4-93ac-7f6c5564afbb admin admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:50:03.606858 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 1128/2255] 10.222.0.13 () {58 vars in 1153 bytes} [Wed Aug 31 09:50:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:50:03.613003 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f59b189-1843-4d56-922c-564219121785 None None] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:50:03.626821 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f59b189-1843-4d56-922c-564219121785 None None] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:50:03.632408 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbab5989-9350-4e11-b0aa-b068f777e712 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:50:03.633043 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbab5989-9350-4e11-b0aa-b068f777e712 admin admin] REQUEST_METHOD: `POST` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:50:03.633315 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbab5989-9350-4e11-b0aa-b068f777e712 admin admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:50:03.633613 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbab5989-9350-4e11-b0aa-b068f777e712 admin admin] PATH_INFO: `/v3/projects` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:50:03.634271 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbab5989-9350-4e11-b0aa-b068f777e712 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:50:03.636279 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbab5989-9350-4e11-b0aa-b068f777e712 admin admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:50:03.696156 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 1128/2256] 10.222.0.13 () {60 vars in 1153 bytes} [Wed Aug 31 09:50:03 2022] POST /identity/v3/projects => generated 377 bytes in 84 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 09:50:03.701075 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05af427b-bb8e-4ba4-93ac-7f6c5564afbb admin admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:50:03.723057 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05af427b-bb8e-4ba4-93ac-7f6c5564afbb admin admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:50:03.763384 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:50:03.806492 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4246ddb-60f5-4101-a01b-9f049f2b57a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:50:03.807144 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4246ddb-60f5-4101-a01b-9f049f2b57a7 admin admin] REQUEST_METHOD: `POST` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:50:03.807421 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4246ddb-60f5-4101-a01b-9f049f2b57a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:50:03.807692 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4246ddb-60f5-4101-a01b-9f049f2b57a7 admin admin] PATH_INFO: `/v3/users` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:50:03.808492 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4246ddb-60f5-4101-a01b-9f049f2b57a7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:50:03.810922 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4246ddb-60f5-4101-a01b-9f049f2b57a7 admin admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:50:03.840113 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 1129/2257] 10.222.0.13 () {60 vars in 1144 bytes} [Wed Aug 31 09:50:03 2022] POST /identity/v3/users => generated 289 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 09:50:03.851167 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbab5989-9350-4e11-b0aa-b068f777e712 admin admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:50:03.870039 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbab5989-9350-4e11-b0aa-b068f777e712 admin admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:50:03.879077 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d92d78d-0e8c-4aa5-a834-d1ebb0da6c19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:50:03.879768 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d92d78d-0e8c-4aa5-a834-d1ebb0da6c19 admin admin] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:50:03.880031 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d92d78d-0e8c-4aa5-a834-d1ebb0da6c19 admin admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:50:03.880320 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d92d78d-0e8c-4aa5-a834-d1ebb0da6c19 admin admin] PATH_INFO: `/v3/roles` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:50:03.880977 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d92d78d-0e8c-4aa5-a834-d1ebb0da6c19 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:50:03.882704 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d92d78d-0e8c-4aa5-a834-d1ebb0da6c19 admin admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:50:03.896370 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 1129/2258] 10.222.0.13 () {58 vars in 1122 bytes} [Wed Aug 31 09:50:03 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 09:50:03.901334 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4246ddb-60f5-4101-a01b-9f049f2b57a7 admin admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:50:03.933701 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4246ddb-60f5-4101-a01b-9f049f2b57a7 admin admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:50:03.939345 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abcd4f03-1fef-408d-92d6-287f8ab42f1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:50:03.940052 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abcd4f03-1fef-408d-92d6-287f8ab42f1a admin admin] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:50:03.940374 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abcd4f03-1fef-408d-92d6-287f8ab42f1a admin admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:50:03.940724 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abcd4f03-1fef-408d-92d6-287f8ab42f1a admin admin] PATH_INFO: `/v3/roles` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:50:03.941652 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abcd4f03-1fef-408d-92d6-287f8ab42f1a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:50:03.943391 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abcd4f03-1fef-408d-92d6-287f8ab42f1a admin admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:50:03.957602 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 1130/2259] 10.222.0.13 () {58 vars in 1122 bytes} [Wed Aug 31 09:50:03 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 09:50:03.963245 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d92d78d-0e8c-4aa5-a834-d1ebb0da6c19 admin admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:50:03.982201 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d92d78d-0e8c-4aa5-a834-d1ebb0da6c19 admin admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:50:03.987854 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b074a6b3-6c0e-4152-ae23-844bc4e9dcdf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:50:03.988580 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b074a6b3-6c0e-4152-ae23-844bc4e9dcdf admin admin] REQUEST_METHOD: `PUT` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:50:03.988863 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b074a6b3-6c0e-4152-ae23-844bc4e9dcdf admin admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:50:03.989132 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b074a6b3-6c0e-4152-ae23-844bc4e9dcdf admin admin] PATH_INFO: `/v3/projects/5eaf9a46dc28462b8bcba413107270e2/users/3b84237e47a94a8fa95a175bacb02b83/roles/6c99f5d9f1f243eaa134d8540f21095d` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:50:04.008536 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b074a6b3-6c0e-4152-ae23-844bc4e9dcdf admin admin] RBAC: Authorizing `identity:create_grant(project_id=5eaf9a46dc28462b8bcba413107270e2, user_id=3b84237e47a94a8fa95a175bacb02b83, role_id=6c99f5d9f1f243eaa134d8540f21095d)` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:50:04.011556 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b074a6b3-6c0e-4152-ae23-844bc4e9dcdf admin admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:50:04.035987 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 1130/2260] 10.222.0.13 () {60 vars in 1483 bytes} [Wed Aug 31 09:50:03 2022] PUT /identity/v3/projects/5eaf9a46dc28462b8bcba413107270e2/users/3b84237e47a94a8fa95a175bacb02b83/roles/6c99f5d9f1f243eaa134d8540f21095d => generated 0 bytes in 74 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:50:04.055932 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f05d7d0-710b-4598-b54a-c49798f0e57e None None] REQUEST_METHOD: `POST` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:50:04.056217 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f05d7d0-710b-4598-b54a-c49798f0e57e None None] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:50:04.056470 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f05d7d0-710b-4598-b54a-c49798f0e57e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:50:04.092187 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.auth.core [None req-3f05d7d0-710b-4598-b54a-c49798f0e57e None None] MFA Rules not processed for user `3b84237e47a94a8fa95a175bacb02b83`. Rule list: `[]` (Enabled: `True`). {{(pid=88113) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:50:04.093456 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.fernet_utils [None req-3f05d7d0-710b-4598-b54a-c49798f0e57e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:50:04.180233 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 1131/2261] 10.222.0.13 () {58 vars in 958 bytes} [Wed Aug 31 09:50:04 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 09:50:04.194418 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b074a6b3-6c0e-4152-ae23-844bc4e9dcdf admin admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:50:04.198711 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.fernet_utils [None req-b074a6b3-6c0e-4152-ae23-844bc4e9dcdf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88114) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:50:04.271772 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b074a6b3-6c0e-4152-ae23-844bc4e9dcdf admin admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:50:04.279507 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7272650-47ad-4504-aabb-f60e90600d4f service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be216d156b54452cbbf978c7734343aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '49548eb44d62415582af820ab66decee', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:50:04.280378 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7272650-47ad-4504-aabb-f60e90600d4f service glance] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:50:04.280739 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7272650-47ad-4504-aabb-f60e90600d4f service glance] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:50:04.281119 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7272650-47ad-4504-aabb-f60e90600d4f service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:50:04.292146 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7272650-47ad-4504-aabb-f60e90600d4f service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:50:04.293688 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7272650-47ad-4504-aabb-f60e90600d4f service glance] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:50:04.305136 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 1131/2262] 10.222.0.13 () {60 vars in 1365 bytes} [Wed Aug 31 09:50:04 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:50:11.373041 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f05d7d0-710b-4598-b54a-c49798f0e57e None None] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:50:11.374949 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.fernet_utils [None req-3f05d7d0-710b-4598-b54a-c49798f0e57e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:50:11.460471 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f05d7d0-710b-4598-b54a-c49798f0e57e None None] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:50:11.474309 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd47ae0b-7d29-44a9-8df0-ecbc00f158de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:50:11.475481 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd47ae0b-7d29-44a9-8df0-ecbc00f158de admin admin] REQUEST_METHOD: `DELETE` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:50:11.475634 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd47ae0b-7d29-44a9-8df0-ecbc00f158de admin admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:50:11.475832 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd47ae0b-7d29-44a9-8df0-ecbc00f158de admin admin] PATH_INFO: `/v3/users/0649160ba37c4f4eba80092d84879f7d` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:50:11.478108 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd47ae0b-7d29-44a9-8df0-ecbc00f158de admin admin] RBAC: Authorizing `identity:delete_user(user_id=0649160ba37c4f4eba80092d84879f7d)` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:50:11.480465 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd47ae0b-7d29-44a9-8df0-ecbc00f158de admin admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:50:11.532453 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=92,keystone:INSERT=3,keystone:DELETE=5 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:50:11.542431 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.notifications [None req-bd47ae0b-7d29-44a9-8df0-ecbc00f158de admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0649160ba37c4f4eba80092d84879f7d'} {{(pid=88113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:50:11.553613 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.notifications [None req-bd47ae0b-7d29-44a9-8df0-ecbc00f158de admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0649160ba37c4f4eba80092d84879f7d'} {{(pid=88113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:50:11.568312 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.notifications [None req-bd47ae0b-7d29-44a9-8df0-ecbc00f158de admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0649160ba37c4f4eba80092d84879f7d'} {{(pid=88113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:50:11.569014 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.notifications [None req-bd47ae0b-7d29-44a9-8df0-ecbc00f158de admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0649160ba37c4f4eba80092d84879f7d'} {{(pid=88113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:50:11.580662 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.notifications [None req-bd47ae0b-7d29-44a9-8df0-ecbc00f158de admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0649160ba37c4f4eba80092d84879f7d'} {{(pid=88113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:50:11.588719 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 1132/2263] 10.222.0.13 () {58 vars in 1224 bytes} [Wed Aug 31 09:50:11 2022] DELETE /identity/v3/users/0649160ba37c4f4eba80092d84879f7d => generated 0 bytes in 216 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:50:11.593333 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686b3f0b-2a4b-4a83-8c93-0b63103fe216 None None] REQUEST_METHOD: `POST` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:50:11.594145 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686b3f0b-2a4b-4a83-8c93-0b63103fe216 None None] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:50:11.594145 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686b3f0b-2a4b-4a83-8c93-0b63103fe216 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:50:11.615122 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.auth.core [None req-686b3f0b-2a4b-4a83-8c93-0b63103fe216 None None] MFA Rules not processed for user `fd836cf590a145df9acf19cf3ae2ac56`. Rule list: `[]` (Enabled: `True`). {{(pid=88114) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:50:11.616357 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.fernet_utils [None req-686b3f0b-2a4b-4a83-8c93-0b63103fe216 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88114) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:50:11.696814 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:DELETE=6,keystone:SELECT=91,keystone:INSERT=3 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:50:11.720254 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 1132/2264] 10.222.0.13 () {58 vars in 958 bytes} [Wed Aug 31 09:50:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 09:50:11.732419 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd47ae0b-7d29-44a9-8df0-ecbc00f158de admin admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:50:11.733970 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.fernet_utils [None req-bd47ae0b-7d29-44a9-8df0-ecbc00f158de admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:50:11.833098 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd47ae0b-7d29-44a9-8df0-ecbc00f158de admin admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:50:11.838419 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-744e2277-401d-42ca-9b6e-1b7c61d28031 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9bdaacd597442c2bcde2c555f2cc140', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '49548eb44d62415582af820ab66decee', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:50:11.839401 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744e2277-401d-42ca-9b6e-1b7c61d28031 service neutron] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:50:11.839802 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744e2277-401d-42ca-9b6e-1b7c61d28031 service neutron] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:50:11.841899 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744e2277-401d-42ca-9b6e-1b7c61d28031 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:50:11.857438 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-744e2277-401d-42ca-9b6e-1b7c61d28031 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:50:11.860428 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-744e2277-401d-42ca-9b6e-1b7c61d28031 service neutron] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:50:11.872242 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 1133/2265] 10.222.0.13 () {60 vars in 1365 bytes} [Wed Aug 31 09:50:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:50:12.388513 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-686b3f0b-2a4b-4a83-8c93-0b63103fe216 None None] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:50:12.389761 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.fernet_utils [None req-686b3f0b-2a4b-4a83-8c93-0b63103fe216 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88114) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:50:12.413792 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-686b3f0b-2a4b-4a83-8c93-0b63103fe216 None None] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:50:12.419993 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb243e13-279c-47f5-91a7-433543813bab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:50:12.420628 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb243e13-279c-47f5-91a7-433543813bab admin admin] REQUEST_METHOD: `DELETE` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:50:12.420963 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb243e13-279c-47f5-91a7-433543813bab admin admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:50:12.421162 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb243e13-279c-47f5-91a7-433543813bab admin admin] PATH_INFO: `/v3/projects/340cdbf71a7a4b82b5250c53611908d7` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:50:12.423977 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb243e13-279c-47f5-91a7-433543813bab admin admin] RBAC: Authorizing `identity:delete_project(project_id=340cdbf71a7a4b82b5250c53611908d7)` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:50:12.429256 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb243e13-279c-47f5-91a7-433543813bab admin admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:50:12.493085 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.notifications [None req-eb243e13-279c-47f5-91a7-433543813bab admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '340cdbf71a7a4b82b5250c53611908d7', 'request_id': 'req-eb243e13-279c-47f5-91a7-433543813bab'} {{(pid=88114) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:50:12.500048 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.notifications [None req-eb243e13-279c-47f5-91a7-433543813bab admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '340cdbf71a7a4b82b5250c53611908d7', 'request_id': 'req-eb243e13-279c-47f5-91a7-433543813bab'} {{(pid=88114) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:50:12.507811 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.notifications [None req-eb243e13-279c-47f5-91a7-433543813bab admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '340cdbf71a7a4b82b5250c53611908d7', 'request_id': 'req-eb243e13-279c-47f5-91a7-433543813bab'} {{(pid=88114) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:50:12.522549 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.notifications [None req-eb243e13-279c-47f5-91a7-433543813bab admin admin] The token cache is being invalidate because project 340cdbf71a7a4b82b5250c53611908d7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88114) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 09:50:12.522841 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.notifications [None req-eb243e13-279c-47f5-91a7-433543813bab admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88114) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:50:12.523488 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.notifications [None req-eb243e13-279c-47f5-91a7-433543813bab admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88114) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:50:12.525151 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 1133/2266] 10.222.0.13 () {58 vars in 1233 bytes} [Wed Aug 31 09:50:12 2022] DELETE /identity/v3/projects/340cdbf71a7a4b82b5250c53611908d7 => generated 0 bytes in 137 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:50:12.531477 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724e208e-c8cc-42be-95f9-b7baabe4e317 None None] REQUEST_METHOD: `POST` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:50:12.532179 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724e208e-c8cc-42be-95f9-b7baabe4e317 None None] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:50:12.532303 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724e208e-c8cc-42be-95f9-b7baabe4e317 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:50:12.558016 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.auth.core [None req-724e208e-c8cc-42be-95f9-b7baabe4e317 None None] MFA Rules not processed for user `fd836cf590a145df9acf19cf3ae2ac56`. Rule list: `[]` (Enabled: `True`). {{(pid=88113) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:50:12.559438 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.fernet_utils [None req-724e208e-c8cc-42be-95f9-b7baabe4e317 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:50:12.655140 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 1134/2267] 10.222.0.13 () {58 vars in 958 bytes} [Wed Aug 31 09:50:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 09:50:12.664771 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3050c6-c447-4812-81b9-412c22a0a3d4 None None] REQUEST_METHOD: `POST` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:50:12.665678 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3050c6-c447-4812-81b9-412c22a0a3d4 None None] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:50:12.665850 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3050c6-c447-4812-81b9-412c22a0a3d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:50:12.692338 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.auth.core [None req-da3050c6-c447-4812-81b9-412c22a0a3d4 None None] MFA Rules not processed for user `fd836cf590a145df9acf19cf3ae2ac56`. Rule list: `[]` (Enabled: `True`). {{(pid=88114) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:50:12.694138 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.fernet_utils [None req-da3050c6-c447-4812-81b9-412c22a0a3d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88114) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:50:12.702654 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 1134/2268] 10.222.0.13 () {58 vars in 958 bytes} [Wed Aug 31 09:50:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:50:12.708499 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-724e208e-c8cc-42be-95f9-b7baabe4e317 None None] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:50:12.724508 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-724e208e-c8cc-42be-95f9-b7baabe4e317 None None] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:50:12.730110 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c3b00c9-2344-4809-bf06-280203745ece admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:50:12.730838 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3b00c9-2344-4809-bf06-280203745ece admin admin] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:50:12.731178 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3b00c9-2344-4809-bf06-280203745ece admin admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:50:12.731378 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3b00c9-2344-4809-bf06-280203745ece admin admin] PATH_INFO: `/v3/domains` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:50:12.733345 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c3b00c9-2344-4809-bf06-280203745ece admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88113) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:50:12.735409 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c3b00c9-2344-4809-bf06-280203745ece admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:50:12.736365 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c3b00c9-2344-4809-bf06-280203745ece admin admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:50:12.750392 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 1135/2269] 10.222.0.13 () {58 vars in 1153 bytes} [Wed Aug 31 09:50:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 09:50:12.754707 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da3050c6-c447-4812-81b9-412c22a0a3d4 None None] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:50:12.764193 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da3050c6-c447-4812-81b9-412c22a0a3d4 None None] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:50:12.769916 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd6bf764-c3b3-4fff-b12c-c34101ec1f38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:50:12.770533 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6bf764-c3b3-4fff-b12c-c34101ec1f38 admin admin] REQUEST_METHOD: `POST` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:50:12.770810 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6bf764-c3b3-4fff-b12c-c34101ec1f38 admin admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:50:12.771073 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6bf764-c3b3-4fff-b12c-c34101ec1f38 admin admin] PATH_INFO: `/v3/projects` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:50:12.771708 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd6bf764-c3b3-4fff-b12c-c34101ec1f38 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:50:12.773628 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd6bf764-c3b3-4fff-b12c-c34101ec1f38 admin admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:50:12.806364 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 1135/2270] 10.222.0.13 () {60 vars in 1153 bytes} [Wed Aug 31 09:50:12 2022] POST /identity/v3/projects => generated 385 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 09:50:12.812317 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c3b00c9-2344-4809-bf06-280203745ece admin admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:50:12.821973 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c3b00c9-2344-4809-bf06-280203745ece admin admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:50:12.884405 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:DELETE=5,keystone:SELECT=94,keystone:INSERT=1 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:50:12.905123 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0562ef73-4c54-43ab-8cf3-095a12e2a86f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:50:12.905812 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0562ef73-4c54-43ab-8cf3-095a12e2a86f admin admin] REQUEST_METHOD: `POST` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:50:12.906769 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0562ef73-4c54-43ab-8cf3-095a12e2a86f admin admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:50:12.907229 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0562ef73-4c54-43ab-8cf3-095a12e2a86f admin admin] PATH_INFO: `/v3/users` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:50:12.907897 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0562ef73-4c54-43ab-8cf3-095a12e2a86f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:50:12.909992 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0562ef73-4c54-43ab-8cf3-095a12e2a86f admin admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:50:12.940019 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 1136/2271] 10.222.0.13 () {60 vars in 1144 bytes} [Wed Aug 31 09:50:12 2022] POST /identity/v3/users => generated 293 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 09:50:12.944277 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd6bf764-c3b3-4fff-b12c-c34101ec1f38 admin admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:50:12.959072 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd6bf764-c3b3-4fff-b12c-c34101ec1f38 admin admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:50:12.964215 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f2266e-e68b-4509-b472-cddb81f3c548 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:50:12.964842 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f2266e-e68b-4509-b472-cddb81f3c548 admin admin] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:50:12.965115 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f2266e-e68b-4509-b472-cddb81f3c548 admin admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:50:12.965405 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f2266e-e68b-4509-b472-cddb81f3c548 admin admin] PATH_INFO: `/v3/roles` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:50:12.966182 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00f2266e-e68b-4509-b472-cddb81f3c548 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:50:12.968619 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00f2266e-e68b-4509-b472-cddb81f3c548 admin admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:50:12.996845 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 1136/2272] 10.222.0.13 () {58 vars in 1122 bytes} [Wed Aug 31 09:50:12 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 09:50:13.003182 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0562ef73-4c54-43ab-8cf3-095a12e2a86f admin admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:50:13.016276 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0562ef73-4c54-43ab-8cf3-095a12e2a86f admin admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:50:13.022286 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a5a0c9b-0968-4572-8f2c-1ec41af22f73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:50:13.022969 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a5a0c9b-0968-4572-8f2c-1ec41af22f73 admin admin] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:50:13.023334 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a5a0c9b-0968-4572-8f2c-1ec41af22f73 admin admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:50:13.023902 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a5a0c9b-0968-4572-8f2c-1ec41af22f73 admin admin] PATH_INFO: `/v3/roles` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:50:13.025357 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a5a0c9b-0968-4572-8f2c-1ec41af22f73 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:50:13.027768 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a5a0c9b-0968-4572-8f2c-1ec41af22f73 admin admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:50:13.039856 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 1137/2273] 10.222.0.13 () {58 vars in 1122 bytes} [Wed Aug 31 09:50:13 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 09:50:13.044222 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f2266e-e68b-4509-b472-cddb81f3c548 admin admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:50:13.054682 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f2266e-e68b-4509-b472-cddb81f3c548 admin admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:50:13.059584 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7190275a-637d-49a8-a6d5-9c55b49f03cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:50:13.060252 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7190275a-637d-49a8-a6d5-9c55b49f03cb admin admin] REQUEST_METHOD: `PUT` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:50:13.060527 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7190275a-637d-49a8-a6d5-9c55b49f03cb admin admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:50:13.060793 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7190275a-637d-49a8-a6d5-9c55b49f03cb admin admin] PATH_INFO: `/v3/projects/ff060921d1dc4d26a4c8e8ec2635b5f8/users/c419c19ee29a49a9b5745db420207817/roles/6c99f5d9f1f243eaa134d8540f21095d` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:50:13.079962 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7190275a-637d-49a8-a6d5-9c55b49f03cb admin admin] RBAC: Authorizing `identity:create_grant(project_id=ff060921d1dc4d26a4c8e8ec2635b5f8, user_id=c419c19ee29a49a9b5745db420207817, role_id=6c99f5d9f1f243eaa134d8540f21095d)` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:50:13.083032 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7190275a-637d-49a8-a6d5-9c55b49f03cb admin admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:50:13.109728 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 1137/2274] 10.222.0.13 () {60 vars in 1483 bytes} [Wed Aug 31 09:50:13 2022] PUT /identity/v3/projects/ff060921d1dc4d26a4c8e8ec2635b5f8/users/c419c19ee29a49a9b5745db420207817/roles/6c99f5d9f1f243eaa134d8540f21095d => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 09:50:13.129051 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255d76b1-42e1-425b-9372-3dab1fa66799 None None] REQUEST_METHOD: `POST` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:50:13.129349 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255d76b1-42e1-425b-9372-3dab1fa66799 None None] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:50:13.129633 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255d76b1-42e1-425b-9372-3dab1fa66799 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:50:13.164498 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.auth.core [None req-255d76b1-42e1-425b-9372-3dab1fa66799 None None] MFA Rules not processed for user `c419c19ee29a49a9b5745db420207817`. Rule list: `[]` (Enabled: `True`). {{(pid=88113) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:50:13.166256 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.fernet_utils [None req-255d76b1-42e1-425b-9372-3dab1fa66799 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:50:13.294749 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 1138/2275] 10.222.0.13 () {58 vars in 958 bytes} [Wed Aug 31 09:50:13 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:50:13.301805 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7190275a-637d-49a8-a6d5-9c55b49f03cb admin admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:50:13.402172 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7190275a-637d-49a8-a6d5-9c55b49f03cb admin admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:50:13.414726 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b69782-7ce6-4d2d-a6b9-093f9a83fe08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:50:13.415476 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b69782-7ce6-4d2d-a6b9-093f9a83fe08 admin admin] REQUEST_METHOD: `POST` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:50:13.415758 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b69782-7ce6-4d2d-a6b9-093f9a83fe08 admin admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:50:13.416029 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b69782-7ce6-4d2d-a6b9-093f9a83fe08 admin admin] PATH_INFO: `/v3/projects` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:50:13.416673 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2b69782-7ce6-4d2d-a6b9-093f9a83fe08 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:50:13.420952 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2b69782-7ce6-4d2d-a6b9-093f9a83fe08 admin admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:50:13.448155 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:50:13.462335 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-255d76b1-42e1-425b-9372-3dab1fa66799 None None] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:50:13.469518 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 1138/2276] 10.222.0.13 () {60 vars in 1153 bytes} [Wed Aug 31 09:50:13 2022] POST /identity/v3/projects => generated 383 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 09:50:13.495438 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-255d76b1-42e1-425b-9372-3dab1fa66799 None None] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:50:13.573093 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f941679a-180c-4c32-889e-a71ed81f7486 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:50:13.573810 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f941679a-180c-4c32-889e-a71ed81f7486 admin admin] REQUEST_METHOD: `POST` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:50:13.574033 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f941679a-180c-4c32-889e-a71ed81f7486 admin admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:50:13.574304 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f941679a-180c-4c32-889e-a71ed81f7486 admin admin] PATH_INFO: `/v3/users` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:50:13.574951 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f941679a-180c-4c32-889e-a71ed81f7486 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:50:13.576904 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f941679a-180c-4c32-889e-a71ed81f7486 admin admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:50:13.609928 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 1139/2277] 10.222.0.13 () {60 vars in 1144 bytes} [Wed Aug 31 09:50:13 2022] POST /identity/v3/users => generated 292 bytes in 145 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 09:50:13.611918 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b69782-7ce6-4d2d-a6b9-093f9a83fe08 admin admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:50:13.630203 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b69782-7ce6-4d2d-a6b9-093f9a83fe08 admin admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:50:13.652090 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-833c648c-38cb-461e-8bf7-3c2c2496c642 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:50:13.652090 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833c648c-38cb-461e-8bf7-3c2c2496c642 admin admin] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:50:13.652090 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833c648c-38cb-461e-8bf7-3c2c2496c642 admin admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:50:13.652090 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833c648c-38cb-461e-8bf7-3c2c2496c642 admin admin] PATH_INFO: `/v3/roles` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:50:13.652682 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-833c648c-38cb-461e-8bf7-3c2c2496c642 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:50:13.652682 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-833c648c-38cb-461e-8bf7-3c2c2496c642 admin admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:50:13.664971 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 1139/2278] 10.222.0.13 () {58 vars in 1122 bytes} [Wed Aug 31 09:50:13 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 09:50:13.666183 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f941679a-180c-4c32-889e-a71ed81f7486 admin admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:50:13.684243 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f941679a-180c-4c32-889e-a71ed81f7486 admin admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:50:13.696133 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9218c0e8-eb9c-4596-a9b1-139268b740ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:50:13.696133 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9218c0e8-eb9c-4596-a9b1-139268b740ad admin admin] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:50:13.696133 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9218c0e8-eb9c-4596-a9b1-139268b740ad admin admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:50:13.696133 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9218c0e8-eb9c-4596-a9b1-139268b740ad admin admin] PATH_INFO: `/v3/roles` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:50:13.696569 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9218c0e8-eb9c-4596-a9b1-139268b740ad admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:50:13.696569 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9218c0e8-eb9c-4596-a9b1-139268b740ad admin admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:50:13.710552 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 1140/2279] 10.222.0.13 () {58 vars in 1122 bytes} [Wed Aug 31 09:50:13 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 09:50:13.712597 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-833c648c-38cb-461e-8bf7-3c2c2496c642 admin admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:50:13.727458 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-833c648c-38cb-461e-8bf7-3c2c2496c642 admin admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:50:13.733544 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff651dbd-4945-4d91-91f2-e1adddd531e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:50:13.735094 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff651dbd-4945-4d91-91f2-e1adddd531e2 admin admin] REQUEST_METHOD: `PUT` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:50:13.736117 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff651dbd-4945-4d91-91f2-e1adddd531e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:50:13.736667 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff651dbd-4945-4d91-91f2-e1adddd531e2 admin admin] PATH_INFO: `/v3/projects/ee75f580391a43d480184849d6ab35f3/users/656c873df2c54bca82a409baf8831b4e/roles/6c99f5d9f1f243eaa134d8540f21095d` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:50:13.762452 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff651dbd-4945-4d91-91f2-e1adddd531e2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ee75f580391a43d480184849d6ab35f3, user_id=656c873df2c54bca82a409baf8831b4e, role_id=6c99f5d9f1f243eaa134d8540f21095d)` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:50:13.766723 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff651dbd-4945-4d91-91f2-e1adddd531e2 admin admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:50:13.792444 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 1140/2280] 10.222.0.13 () {60 vars in 1483 bytes} [Wed Aug 31 09:50:13 2022] PUT /identity/v3/projects/ee75f580391a43d480184849d6ab35f3/users/656c873df2c54bca82a409baf8831b4e/roles/6c99f5d9f1f243eaa134d8540f21095d => generated 0 bytes in 80 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:50:13.805446 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9df6cb6-6f71-4d36-a34c-b027eb68ce42 None None] REQUEST_METHOD: `POST` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:50:13.805837 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9df6cb6-6f71-4d36-a34c-b027eb68ce42 None None] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:50:13.806396 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9df6cb6-6f71-4d36-a34c-b027eb68ce42 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:50:13.843965 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.auth.core [None req-c9df6cb6-6f71-4d36-a34c-b027eb68ce42 None None] MFA Rules not processed for user `656c873df2c54bca82a409baf8831b4e`. Rule list: `[]` (Enabled: `True`). {{(pid=88113) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:50:13.845361 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.fernet_utils [None req-c9df6cb6-6f71-4d36-a34c-b027eb68ce42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:50:13.878235 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:50:13.933036 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 1141/2281] 10.222.0.13 () {58 vars in 958 bytes} [Wed Aug 31 09:50:13 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:50:13.939762 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff651dbd-4945-4d91-91f2-e1adddd531e2 admin admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:50:14.009552 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff651dbd-4945-4d91-91f2-e1adddd531e2 admin admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:50:14.015759 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f2f095-fbd8-4873-a7b9-6c5f42c5a784 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:50:14.017209 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f2f095-fbd8-4873-a7b9-6c5f42c5a784 admin admin] REQUEST_METHOD: `POST` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:50:14.017706 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f2f095-fbd8-4873-a7b9-6c5f42c5a784 admin admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:50:14.018041 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f2f095-fbd8-4873-a7b9-6c5f42c5a784 admin admin] PATH_INFO: `/v3/projects` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:50:14.018836 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58f2f095-fbd8-4873-a7b9-6c5f42c5a784 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:50:14.021562 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58f2f095-fbd8-4873-a7b9-6c5f42c5a784 admin admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:50:14.063797 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 1141/2282] 10.222.0.13 () {60 vars in 1153 bytes} [Wed Aug 31 09:50:13 2022] POST /identity/v3/projects => generated 383 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 09:50:14.069110 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9df6cb6-6f71-4d36-a34c-b027eb68ce42 None None] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:50:14.081061 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9df6cb6-6f71-4d36-a34c-b027eb68ce42 None None] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:50:14.153654 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6200d021-7209-463c-a21d-cc0673d901b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:50:14.154281 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6200d021-7209-463c-a21d-cc0673d901b3 admin admin] REQUEST_METHOD: `POST` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:50:14.154658 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6200d021-7209-463c-a21d-cc0673d901b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:50:14.154886 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6200d021-7209-463c-a21d-cc0673d901b3 admin admin] PATH_INFO: `/v3/users` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:50:14.155491 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6200d021-7209-463c-a21d-cc0673d901b3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:50:14.158229 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6200d021-7209-463c-a21d-cc0673d901b3 admin admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:50:14.193763 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 1142/2283] 10.222.0.13 () {60 vars in 1144 bytes} [Wed Aug 31 09:50:14 2022] POST /identity/v3/users => generated 298 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 09:50:14.198301 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f2f095-fbd8-4873-a7b9-6c5f42c5a784 admin admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:50:14.209599 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f2f095-fbd8-4873-a7b9-6c5f42c5a784 admin admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:50:14.214601 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2848d5e7-613d-4293-92cf-7546976c96ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:50:14.215204 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2848d5e7-613d-4293-92cf-7546976c96ca admin admin] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:50:14.215479 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2848d5e7-613d-4293-92cf-7546976c96ca admin admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:50:14.215747 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2848d5e7-613d-4293-92cf-7546976c96ca admin admin] PATH_INFO: `/v3/roles` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:50:14.216446 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2848d5e7-613d-4293-92cf-7546976c96ca admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:50:14.218097 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2848d5e7-613d-4293-92cf-7546976c96ca admin admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:50:14.232051 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 1142/2284] 10.222.0.13 () {58 vars in 1122 bytes} [Wed Aug 31 09:50:14 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 09:50:14.236986 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6200d021-7209-463c-a21d-cc0673d901b3 admin admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:50:14.247173 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6200d021-7209-463c-a21d-cc0673d901b3 admin admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:50:14.252907 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cffd010-03e8-46f8-86d5-f2db0f98fba2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:50:14.253609 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cffd010-03e8-46f8-86d5-f2db0f98fba2 admin admin] REQUEST_METHOD: `PUT` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:50:14.253890 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cffd010-03e8-46f8-86d5-f2db0f98fba2 admin admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:50:14.254171 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cffd010-03e8-46f8-86d5-f2db0f98fba2 admin admin] PATH_INFO: `/v3/projects/cc64571db46543b7a20198f3e3e73a95/users/f80213dfdd0a4427b47313ab6adbe3b0/roles/5d4ee63a3add47fc9a6b8b16d39e0a1f` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:50:14.271342 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cffd010-03e8-46f8-86d5-f2db0f98fba2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cc64571db46543b7a20198f3e3e73a95, user_id=f80213dfdd0a4427b47313ab6adbe3b0, role_id=5d4ee63a3add47fc9a6b8b16d39e0a1f)` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:50:14.274063 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cffd010-03e8-46f8-86d5-f2db0f98fba2 admin admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:50:14.298738 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 1143/2285] 10.222.0.13 () {60 vars in 1483 bytes} [Wed Aug 31 09:50:14 2022] PUT /identity/v3/projects/cc64571db46543b7a20198f3e3e73a95/users/f80213dfdd0a4427b47313ab6adbe3b0/roles/5d4ee63a3add47fc9a6b8b16d39e0a1f => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 09:50:14.303003 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2848d5e7-613d-4293-92cf-7546976c96ca admin admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:50:14.314884 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2848d5e7-613d-4293-92cf-7546976c96ca admin admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:50:14.397942 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb955a1-585a-4153-84d1-c30b3402bd15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:50:14.398597 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb955a1-585a-4153-84d1-c30b3402bd15 admin admin] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:50:14.398903 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb955a1-585a-4153-84d1-c30b3402bd15 admin admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:50:14.399172 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb955a1-585a-4153-84d1-c30b3402bd15 admin admin] PATH_INFO: `/v3/roles` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:50:14.399843 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efb955a1-585a-4153-84d1-c30b3402bd15 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:50:14.401724 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efb955a1-585a-4153-84d1-c30b3402bd15 admin admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:50:14.413933 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 1143/2286] 10.222.0.13 () {58 vars in 1122 bytes} [Wed Aug 31 09:50:14 2022] GET /identity/v3/roles => generated 1420 bytes in 112 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 09:50:14.418794 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cffd010-03e8-46f8-86d5-f2db0f98fba2 admin admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:50:14.430233 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cffd010-03e8-46f8-86d5-f2db0f98fba2 admin admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:50:14.436652 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd61ef5-21ca-4cda-a71a-053c21670c34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:50:14.437822 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd61ef5-21ca-4cda-a71a-053c21670c34 admin admin] REQUEST_METHOD: `PUT` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:50:14.438393 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd61ef5-21ca-4cda-a71a-053c21670c34 admin admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:50:14.438853 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd61ef5-21ca-4cda-a71a-053c21670c34 admin admin] PATH_INFO: `/v3/projects/cc64571db46543b7a20198f3e3e73a95/users/f80213dfdd0a4427b47313ab6adbe3b0/roles/5d4ee63a3add47fc9a6b8b16d39e0a1f` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:50:14.456297 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cd61ef5-21ca-4cda-a71a-053c21670c34 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cc64571db46543b7a20198f3e3e73a95, user_id=f80213dfdd0a4427b47313ab6adbe3b0, role_id=5d4ee63a3add47fc9a6b8b16d39e0a1f)` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:50:14.456297 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cd61ef5-21ca-4cda-a71a-053c21670c34 admin admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:50:14.462470 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 1144/2287] 10.222.0.13 () {60 vars in 1483 bytes} [Wed Aug 31 09:50:14 2022] PUT /identity/v3/projects/cc64571db46543b7a20198f3e3e73a95/users/f80213dfdd0a4427b47313ab6adbe3b0/roles/5d4ee63a3add47fc9a6b8b16d39e0a1f => generated 0 bytes in 44 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:50:14.474207 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a0e009-031d-480c-a534-26643d4b0fac None None] REQUEST_METHOD: `POST` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:50:14.474487 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a0e009-031d-480c-a534-26643d4b0fac None None] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:50:14.474769 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a0e009-031d-480c-a534-26643d4b0fac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:50:14.524967 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.auth.core [None req-27a0e009-031d-480c-a534-26643d4b0fac None None] MFA Rules not processed for user `f80213dfdd0a4427b47313ab6adbe3b0`. Rule list: `[]` (Enabled: `True`). {{(pid=88114) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:50:14.527672 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.fernet_utils [None req-27a0e009-031d-480c-a534-26643d4b0fac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88114) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:50:14.583138 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:50:14.650516 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 1144/2288] 10.222.0.13 () {58 vars in 958 bytes} [Wed Aug 31 09:50:14 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:50:14.670672 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd61ef5-21ca-4cda-a71a-053c21670c34 admin admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:50:14.673231 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.fernet_utils [None req-0cd61ef5-21ca-4cda-a71a-053c21670c34 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:50:14.748107 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd61ef5-21ca-4cda-a71a-053c21670c34 admin admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:50:14.753626 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d4af04a-a8b8-426d-a413-b5a5c45fd81d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d52b69c8e94441dacfcff944326f169', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '49548eb44d62415582af820ab66decee', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:50:14.754245 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4af04a-a8b8-426d-a413-b5a5c45fd81d service cinder] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:50:14.754523 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4af04a-a8b8-426d-a413-b5a5c45fd81d service cinder] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:50:14.754788 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4af04a-a8b8-426d-a413-b5a5c45fd81d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:50:14.789856 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:50:14.839987 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d4af04a-a8b8-426d-a413-b5a5c45fd81d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:50:14.841547 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d4af04a-a8b8-426d-a413-b5a5c45fd81d service cinder] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:50:14.851929 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 1145/2289] 10.222.0.13 () {60 vars in 1365 bytes} [Wed Aug 31 09:50:14 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 182 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:50:17.717279 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a0e009-031d-480c-a534-26643d4b0fac None None] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:50:17.731913 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a0e009-031d-480c-a534-26643d4b0fac None None] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:50:17.736781 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-225909e1-76af-4284-b61b-94fb6f71492f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d52b69c8e94441dacfcff944326f169', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '49548eb44d62415582af820ab66decee', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:50:17.738934 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225909e1-76af-4284-b61b-94fb6f71492f service cinder] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:50:17.738934 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225909e1-76af-4284-b61b-94fb6f71492f service cinder] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:50:17.738934 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225909e1-76af-4284-b61b-94fb6f71492f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:50:17.835858 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-225909e1-76af-4284-b61b-94fb6f71492f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:50:17.837341 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-225909e1-76af-4284-b61b-94fb6f71492f service cinder] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:50:17.849184 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 1145/2290] 10.222.0.13 () {60 vars in 1365 bytes} [Wed Aug 31 09:50:17 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:50:18.893644 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d4af04a-a8b8-426d-a413-b5a5c45fd81d service cinder] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:50:18.903000 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d4af04a-a8b8-426d-a413-b5a5c45fd81d service cinder] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:50:18.910225 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d20fc8ea-af8b-4daa-b735-8e030d9ba80b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d52b69c8e94441dacfcff944326f169', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '49548eb44d62415582af820ab66decee', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:50:18.910845 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20fc8ea-af8b-4daa-b735-8e030d9ba80b service cinder] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:50:18.911125 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20fc8ea-af8b-4daa-b735-8e030d9ba80b service cinder] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:50:18.911709 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20fc8ea-af8b-4daa-b735-8e030d9ba80b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:50:18.925566 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d20fc8ea-af8b-4daa-b735-8e030d9ba80b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:50:18.936676 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d20fc8ea-af8b-4daa-b735-8e030d9ba80b service cinder] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:50:18.949881 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 1146/2291] 10.222.0.13 () {60 vars in 1365 bytes} [Wed Aug 31 09:50:18 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 57 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:50:27.847720 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=47 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:50:28.945221 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=20 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:50:36.134976 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa4d2a4-e334-4ce3-81e4-823929e784b2 None None] REQUEST_METHOD: `POST` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:50:36.135565 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa4d2a4-e334-4ce3-81e4-823929e784b2 None None] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:50:36.135872 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa4d2a4-e334-4ce3-81e4-823929e784b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:50:36.162333 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.auth.core [None req-3aa4d2a4-e334-4ce3-81e4-823929e784b2 None None] MFA Rules not processed for user `768cb67c1ef24251aaa113c12d81a250`. Rule list: `[]` (Enabled: `True`). {{(pid=88114) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:50:36.163894 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.fernet_utils [None req-3aa4d2a4-e334-4ce3-81e4-823929e784b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88114) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:50:36.226415 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 1146/2292] 10.222.0.13 () {58 vars in 1009 bytes} [Wed Aug 31 09:50:36 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 09:50:36.295187 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d20fc8ea-af8b-4daa-b735-8e030d9ba80b service cinder] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:50:36.296863 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.fernet_utils [None req-d20fc8ea-af8b-4daa-b735-8e030d9ba80b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:50:36.412518 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d20fc8ea-af8b-4daa-b735-8e030d9ba80b service cinder] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:50:36.422190 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1220fccf-ec4d-49f0-a5f3-608e956e4344 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '202bb9db7a8a4977ba788a290ed9a3fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '49548eb44d62415582af820ab66decee', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:50:36.423227 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1220fccf-ec4d-49f0-a5f3-608e956e4344 service swift] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:50:36.423967 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1220fccf-ec4d-49f0-a5f3-608e956e4344 service swift] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:50:36.423967 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1220fccf-ec4d-49f0-a5f3-608e956e4344 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:50:36.434031 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1220fccf-ec4d-49f0-a5f3-608e956e4344 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:50:36.435701 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1220fccf-ec4d-49f0-a5f3-608e956e4344 service swift] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:50:36.450399 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 1147/2293] 10.222.0.13 () {60 vars in 1384 bytes} [Wed Aug 31 09:50:36 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 156 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 31 09:50:41.649351 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa4d2a4-e334-4ce3-81e4-823929e784b2 None None] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:50:41.750595 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa4d2a4-e334-4ce3-81e4-823929e784b2 None None] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:50:41.770155 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df4e4770-2cfb-46bf-baeb-744164ad8a39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:50:41.770838 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4e4770-2cfb-46bf-baeb-744164ad8a39 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:50:41.771116 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4e4770-2cfb-46bf-baeb-744164ad8a39 admin admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:50:41.771384 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4e4770-2cfb-46bf-baeb-744164ad8a39 admin admin] PATH_INFO: `/v3/users/c419c19ee29a49a9b5745db420207817` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:50:41.772852 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df4e4770-2cfb-46bf-baeb-744164ad8a39 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c419c19ee29a49a9b5745db420207817)` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:50:41.774955 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df4e4770-2cfb-46bf-baeb-744164ad8a39 admin admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:50:41.878914 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.notifications [None req-df4e4770-2cfb-46bf-baeb-744164ad8a39 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c419c19ee29a49a9b5745db420207817'} {{(pid=88114) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:50:41.890888 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.notifications [None req-df4e4770-2cfb-46bf-baeb-744164ad8a39 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c419c19ee29a49a9b5745db420207817'} {{(pid=88114) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:50:41.902217 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.notifications [None req-df4e4770-2cfb-46bf-baeb-744164ad8a39 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c419c19ee29a49a9b5745db420207817'} {{(pid=88114) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:50:41.903011 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.notifications [None req-df4e4770-2cfb-46bf-baeb-744164ad8a39 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c419c19ee29a49a9b5745db420207817'} {{(pid=88114) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:50:41.963362 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.notifications [None req-df4e4770-2cfb-46bf-baeb-744164ad8a39 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c419c19ee29a49a9b5745db420207817'} {{(pid=88114) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:50:41.972721 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 1147/2294] 10.222.0.13 () {58 vars in 1224 bytes} [Wed Aug 31 09:50:41 2022] DELETE /identity/v3/users/c419c19ee29a49a9b5745db420207817 => generated 0 bytes in 324 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:50:41.977412 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e223cce5-8797-407b-b6f4-70f4725049e2 None None] REQUEST_METHOD: `POST` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:50:41.978164 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e223cce5-8797-407b-b6f4-70f4725049e2 None None] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:50:41.978164 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e223cce5-8797-407b-b6f4-70f4725049e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:50:42.001780 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.auth.core [None req-e223cce5-8797-407b-b6f4-70f4725049e2 None None] MFA Rules not processed for user `fd836cf590a145df9acf19cf3ae2ac56`. Rule list: `[]` (Enabled: `True`). {{(pid=88113) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:50:42.003540 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.fernet_utils [None req-e223cce5-8797-407b-b6f4-70f4725049e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:50:42.097634 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 1148/2295] 10.222.0.13 () {58 vars in 958 bytes} [Wed Aug 31 09:50:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:50:42.111670 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df4e4770-2cfb-46bf-baeb-744164ad8a39 admin admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:50:42.113607 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.fernet_utils [None req-df4e4770-2cfb-46bf-baeb-744164ad8a39 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88114) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:50:42.147617 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:50:42.184274 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df4e4770-2cfb-46bf-baeb-744164ad8a39 admin admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:50:42.188078 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65202d13-71e9-4339-b5b3-3364a15e8ee4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9bdaacd597442c2bcde2c555f2cc140', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '49548eb44d62415582af820ab66decee', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:50:42.188873 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65202d13-71e9-4339-b5b3-3364a15e8ee4 service neutron] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:50:42.189362 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65202d13-71e9-4339-b5b3-3364a15e8ee4 service neutron] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:50:42.189744 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65202d13-71e9-4339-b5b3-3364a15e8ee4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:50:42.199825 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65202d13-71e9-4339-b5b3-3364a15e8ee4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:50:42.201280 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65202d13-71e9-4339-b5b3-3364a15e8ee4 service neutron] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:50:42.213176 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 1148/2296] 10.222.0.13 () {60 vars in 1365 bytes} [Wed Aug 31 09:50:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:50:42.748273 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e223cce5-8797-407b-b6f4-70f4725049e2 None None] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:50:42.751626 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.fernet_utils [None req-e223cce5-8797-407b-b6f4-70f4725049e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:50:42.783801 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e223cce5-8797-407b-b6f4-70f4725049e2 None None] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:50:42.792203 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37416e46-c4e0-49d8-b3bb-6cb3ff6fc7c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:50:42.792955 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37416e46-c4e0-49d8-b3bb-6cb3ff6fc7c6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:50:42.793311 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37416e46-c4e0-49d8-b3bb-6cb3ff6fc7c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:50:42.793730 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37416e46-c4e0-49d8-b3bb-6cb3ff6fc7c6 admin admin] PATH_INFO: `/v3/projects/ff060921d1dc4d26a4c8e8ec2635b5f8` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:50:42.795298 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37416e46-c4e0-49d8-b3bb-6cb3ff6fc7c6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ff060921d1dc4d26a4c8e8ec2635b5f8)` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:50:42.797605 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37416e46-c4e0-49d8-b3bb-6cb3ff6fc7c6 admin admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:50:42.886832 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.notifications [None req-37416e46-c4e0-49d8-b3bb-6cb3ff6fc7c6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ff060921d1dc4d26a4c8e8ec2635b5f8', 'request_id': 'req-37416e46-c4e0-49d8-b3bb-6cb3ff6fc7c6'} {{(pid=88113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:50:42.903727 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.notifications [None req-37416e46-c4e0-49d8-b3bb-6cb3ff6fc7c6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ff060921d1dc4d26a4c8e8ec2635b5f8', 'request_id': 'req-37416e46-c4e0-49d8-b3bb-6cb3ff6fc7c6'} {{(pid=88113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:50:42.916790 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.notifications [None req-37416e46-c4e0-49d8-b3bb-6cb3ff6fc7c6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ff060921d1dc4d26a4c8e8ec2635b5f8', 'request_id': 'req-37416e46-c4e0-49d8-b3bb-6cb3ff6fc7c6'} {{(pid=88113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:50:42.945332 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.notifications [None req-37416e46-c4e0-49d8-b3bb-6cb3ff6fc7c6 admin admin] The token cache is being invalidate because project ff060921d1dc4d26a4c8e8ec2635b5f8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88113) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 09:50:42.945652 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.notifications [None req-37416e46-c4e0-49d8-b3bb-6cb3ff6fc7c6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:50:42.946376 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.notifications [None req-37416e46-c4e0-49d8-b3bb-6cb3ff6fc7c6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:50:42.947922 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 1149/2297] 10.222.0.13 () {58 vars in 1233 bytes} [Wed Aug 31 09:50:42 2022] DELETE /identity/v3/projects/ff060921d1dc4d26a4c8e8ec2635b5f8 => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:50:42.952458 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65202d13-71e9-4339-b5b3-3364a15e8ee4 service neutron] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:50:42.967965 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65202d13-71e9-4339-b5b3-3364a15e8ee4 service neutron] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:50:43.039890 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da99f99-3e69-4a5b-931b-ac95a3ba244d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:50:43.040536 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da99f99-3e69-4a5b-931b-ac95a3ba244d admin admin] REQUEST_METHOD: `DELETE` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:50:43.040840 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da99f99-3e69-4a5b-931b-ac95a3ba244d admin admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:50:43.041168 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da99f99-3e69-4a5b-931b-ac95a3ba244d admin admin] PATH_INFO: `/v3/users/656c873df2c54bca82a409baf8831b4e` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:50:43.042367 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0da99f99-3e69-4a5b-931b-ac95a3ba244d admin admin] RBAC: Authorizing `identity:delete_user(user_id=656c873df2c54bca82a409baf8831b4e)` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:50:43.044334 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0da99f99-3e69-4a5b-931b-ac95a3ba244d admin admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:50:43.106017 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.notifications [None req-0da99f99-3e69-4a5b-931b-ac95a3ba244d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '656c873df2c54bca82a409baf8831b4e'} {{(pid=88114) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:50:43.116724 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.notifications [None req-0da99f99-3e69-4a5b-931b-ac95a3ba244d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '656c873df2c54bca82a409baf8831b4e'} {{(pid=88114) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:50:43.129069 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.notifications [None req-0da99f99-3e69-4a5b-931b-ac95a3ba244d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '656c873df2c54bca82a409baf8831b4e'} {{(pid=88114) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:50:43.129950 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.notifications [None req-0da99f99-3e69-4a5b-931b-ac95a3ba244d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '656c873df2c54bca82a409baf8831b4e'} {{(pid=88114) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:50:43.150001 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.notifications [None req-0da99f99-3e69-4a5b-931b-ac95a3ba244d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '656c873df2c54bca82a409baf8831b4e'} {{(pid=88114) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:50:43.165145 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 1149/2298] 10.222.0.13 () {58 vars in 1224 bytes} [Wed Aug 31 09:50:42 2022] DELETE /identity/v3/users/656c873df2c54bca82a409baf8831b4e => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 09:50:43.846644 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37416e46-c4e0-49d8-b3bb-6cb3ff6fc7c6 admin admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:50:43.848569 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.fernet_utils [None req-37416e46-c4e0-49d8-b3bb-6cb3ff6fc7c6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:50:43.883786 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37416e46-c4e0-49d8-b3bb-6cb3ff6fc7c6 admin admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:50:43.889536 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da99f99-3e69-4a5b-931b-ac95a3ba244d admin admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:50:43.892499 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.fernet_utils [None req-0da99f99-3e69-4a5b-931b-ac95a3ba244d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88114) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:50:43.913155 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:50:44.045736 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:50:44.050499 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de3103d1-65f4-4630-9d28-7c3b911773a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:50:44.051856 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3103d1-65f4-4630-9d28-7c3b911773a6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:50:44.052352 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3103d1-65f4-4630-9d28-7c3b911773a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:50:44.053055 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3103d1-65f4-4630-9d28-7c3b911773a6 admin admin] PATH_INFO: `/v3/projects/ee75f580391a43d480184849d6ab35f3` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:50:44.056149 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de3103d1-65f4-4630-9d28-7c3b911773a6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ee75f580391a43d480184849d6ab35f3)` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:50:44.059924 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de3103d1-65f4-4630-9d28-7c3b911773a6 admin admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:50:44.097565 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da99f99-3e69-4a5b-931b-ac95a3ba244d admin admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:50:44.108281 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b667c942-66dc-42a7-a96f-cd41dab2a30b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:50:44.109429 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b667c942-66dc-42a7-a96f-cd41dab2a30b admin admin] REQUEST_METHOD: `DELETE` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:50:44.109921 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b667c942-66dc-42a7-a96f-cd41dab2a30b admin admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:50:44.112244 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b667c942-66dc-42a7-a96f-cd41dab2a30b admin admin] PATH_INFO: `/v3/users/1a2cb7562b3d4332a5ac181e49236fc3` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:50:44.117719 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b667c942-66dc-42a7-a96f-cd41dab2a30b admin admin] RBAC: Authorizing `identity:delete_user(user_id=1a2cb7562b3d4332a5ac181e49236fc3)` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:50:44.119951 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b667c942-66dc-42a7-a96f-cd41dab2a30b admin admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:50:44.174447 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.notifications [None req-de3103d1-65f4-4630-9d28-7c3b911773a6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ee75f580391a43d480184849d6ab35f3', 'request_id': 'req-de3103d1-65f4-4630-9d28-7c3b911773a6'} {{(pid=88113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:50:44.186888 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.notifications [None req-de3103d1-65f4-4630-9d28-7c3b911773a6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ee75f580391a43d480184849d6ab35f3', 'request_id': 'req-de3103d1-65f4-4630-9d28-7c3b911773a6'} {{(pid=88113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:50:44.208689 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.notifications [None req-de3103d1-65f4-4630-9d28-7c3b911773a6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ee75f580391a43d480184849d6ab35f3', 'request_id': 'req-de3103d1-65f4-4630-9d28-7c3b911773a6'} {{(pid=88113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:50:44.255910 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.notifications [None req-b667c942-66dc-42a7-a96f-cd41dab2a30b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1a2cb7562b3d4332a5ac181e49236fc3'} {{(pid=88114) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:50:44.258387 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.notifications [None req-de3103d1-65f4-4630-9d28-7c3b911773a6 admin admin] The token cache is being invalidate because project ee75f580391a43d480184849d6ab35f3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88113) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 09:50:44.259168 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.notifications [None req-de3103d1-65f4-4630-9d28-7c3b911773a6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:50:44.260515 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.notifications [None req-de3103d1-65f4-4630-9d28-7c3b911773a6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:50:44.263795 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 1150/2299] 10.222.0.13 () {58 vars in 1233 bytes} [Wed Aug 31 09:50:43 2022] DELETE /identity/v3/projects/ee75f580391a43d480184849d6ab35f3 => generated 0 bytes in 418 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:50:44.271132 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de3103d1-65f4-4630-9d28-7c3b911773a6 admin admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:50:44.273629 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.fernet_utils [None req-de3103d1-65f4-4630-9d28-7c3b911773a6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:50:44.276053 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.notifications [None req-b667c942-66dc-42a7-a96f-cd41dab2a30b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1a2cb7562b3d4332a5ac181e49236fc3'} {{(pid=88114) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:50:44.290997 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.notifications [None req-b667c942-66dc-42a7-a96f-cd41dab2a30b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1a2cb7562b3d4332a5ac181e49236fc3'} {{(pid=88114) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:50:44.292027 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.notifications [None req-b667c942-66dc-42a7-a96f-cd41dab2a30b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1a2cb7562b3d4332a5ac181e49236fc3'} {{(pid=88114) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:50:44.320400 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.notifications [None req-b667c942-66dc-42a7-a96f-cd41dab2a30b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1a2cb7562b3d4332a5ac181e49236fc3'} {{(pid=88114) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:50:44.335852 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 1150/2300] 10.222.0.13 () {58 vars in 1224 bytes} [Wed Aug 31 09:50:43 2022] DELETE /identity/v3/users/1a2cb7562b3d4332a5ac181e49236fc3 => generated 0 bytes in 447 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:50:44.345958 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3048b564-797e-4a4a-a0b5-21d3ff5e9785 None None] REQUEST_METHOD: `POST` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:50:44.346564 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3048b564-797e-4a4a-a0b5-21d3ff5e9785 None None] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:50:44.347025 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3048b564-797e-4a4a-a0b5-21d3ff5e9785 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:50:44.385822 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.auth.core [None req-3048b564-797e-4a4a-a0b5-21d3ff5e9785 None None] MFA Rules not processed for user `fd836cf590a145df9acf19cf3ae2ac56`. Rule list: `[]` (Enabled: `True`). {{(pid=88114) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:50:44.391224 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.fernet_utils [None req-3048b564-797e-4a4a-a0b5-21d3ff5e9785 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88114) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:50:44.455134 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de3103d1-65f4-4630-9d28-7c3b911773a6 admin admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:50:44.646680 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 1151/2301] 10.222.0.13 () {58 vars in 958 bytes} [Wed Aug 31 09:50:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 304 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:50:44.650366 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:50:44.672995 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdba87e3-e45f-424e-a4f6-fadb45a82e3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:50:44.676780 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdba87e3-e45f-424e-a4f6-fadb45a82e3d admin admin] REQUEST_METHOD: `DELETE` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:50:44.676780 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdba87e3-e45f-424e-a4f6-fadb45a82e3d admin admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:50:44.677319 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdba87e3-e45f-424e-a4f6-fadb45a82e3d admin admin] PATH_INFO: `/v3/users/f80213dfdd0a4427b47313ab6adbe3b0` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:50:44.680122 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdba87e3-e45f-424e-a4f6-fadb45a82e3d admin admin] RBAC: Authorizing `identity:delete_user(user_id=f80213dfdd0a4427b47313ab6adbe3b0)` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:50:44.680732 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdba87e3-e45f-424e-a4f6-fadb45a82e3d admin admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:50:44.699236 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3048b564-797e-4a4a-a0b5-21d3ff5e9785 None None] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:50:44.702139 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.fernet_utils [None req-3048b564-797e-4a4a-a0b5-21d3ff5e9785 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88114) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:50:44.825848 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3048b564-797e-4a4a-a0b5-21d3ff5e9785 None None] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:50:44.831436 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b5fddd-3738-4534-81e2-a1e9de2b19ec service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9bdaacd597442c2bcde2c555f2cc140', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '49548eb44d62415582af820ab66decee', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:50:44.831436 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b5fddd-3738-4534-81e2-a1e9de2b19ec service neutron] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:50:44.833547 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b5fddd-3738-4534-81e2-a1e9de2b19ec service neutron] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:50:44.833547 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b5fddd-3738-4534-81e2-a1e9de2b19ec service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:50:44.874457 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.notifications [None req-bdba87e3-e45f-424e-a4f6-fadb45a82e3d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f80213dfdd0a4427b47313ab6adbe3b0'} {{(pid=88113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:50:44.878762 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55b5fddd-3738-4534-81e2-a1e9de2b19ec service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:50:44.881961 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55b5fddd-3738-4534-81e2-a1e9de2b19ec service neutron] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:50:44.903362 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 1152/2302] 10.222.0.13 () {60 vars in 1365 bytes} [Wed Aug 31 09:50:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 206 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:50:44.904829 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.notifications [None req-bdba87e3-e45f-424e-a4f6-fadb45a82e3d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f80213dfdd0a4427b47313ab6adbe3b0'} {{(pid=88113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:50:44.930865 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.notifications [None req-bdba87e3-e45f-424e-a4f6-fadb45a82e3d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f80213dfdd0a4427b47313ab6adbe3b0'} {{(pid=88113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:50:44.931985 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.notifications [None req-bdba87e3-e45f-424e-a4f6-fadb45a82e3d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f80213dfdd0a4427b47313ab6adbe3b0'} {{(pid=88113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:50:44.954851 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.notifications [None req-bdba87e3-e45f-424e-a4f6-fadb45a82e3d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f80213dfdd0a4427b47313ab6adbe3b0'} {{(pid=88113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:50:44.987456 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 1151/2303] 10.222.0.13 () {58 vars in 1224 bytes} [Wed Aug 31 09:50:44 2022] DELETE /identity/v3/users/f80213dfdd0a4427b47313ab6adbe3b0 => generated 0 bytes in 716 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:50:45.706179 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b5fddd-3738-4534-81e2-a1e9de2b19ec service neutron] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:50:45.708872 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.fernet_utils [None req-55b5fddd-3738-4534-81e2-a1e9de2b19ec service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88114) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:50:45.729333 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b5fddd-3738-4534-81e2-a1e9de2b19ec service neutron] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:50:45.737247 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:50:45.906006 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdba87e3-e45f-424e-a4f6-fadb45a82e3d admin admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:50:45.918398 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.fernet_utils [None req-bdba87e3-e45f-424e-a4f6-fadb45a82e3d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:50:45.928858 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d73b0a9e-2e5d-4a83-a3e2-4645752063aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:50:45.929695 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73b0a9e-2e5d-4a83-a3e2-4645752063aa admin admin] REQUEST_METHOD: `DELETE` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:50:45.930107 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73b0a9e-2e5d-4a83-a3e2-4645752063aa admin admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:50:45.930387 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73b0a9e-2e5d-4a83-a3e2-4645752063aa admin admin] PATH_INFO: `/v3/projects/cc64571db46543b7a20198f3e3e73a95` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:50:45.931621 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d73b0a9e-2e5d-4a83-a3e2-4645752063aa admin admin] RBAC: Authorizing `identity:delete_project(project_id=cc64571db46543b7a20198f3e3e73a95)` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:50:45.933986 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d73b0a9e-2e5d-4a83-a3e2-4645752063aa admin admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:50:45.969244 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdba87e3-e45f-424e-a4f6-fadb45a82e3d admin admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:50:45.981598 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f655aa1a-4103-4b60-a6b0-863a8848afa6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:50:45.982446 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f655aa1a-4103-4b60-a6b0-863a8848afa6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:50:45.982831 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f655aa1a-4103-4b60-a6b0-863a8848afa6 admin admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:50:45.983352 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f655aa1a-4103-4b60-a6b0-863a8848afa6 admin admin] PATH_INFO: `/v3/projects/722c32420cf54ef2b9357d78ecbe3460` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:50:45.985713 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f655aa1a-4103-4b60-a6b0-863a8848afa6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=722c32420cf54ef2b9357d78ecbe3460)` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:50:45.988090 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f655aa1a-4103-4b60-a6b0-863a8848afa6 admin admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:50:46.003229 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.notifications [None req-d73b0a9e-2e5d-4a83-a3e2-4645752063aa admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cc64571db46543b7a20198f3e3e73a95', 'request_id': 'req-d73b0a9e-2e5d-4a83-a3e2-4645752063aa'} {{(pid=88114) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:50:46.011942 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.notifications [None req-d73b0a9e-2e5d-4a83-a3e2-4645752063aa admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cc64571db46543b7a20198f3e3e73a95', 'request_id': 'req-d73b0a9e-2e5d-4a83-a3e2-4645752063aa'} {{(pid=88114) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:50:46.021440 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.notifications [None req-d73b0a9e-2e5d-4a83-a3e2-4645752063aa admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cc64571db46543b7a20198f3e3e73a95', 'request_id': 'req-d73b0a9e-2e5d-4a83-a3e2-4645752063aa'} {{(pid=88114) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:50:46.043573 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.notifications [None req-d73b0a9e-2e5d-4a83-a3e2-4645752063aa admin admin] The token cache is being invalidate because project cc64571db46543b7a20198f3e3e73a95 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88114) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 09:50:46.043876 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.notifications [None req-d73b0a9e-2e5d-4a83-a3e2-4645752063aa admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88114) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:50:46.045467 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.notifications [None req-d73b0a9e-2e5d-4a83-a3e2-4645752063aa admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88114) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:50:46.062798 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 1153/2304] 10.222.0.13 () {58 vars in 1233 bytes} [Wed Aug 31 09:50:45 2022] DELETE /identity/v3/projects/cc64571db46543b7a20198f3e3e73a95 => generated 0 bytes in 345 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:50:46.077422 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.notifications [None req-f655aa1a-4103-4b60-a6b0-863a8848afa6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '722c32420cf54ef2b9357d78ecbe3460', 'request_id': 'req-f655aa1a-4103-4b60-a6b0-863a8848afa6'} {{(pid=88113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:50:46.085696 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.notifications [None req-f655aa1a-4103-4b60-a6b0-863a8848afa6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '722c32420cf54ef2b9357d78ecbe3460', 'request_id': 'req-f655aa1a-4103-4b60-a6b0-863a8848afa6'} {{(pid=88113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:50:46.097667 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.notifications [None req-f655aa1a-4103-4b60-a6b0-863a8848afa6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '722c32420cf54ef2b9357d78ecbe3460', 'request_id': 'req-f655aa1a-4103-4b60-a6b0-863a8848afa6'} {{(pid=88113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:50:46.146825 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.notifications [None req-f655aa1a-4103-4b60-a6b0-863a8848afa6 admin admin] The token cache is being invalidate because project 722c32420cf54ef2b9357d78ecbe3460 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88113) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 09:50:46.147454 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.notifications [None req-f655aa1a-4103-4b60-a6b0-863a8848afa6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:50:46.149573 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.notifications [None req-f655aa1a-4103-4b60-a6b0-863a8848afa6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:50:46.158656 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 1152/2305] 10.222.0.13 () {58 vars in 1233 bytes} [Wed Aug 31 09:50:45 2022] DELETE /identity/v3/projects/722c32420cf54ef2b9357d78ecbe3460 => generated 0 bytes in 254 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:50:48.003406 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d73b0a9e-2e5d-4a83-a3e2-4645752063aa admin admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:50:48.005781 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.fernet_utils [None req-d73b0a9e-2e5d-4a83-a3e2-4645752063aa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88114) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:50:48.120157 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d73b0a9e-2e5d-4a83-a3e2-4645752063aa admin admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:50:48.182340 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:50:48.199040 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f59fbec-0341-4eb9-850d-258065f0b641 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:50:48.202677 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f59fbec-0341-4eb9-850d-258065f0b641 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:50:48.202947 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f59fbec-0341-4eb9-850d-258065f0b641 admin admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:50:48.203350 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f59fbec-0341-4eb9-850d-258065f0b641 admin admin] PATH_INFO: `/v3/users/053705cd245541bd8be740b7e3d00d3f` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:50:48.204944 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f59fbec-0341-4eb9-850d-258065f0b641 admin admin] RBAC: Authorizing `identity:delete_user(user_id=053705cd245541bd8be740b7e3d00d3f)` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:50:48.208730 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f59fbec-0341-4eb9-850d-258065f0b641 admin admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:50:48.327074 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.notifications [None req-9f59fbec-0341-4eb9-850d-258065f0b641 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '053705cd245541bd8be740b7e3d00d3f'} {{(pid=88114) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:50:48.339848 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.notifications [None req-9f59fbec-0341-4eb9-850d-258065f0b641 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '053705cd245541bd8be740b7e3d00d3f'} {{(pid=88114) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:50:48.356156 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.notifications [None req-9f59fbec-0341-4eb9-850d-258065f0b641 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '053705cd245541bd8be740b7e3d00d3f'} {{(pid=88114) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:50:48.357944 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.notifications [None req-9f59fbec-0341-4eb9-850d-258065f0b641 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '053705cd245541bd8be740b7e3d00d3f'} {{(pid=88114) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:50:48.386327 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.notifications [None req-9f59fbec-0341-4eb9-850d-258065f0b641 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '053705cd245541bd8be740b7e3d00d3f'} {{(pid=88114) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:50:48.404740 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 1154/2306] 10.222.0.13 () {58 vars in 1224 bytes} [Wed Aug 31 09:50:48 2022] DELETE /identity/v3/users/053705cd245541bd8be740b7e3d00d3f => generated 0 bytes in 398 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:50:48.407968 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010300db-c51c-451e-8a8d-59de87f188f7 None None] REQUEST_METHOD: `POST` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:50:48.408554 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010300db-c51c-451e-8a8d-59de87f188f7 None None] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:50:48.408846 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010300db-c51c-451e-8a8d-59de87f188f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:50:48.433487 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.auth.core [None req-010300db-c51c-451e-8a8d-59de87f188f7 None None] MFA Rules not processed for user `fd836cf590a145df9acf19cf3ae2ac56`. Rule list: `[]` (Enabled: `True`). {{(pid=88113) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:50:48.435539 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.fernet_utils [None req-010300db-c51c-451e-8a8d-59de87f188f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:50:48.516403 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:50:48.541714 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 1153/2307] 10.222.0.13 () {58 vars in 958 bytes} [Wed Aug 31 09:50:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:50:48.550401 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f59fbec-0341-4eb9-850d-258065f0b641 admin admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:50:48.553092 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.fernet_utils [None req-9f59fbec-0341-4eb9-850d-258065f0b641 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88114) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:50:48.672747 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f59fbec-0341-4eb9-850d-258065f0b641 admin admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:50:48.677696 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0bac3e8-de56-459c-8070-ad1a0b1b77ee service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9bdaacd597442c2bcde2c555f2cc140', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '49548eb44d62415582af820ab66decee', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:50:48.678157 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0bac3e8-de56-459c-8070-ad1a0b1b77ee service neutron] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:50:48.678503 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0bac3e8-de56-459c-8070-ad1a0b1b77ee service neutron] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:50:48.678845 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0bac3e8-de56-459c-8070-ad1a0b1b77ee service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:50:48.691682 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0bac3e8-de56-459c-8070-ad1a0b1b77ee service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:50:48.695853 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0bac3e8-de56-459c-8070-ad1a0b1b77ee service neutron] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:50:48.706553 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 1155/2308] 10.222.0.13 () {60 vars in 1365 bytes} [Wed Aug 31 09:50:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 157 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:50:49.244417 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-010300db-c51c-451e-8a8d-59de87f188f7 None None] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:50:49.246169 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.fernet_utils [None req-010300db-c51c-451e-8a8d-59de87f188f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:50:49.267257 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-010300db-c51c-451e-8a8d-59de87f188f7 None None] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:50:49.275598 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-992aaec0-de5b-47e7-b833-64c2a4fa2f95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:50:49.276371 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992aaec0-de5b-47e7-b833-64c2a4fa2f95 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:50:49.276736 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992aaec0-de5b-47e7-b833-64c2a4fa2f95 admin admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:50:49.277265 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992aaec0-de5b-47e7-b833-64c2a4fa2f95 admin admin] PATH_INFO: `/v3/projects/77311e37f4df4dbb8b44aecd8ec83df1` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:50:49.279184 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-992aaec0-de5b-47e7-b833-64c2a4fa2f95 admin admin] RBAC: Authorizing `identity:delete_project(project_id=77311e37f4df4dbb8b44aecd8ec83df1)` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:50:49.281527 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-992aaec0-de5b-47e7-b833-64c2a4fa2f95 admin admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:50:49.371133 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.notifications [None req-992aaec0-de5b-47e7-b833-64c2a4fa2f95 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '77311e37f4df4dbb8b44aecd8ec83df1', 'request_id': 'req-992aaec0-de5b-47e7-b833-64c2a4fa2f95'} {{(pid=88113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:50:49.379449 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.notifications [None req-992aaec0-de5b-47e7-b833-64c2a4fa2f95 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '77311e37f4df4dbb8b44aecd8ec83df1', 'request_id': 'req-992aaec0-de5b-47e7-b833-64c2a4fa2f95'} {{(pid=88113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:50:49.408883 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.notifications [None req-992aaec0-de5b-47e7-b833-64c2a4fa2f95 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '77311e37f4df4dbb8b44aecd8ec83df1', 'request_id': 'req-992aaec0-de5b-47e7-b833-64c2a4fa2f95'} {{(pid=88113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:50:49.419122 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.notifications [None req-992aaec0-de5b-47e7-b833-64c2a4fa2f95 admin admin] The token cache is being invalidate because project 77311e37f4df4dbb8b44aecd8ec83df1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88113) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 09:50:49.419608 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.notifications [None req-992aaec0-de5b-47e7-b833-64c2a4fa2f95 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:50:49.429467 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.notifications [None req-992aaec0-de5b-47e7-b833-64c2a4fa2f95 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:50:49.432796 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 1154/2309] 10.222.0.13 () {58 vars in 1233 bytes} [Wed Aug 31 09:50:49 2022] DELETE /identity/v3/projects/77311e37f4df4dbb8b44aecd8ec83df1 => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:50:49.440032 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82942320-d6c4-498b-b5f6-1dd190f3b492 None None] REQUEST_METHOD: `POST` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:50:49.441441 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82942320-d6c4-498b-b5f6-1dd190f3b492 None None] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:50:49.441850 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82942320-d6c4-498b-b5f6-1dd190f3b492 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:50:49.466521 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.auth.core [None req-82942320-d6c4-498b-b5f6-1dd190f3b492 None None] MFA Rules not processed for user `fd836cf590a145df9acf19cf3ae2ac56`. Rule list: `[]` (Enabled: `True`). {{(pid=88114) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:50:49.468011 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.fernet_utils [None req-82942320-d6c4-498b-b5f6-1dd190f3b492 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88114) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:50:49.616695 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 1156/2310] 10.222.0.13 () {58 vars in 958 bytes} [Wed Aug 31 09:50:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:50:49.625059 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6976313-094b-4593-bfe2-7334bd057a9f None None] REQUEST_METHOD: `POST` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:50:49.625333 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6976313-094b-4593-bfe2-7334bd057a9f None None] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:50:49.625621 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6976313-094b-4593-bfe2-7334bd057a9f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:50:49.650339 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.auth.core [None req-d6976313-094b-4593-bfe2-7334bd057a9f None None] MFA Rules not processed for user `fd836cf590a145df9acf19cf3ae2ac56`. Rule list: `[]` (Enabled: `True`). {{(pid=88113) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:50:49.651638 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.fernet_utils [None req-d6976313-094b-4593-bfe2-7334bd057a9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:50:49.664364 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 1155/2311] 10.222.0.13 () {58 vars in 958 bytes} [Wed Aug 31 09:50:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:50:49.669777 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82942320-d6c4-498b-b5f6-1dd190f3b492 None None] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:50:49.686978 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82942320-d6c4-498b-b5f6-1dd190f3b492 None None] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:50:49.694169 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e218b217-b971-4b92-8037-fb7ada030a72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:50:49.694900 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e218b217-b971-4b92-8037-fb7ada030a72 admin admin] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:50:49.695283 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e218b217-b971-4b92-8037-fb7ada030a72 admin admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:50:49.695607 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e218b217-b971-4b92-8037-fb7ada030a72 admin admin] PATH_INFO: `/v3/domains` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:50:49.696313 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e218b217-b971-4b92-8037-fb7ada030a72 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88114) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:50:49.696715 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e218b217-b971-4b92-8037-fb7ada030a72 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:50:49.698492 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e218b217-b971-4b92-8037-fb7ada030a72 admin admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:50:49.709847 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:50:49.740582 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 1157/2312] 10.222.0.13 () {58 vars in 1153 bytes} [Wed Aug 31 09:50:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 72 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:50:49.742466 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6976313-094b-4593-bfe2-7334bd057a9f None None] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:50:49.762901 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6976313-094b-4593-bfe2-7334bd057a9f None None] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:50:49.773663 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b335710-860f-49f4-bd0e-ba40abf10caa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:50:49.774768 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b335710-860f-49f4-bd0e-ba40abf10caa admin admin] REQUEST_METHOD: `POST` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:50:49.775282 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b335710-860f-49f4-bd0e-ba40abf10caa admin admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:50:49.775784 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b335710-860f-49f4-bd0e-ba40abf10caa admin admin] PATH_INFO: `/v3/projects` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:50:49.776974 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b335710-860f-49f4-bd0e-ba40abf10caa admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:50:49.781659 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b335710-860f-49f4-bd0e-ba40abf10caa admin admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:50:49.841290 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 1156/2313] 10.222.0.13 () {60 vars in 1153 bytes} [Wed Aug 31 09:50:49 2022] POST /identity/v3/projects => generated 375 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 09:50:49.843813 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e218b217-b971-4b92-8037-fb7ada030a72 admin admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:50:49.865566 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e218b217-b971-4b92-8037-fb7ada030a72 admin admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:50:49.940015 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae2d8c3f-9455-403c-a547-eb41fc329d57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:50:49.940619 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2d8c3f-9455-403c-a547-eb41fc329d57 admin admin] REQUEST_METHOD: `POST` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:50:49.940892 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2d8c3f-9455-403c-a547-eb41fc329d57 admin admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:50:49.941158 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2d8c3f-9455-403c-a547-eb41fc329d57 admin admin] PATH_INFO: `/v3/users` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:50:49.941822 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae2d8c3f-9455-403c-a547-eb41fc329d57 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:50:49.944073 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae2d8c3f-9455-403c-a547-eb41fc329d57 admin admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:50:49.974312 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 1158/2314] 10.222.0.13 () {60 vars in 1144 bytes} [Wed Aug 31 09:50:49 2022] POST /identity/v3/users => generated 288 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 09:50:49.978718 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b335710-860f-49f4-bd0e-ba40abf10caa admin admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:50:49.989872 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b335710-860f-49f4-bd0e-ba40abf10caa admin admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:50:49.995146 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d559b9bd-2c16-4723-90f5-d186790089a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:50:49.995751 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d559b9bd-2c16-4723-90f5-d186790089a2 admin admin] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:50:49.996028 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d559b9bd-2c16-4723-90f5-d186790089a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:50:49.996292 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d559b9bd-2c16-4723-90f5-d186790089a2 admin admin] PATH_INFO: `/v3/roles` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:50:49.996934 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d559b9bd-2c16-4723-90f5-d186790089a2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:50:49.998568 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d559b9bd-2c16-4723-90f5-d186790089a2 admin admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:50:50.022444 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 1157/2315] 10.222.0.13 () {58 vars in 1122 bytes} [Wed Aug 31 09:50:49 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 09:50:50.023727 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae2d8c3f-9455-403c-a547-eb41fc329d57 admin admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:50:50.039990 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae2d8c3f-9455-403c-a547-eb41fc329d57 admin admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:50:50.050369 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a50015-b9b0-4409-963c-749a9ab626f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:50:50.051804 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a50015-b9b0-4409-963c-749a9ab626f9 admin admin] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:50:50.052509 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a50015-b9b0-4409-963c-749a9ab626f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:50:50.053604 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a50015-b9b0-4409-963c-749a9ab626f9 admin admin] PATH_INFO: `/v3/roles` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:50:50.054820 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28a50015-b9b0-4409-963c-749a9ab626f9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:50:50.057625 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28a50015-b9b0-4409-963c-749a9ab626f9 admin admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:50:50.071664 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 1159/2316] 10.222.0.13 () {58 vars in 1122 bytes} [Wed Aug 31 09:50:50 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 09:50:50.078330 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d559b9bd-2c16-4723-90f5-d186790089a2 admin admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:50:50.088926 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d559b9bd-2c16-4723-90f5-d186790089a2 admin admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:50:50.093926 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e395a507-76f4-4a05-bd4c-aee3232f3077 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:50:50.095232 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e395a507-76f4-4a05-bd4c-aee3232f3077 admin admin] REQUEST_METHOD: `PUT` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:50:50.095709 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e395a507-76f4-4a05-bd4c-aee3232f3077 admin admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:50:50.095988 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e395a507-76f4-4a05-bd4c-aee3232f3077 admin admin] PATH_INFO: `/v3/projects/32966fbcfc754f8d87f390972b0756b5/users/9cd8955a84b04ca49eb79ad9f1e375eb/roles/6c99f5d9f1f243eaa134d8540f21095d` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:50:50.112790 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e395a507-76f4-4a05-bd4c-aee3232f3077 admin admin] RBAC: Authorizing `identity:create_grant(project_id=32966fbcfc754f8d87f390972b0756b5, user_id=9cd8955a84b04ca49eb79ad9f1e375eb, role_id=6c99f5d9f1f243eaa134d8540f21095d)` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:50:50.115576 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e395a507-76f4-4a05-bd4c-aee3232f3077 admin admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:50:50.130185 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 1158/2317] 10.222.0.13 () {60 vars in 1483 bytes} [Wed Aug 31 09:50:50 2022] PUT /identity/v3/projects/32966fbcfc754f8d87f390972b0756b5/users/9cd8955a84b04ca49eb79ad9f1e375eb/roles/6c99f5d9f1f243eaa134d8540f21095d => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:50:50.140873 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7358d1e0-5a60-4b94-bac5-6909ea7b1daf None None] REQUEST_METHOD: `POST` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:50:50.141229 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7358d1e0-5a60-4b94-bac5-6909ea7b1daf None None] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:50:50.141536 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7358d1e0-5a60-4b94-bac5-6909ea7b1daf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:50:50.174917 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.auth.core [None req-7358d1e0-5a60-4b94-bac5-6909ea7b1daf None None] MFA Rules not processed for user `9cd8955a84b04ca49eb79ad9f1e375eb`. Rule list: `[]` (Enabled: `True`). {{(pid=88114) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:50:50.178853 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.fernet_utils [None req-7358d1e0-5a60-4b94-bac5-6909ea7b1daf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88114) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:50:50.259122 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 1160/2318] 10.222.0.13 () {58 vars in 958 bytes} [Wed Aug 31 09:50:50 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:50:50.270642 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e395a507-76f4-4a05-bd4c-aee3232f3077 admin admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:50:50.272827 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.fernet_utils [None req-e395a507-76f4-4a05-bd4c-aee3232f3077 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:50:50.348050 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e395a507-76f4-4a05-bd4c-aee3232f3077 admin admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:50:50.357506 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b2902c7-0055-4011-8d2a-db812818ddf2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d52b69c8e94441dacfcff944326f169', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '49548eb44d62415582af820ab66decee', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:50:50.358419 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2902c7-0055-4011-8d2a-db812818ddf2 service cinder] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:50:50.358722 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2902c7-0055-4011-8d2a-db812818ddf2 service cinder] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:50:50.358994 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2902c7-0055-4011-8d2a-db812818ddf2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:50:50.371154 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b2902c7-0055-4011-8d2a-db812818ddf2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:50:50.372628 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b2902c7-0055-4011-8d2a-db812818ddf2 service cinder] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:50:50.383645 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 1159/2319] 10.222.0.13 () {60 vars in 1365 bytes} [Wed Aug 31 09:50:50 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:50:52.218035 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7358d1e0-5a60-4b94-bac5-6909ea7b1daf None None] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:50:52.219825 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.fernet_utils [None req-7358d1e0-5a60-4b94-bac5-6909ea7b1daf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88114) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:50:52.362070 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7358d1e0-5a60-4b94-bac5-6909ea7b1daf None None] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:50:52.371901 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91fd95f4-d0f8-4bf3-9b29-3d91ab003b2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:50:52.373338 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91fd95f4-d0f8-4bf3-9b29-3d91ab003b2f admin admin] REQUEST_METHOD: `DELETE` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:50:52.374544 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91fd95f4-d0f8-4bf3-9b29-3d91ab003b2f admin admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:50:52.375325 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91fd95f4-d0f8-4bf3-9b29-3d91ab003b2f admin admin] PATH_INFO: `/v3/users/3b84237e47a94a8fa95a175bacb02b83` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:50:52.378289 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91fd95f4-d0f8-4bf3-9b29-3d91ab003b2f admin admin] RBAC: Authorizing `identity:delete_user(user_id=3b84237e47a94a8fa95a175bacb02b83)` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:50:52.380387 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91fd95f4-d0f8-4bf3-9b29-3d91ab003b2f admin admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:50:52.388333 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:50:52.485426 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.notifications [None req-91fd95f4-d0f8-4bf3-9b29-3d91ab003b2f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3b84237e47a94a8fa95a175bacb02b83'} {{(pid=88114) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:50:52.496710 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.notifications [None req-91fd95f4-d0f8-4bf3-9b29-3d91ab003b2f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3b84237e47a94a8fa95a175bacb02b83'} {{(pid=88114) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:50:52.513114 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.notifications [None req-91fd95f4-d0f8-4bf3-9b29-3d91ab003b2f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3b84237e47a94a8fa95a175bacb02b83'} {{(pid=88114) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:50:52.513920 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.notifications [None req-91fd95f4-d0f8-4bf3-9b29-3d91ab003b2f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3b84237e47a94a8fa95a175bacb02b83'} {{(pid=88114) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:50:52.534664 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.notifications [None req-91fd95f4-d0f8-4bf3-9b29-3d91ab003b2f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3b84237e47a94a8fa95a175bacb02b83'} {{(pid=88114) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:50:52.552408 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 1161/2320] 10.222.0.13 () {58 vars in 1224 bytes} [Wed Aug 31 09:50:52 2022] DELETE /identity/v3/users/3b84237e47a94a8fa95a175bacb02b83 => generated 0 bytes in 336 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:50:52.558557 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff1434a-2c86-45e1-bd4c-57d1984de8a8 None None] REQUEST_METHOD: `POST` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:50:52.559881 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff1434a-2c86-45e1-bd4c-57d1984de8a8 None None] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:50:52.560338 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff1434a-2c86-45e1-bd4c-57d1984de8a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:50:52.596613 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.auth.core [None req-bff1434a-2c86-45e1-bd4c-57d1984de8a8 None None] MFA Rules not processed for user `fd836cf590a145df9acf19cf3ae2ac56`. Rule list: `[]` (Enabled: `True`). {{(pid=88113) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:50:52.598363 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.fernet_utils [None req-bff1434a-2c86-45e1-bd4c-57d1984de8a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:50:52.616629 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:50:52.716401 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 1160/2321] 10.222.0.13 () {58 vars in 958 bytes} [Wed Aug 31 09:50:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:50:52.733006 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91fd95f4-d0f8-4bf3-9b29-3d91ab003b2f admin admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:50:52.736758 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.fernet_utils [None req-91fd95f4-d0f8-4bf3-9b29-3d91ab003b2f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88114) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:50:52.845769 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91fd95f4-d0f8-4bf3-9b29-3d91ab003b2f admin admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:50:52.851138 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2435fcd-195e-4a5b-8d65-7238cb4f6a0d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9bdaacd597442c2bcde2c555f2cc140', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '49548eb44d62415582af820ab66decee', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:50:52.851901 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2435fcd-195e-4a5b-8d65-7238cb4f6a0d service neutron] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:50:52.852233 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2435fcd-195e-4a5b-8d65-7238cb4f6a0d service neutron] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:50:52.852695 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2435fcd-195e-4a5b-8d65-7238cb4f6a0d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:50:52.869116 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2435fcd-195e-4a5b-8d65-7238cb4f6a0d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:50:52.870746 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2435fcd-195e-4a5b-8d65-7238cb4f6a0d service neutron] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:50:52.923943 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 1162/2322] 10.222.0.13 () {60 vars in 1365 bytes} [Wed Aug 31 09:50:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 193 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:50:53.360141 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bff1434a-2c86-45e1-bd4c-57d1984de8a8 None None] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:50:53.362806 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.fernet_utils [None req-bff1434a-2c86-45e1-bd4c-57d1984de8a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:50:53.421420 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bff1434a-2c86-45e1-bd4c-57d1984de8a8 None None] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:50:53.445574 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd1fbed-4242-4d55-b25e-3fd3f80ed750 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:50:53.448349 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd1fbed-4242-4d55-b25e-3fd3f80ed750 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:50:53.448825 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd1fbed-4242-4d55-b25e-3fd3f80ed750 admin admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:50:53.449311 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd1fbed-4242-4d55-b25e-3fd3f80ed750 admin admin] PATH_INFO: `/v3/projects/5eaf9a46dc28462b8bcba413107270e2` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:50:53.452275 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cd1fbed-4242-4d55-b25e-3fd3f80ed750 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5eaf9a46dc28462b8bcba413107270e2)` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:50:53.464934 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cd1fbed-4242-4d55-b25e-3fd3f80ed750 admin admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:50:53.589311 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.notifications [None req-7cd1fbed-4242-4d55-b25e-3fd3f80ed750 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5eaf9a46dc28462b8bcba413107270e2', 'request_id': 'req-7cd1fbed-4242-4d55-b25e-3fd3f80ed750'} {{(pid=88113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:50:53.600456 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.notifications [None req-7cd1fbed-4242-4d55-b25e-3fd3f80ed750 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5eaf9a46dc28462b8bcba413107270e2', 'request_id': 'req-7cd1fbed-4242-4d55-b25e-3fd3f80ed750'} {{(pid=88113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:50:53.614255 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.notifications [None req-7cd1fbed-4242-4d55-b25e-3fd3f80ed750 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5eaf9a46dc28462b8bcba413107270e2', 'request_id': 'req-7cd1fbed-4242-4d55-b25e-3fd3f80ed750'} {{(pid=88113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:50:53.639167 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.notifications [None req-7cd1fbed-4242-4d55-b25e-3fd3f80ed750 admin admin] The token cache is being invalidate because project 5eaf9a46dc28462b8bcba413107270e2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88113) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 09:50:53.640024 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.notifications [None req-7cd1fbed-4242-4d55-b25e-3fd3f80ed750 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:50:53.641872 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.notifications [None req-7cd1fbed-4242-4d55-b25e-3fd3f80ed750 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:50:53.645129 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 1161/2323] 10.222.0.13 () {58 vars in 1233 bytes} [Wed Aug 31 09:50:53 2022] DELETE /identity/v3/projects/5eaf9a46dc28462b8bcba413107270e2 => generated 0 bytes in 286 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:50:53.652400 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442689cb-f0a7-4bad-8e9b-21313b52bb4b None None] REQUEST_METHOD: `POST` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:50:53.653226 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442689cb-f0a7-4bad-8e9b-21313b52bb4b None None] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:50:53.653226 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442689cb-f0a7-4bad-8e9b-21313b52bb4b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:50:53.679780 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.auth.core [None req-442689cb-f0a7-4bad-8e9b-21313b52bb4b None None] MFA Rules not processed for user `fd836cf590a145df9acf19cf3ae2ac56`. Rule list: `[]` (Enabled: `True`). {{(pid=88114) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:50:53.681260 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.fernet_utils [None req-442689cb-f0a7-4bad-8e9b-21313b52bb4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88114) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:50:53.772439 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 1163/2324] 10.222.0.13 () {58 vars in 958 bytes} [Wed Aug 31 09:50:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:50:53.781068 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450a84cc-c4b0-4ec0-aa1e-eaf85ae6b28c None None] REQUEST_METHOD: `POST` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:50:53.781517 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450a84cc-c4b0-4ec0-aa1e-eaf85ae6b28c None None] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:50:53.783185 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450a84cc-c4b0-4ec0-aa1e-eaf85ae6b28c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:50:53.821121 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.auth.core [None req-450a84cc-c4b0-4ec0-aa1e-eaf85ae6b28c None None] MFA Rules not processed for user `fd836cf590a145df9acf19cf3ae2ac56`. Rule list: `[]` (Enabled: `True`). {{(pid=88113) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:50:53.822533 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.fernet_utils [None req-450a84cc-c4b0-4ec0-aa1e-eaf85ae6b28c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:50:53.838267 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 1162/2325] 10.222.0.13 () {58 vars in 958 bytes} [Wed Aug 31 09:50:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 54 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:50:53.841611 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-442689cb-f0a7-4bad-8e9b-21313b52bb4b None None] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:50:53.859050 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-442689cb-f0a7-4bad-8e9b-21313b52bb4b None None] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:50:53.867000 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71aa21a9-e441-4a77-b145-a9fb10f3f68f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:50:53.868229 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71aa21a9-e441-4a77-b145-a9fb10f3f68f admin admin] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:50:53.868229 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71aa21a9-e441-4a77-b145-a9fb10f3f68f admin admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:50:53.868572 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71aa21a9-e441-4a77-b145-a9fb10f3f68f admin admin] PATH_INFO: `/v3/domains` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:50:53.869510 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71aa21a9-e441-4a77-b145-a9fb10f3f68f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88114) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:50:53.870044 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71aa21a9-e441-4a77-b145-a9fb10f3f68f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:50:53.875418 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71aa21a9-e441-4a77-b145-a9fb10f3f68f admin admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:50:53.905573 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 1164/2326] 10.222.0.13 () {58 vars in 1153 bytes} [Wed Aug 31 09:50:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 09:50:53.913785 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-450a84cc-c4b0-4ec0-aa1e-eaf85ae6b28c None None] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:50:53.946824 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-450a84cc-c4b0-4ec0-aa1e-eaf85ae6b28c None None] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:50:53.964348 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c07bff4c-5d5c-4823-adcc-d71228c9c27c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:50:53.965488 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07bff4c-5d5c-4823-adcc-d71228c9c27c admin admin] REQUEST_METHOD: `POST` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:50:53.966022 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07bff4c-5d5c-4823-adcc-d71228c9c27c admin admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:50:53.966531 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07bff4c-5d5c-4823-adcc-d71228c9c27c admin admin] PATH_INFO: `/v3/projects` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:50:53.967637 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c07bff4c-5d5c-4823-adcc-d71228c9c27c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:50:53.971991 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c07bff4c-5d5c-4823-adcc-d71228c9c27c admin admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:50:54.050594 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 1163/2327] 10.222.0.13 () {60 vars in 1153 bytes} [Wed Aug 31 09:50:53 2022] POST /identity/v3/projects => generated 383 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 09:50:54.056922 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71aa21a9-e441-4a77-b145-a9fb10f3f68f admin admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:50:54.069967 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71aa21a9-e441-4a77-b145-a9fb10f3f68f admin admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:50:54.085782 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:50:54.189434 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b487c47-8a16-433b-bb85-d5d01dd45f56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:50:54.189770 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b487c47-8a16-433b-bb85-d5d01dd45f56 admin admin] REQUEST_METHOD: `POST` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:50:54.190758 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b487c47-8a16-433b-bb85-d5d01dd45f56 admin admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:50:54.190758 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b487c47-8a16-433b-bb85-d5d01dd45f56 admin admin] PATH_INFO: `/v3/users` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:50:54.198184 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b487c47-8a16-433b-bb85-d5d01dd45f56 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:50:54.200243 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b487c47-8a16-433b-bb85-d5d01dd45f56 admin admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:50:54.243798 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 1165/2328] 10.222.0.13 () {60 vars in 1144 bytes} [Wed Aug 31 09:50:54 2022] POST /identity/v3/users => generated 292 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 09:50:54.248126 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c07bff4c-5d5c-4823-adcc-d71228c9c27c admin admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:50:54.267619 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c07bff4c-5d5c-4823-adcc-d71228c9c27c admin admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:50:54.275896 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7514301-b860-49a2-8149-d67e2900aa5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:50:54.276174 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7514301-b860-49a2-8149-d67e2900aa5b admin admin] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:50:54.276276 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7514301-b860-49a2-8149-d67e2900aa5b admin admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:50:54.276541 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7514301-b860-49a2-8149-d67e2900aa5b admin admin] PATH_INFO: `/v3/roles` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:50:54.277201 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7514301-b860-49a2-8149-d67e2900aa5b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:50:54.278884 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7514301-b860-49a2-8149-d67e2900aa5b admin admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:50:54.296661 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 1164/2329] 10.222.0.13 () {58 vars in 1122 bytes} [Wed Aug 31 09:50:54 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 09:50:54.298651 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b487c47-8a16-433b-bb85-d5d01dd45f56 admin admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:50:54.312660 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b487c47-8a16-433b-bb85-d5d01dd45f56 admin admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:50:54.320072 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a77362-1ba7-4597-a87e-86fa2afbd8a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:50:54.320870 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a77362-1ba7-4597-a87e-86fa2afbd8a4 admin admin] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:50:54.321281 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a77362-1ba7-4597-a87e-86fa2afbd8a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:50:54.321690 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a77362-1ba7-4597-a87e-86fa2afbd8a4 admin admin] PATH_INFO: `/v3/roles` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:50:54.322497 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77a77362-1ba7-4597-a87e-86fa2afbd8a4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:50:54.324887 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77a77362-1ba7-4597-a87e-86fa2afbd8a4 admin admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:50:54.341746 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 1166/2330] 10.222.0.13 () {58 vars in 1122 bytes} [Wed Aug 31 09:50:54 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 09:50:54.346404 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7514301-b860-49a2-8149-d67e2900aa5b admin admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:50:54.362203 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7514301-b860-49a2-8149-d67e2900aa5b admin admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:50:54.379978 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e68bf6d-c04f-4459-8049-e5dd89868d98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:50:54.380886 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e68bf6d-c04f-4459-8049-e5dd89868d98 admin admin] REQUEST_METHOD: `PUT` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:50:54.381273 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e68bf6d-c04f-4459-8049-e5dd89868d98 admin admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:50:54.381696 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e68bf6d-c04f-4459-8049-e5dd89868d98 admin admin] PATH_INFO: `/v3/projects/b7ce73b883774578b69c9267d2f80e41/users/e76879f5f5e649eab7ed7105537c0e97/roles/6c99f5d9f1f243eaa134d8540f21095d` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:50:54.421525 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e68bf6d-c04f-4459-8049-e5dd89868d98 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b7ce73b883774578b69c9267d2f80e41, user_id=e76879f5f5e649eab7ed7105537c0e97, role_id=6c99f5d9f1f243eaa134d8540f21095d)` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:50:54.425687 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e68bf6d-c04f-4459-8049-e5dd89868d98 admin admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:50:54.469496 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 1165/2331] 10.222.0.13 () {60 vars in 1483 bytes} [Wed Aug 31 09:50:54 2022] PUT /identity/v3/projects/b7ce73b883774578b69c9267d2f80e41/users/e76879f5f5e649eab7ed7105537c0e97/roles/6c99f5d9f1f243eaa134d8540f21095d => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:50:54.484317 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063b5ae9-12cb-4fd2-8cf8-d7c8c5d14e6f None None] REQUEST_METHOD: `POST` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:50:54.484588 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063b5ae9-12cb-4fd2-8cf8-d7c8c5d14e6f None None] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:50:54.485072 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063b5ae9-12cb-4fd2-8cf8-d7c8c5d14e6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:50:54.576883 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.auth.core [None req-063b5ae9-12cb-4fd2-8cf8-d7c8c5d14e6f None None] MFA Rules not processed for user `e76879f5f5e649eab7ed7105537c0e97`. Rule list: `[]` (Enabled: `True`). {{(pid=88114) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:50:54.579174 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.fernet_utils [None req-063b5ae9-12cb-4fd2-8cf8-d7c8c5d14e6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88114) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:50:54.704355 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 1167/2332] 10.222.0.13 () {58 vars in 958 bytes} [Wed Aug 31 09:50:54 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 221 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:50:54.728153 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e68bf6d-c04f-4459-8049-e5dd89868d98 admin admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:50:54.732221 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.fernet_utils [None req-8e68bf6d-c04f-4459-8049-e5dd89868d98 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:50:54.859760 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:50:54.899560 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e68bf6d-c04f-4459-8049-e5dd89868d98 admin admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:50:54.907581 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-886b2947-8ad5-4c61-b0ca-99b0b3bc2d76 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d52b69c8e94441dacfcff944326f169', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '49548eb44d62415582af820ab66decee', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:50:54.908861 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886b2947-8ad5-4c61-b0ca-99b0b3bc2d76 service cinder] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:50:54.909679 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886b2947-8ad5-4c61-b0ca-99b0b3bc2d76 service cinder] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:50:54.910457 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886b2947-8ad5-4c61-b0ca-99b0b3bc2d76 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:50:54.941850 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-886b2947-8ad5-4c61-b0ca-99b0b3bc2d76 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:50:54.943745 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-886b2947-8ad5-4c61-b0ca-99b0b3bc2d76 service cinder] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:50:54.963157 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 1166/2333] 10.222.0.13 () {60 vars in 1365 bytes} [Wed Aug 31 09:50:54 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 236 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 09:50:56.772750 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3090ae26-e8b7-4243-b6bd-133717c1dbe0 None None] REQUEST_METHOD: `POST` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:50:56.772750 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3090ae26-e8b7-4243-b6bd-133717c1dbe0 None None] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:50:56.772750 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3090ae26-e8b7-4243-b6bd-133717c1dbe0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:50:56.797836 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.auth.core [None req-3090ae26-e8b7-4243-b6bd-133717c1dbe0 None None] MFA Rules not processed for user `886e40cf192d46c2b2e7eb2a95ec62e1`. Rule list: `[]` (Enabled: `True`). {{(pid=88114) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:50:56.799471 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.fernet_utils [None req-3090ae26-e8b7-4243-b6bd-133717c1dbe0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88114) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:50:56.905583 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 1168/2334] 10.222.0.13 () {58 vars in 958 bytes} [Wed Aug 31 09:50:56 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 09:51:02.691174 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-886b2947-8ad5-4c61-b0ca-99b0b3bc2d76 service cinder] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:51:02.693306 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.fernet_utils [None req-886b2947-8ad5-4c61-b0ca-99b0b3bc2d76 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:51:02.786034 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-886b2947-8ad5-4c61-b0ca-99b0b3bc2d76 service cinder] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:51:02.791902 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f01b63c-fecc-43d9-9eab-515b803a923b service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ff3aff5ef444910be6dbb989d0e3194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '49548eb44d62415582af820ab66decee', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:51:02.792538 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f01b63c-fecc-43d9-9eab-515b803a923b service placement] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:51:02.792821 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f01b63c-fecc-43d9-9eab-515b803a923b service placement] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:51:02.793098 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f01b63c-fecc-43d9-9eab-515b803a923b service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:51:02.794994 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.fernet_utils [None req-3f01b63c-fecc-43d9-9eab-515b803a923b service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:51:02.808761 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f01b63c-fecc-43d9-9eab-515b803a923b service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:51:02.810792 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f01b63c-fecc-43d9-9eab-515b803a923b service placement] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:51:02.823499 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 1167/2335] 10.222.0.13 () {60 vars in 1365 bytes} [Wed Aug 31 09:51:02 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:51:04.575440 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca9780a-eb8d-41b9-93bd-bb0d33232578 None None] REQUEST_METHOD: `POST` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:51:04.576057 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca9780a-eb8d-41b9-93bd-bb0d33232578 None None] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:51:04.576057 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca9780a-eb8d-41b9-93bd-bb0d33232578 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:51:04.599453 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.auth.core [None req-8ca9780a-eb8d-41b9-93bd-bb0d33232578 None None] MFA Rules not processed for user `768cb67c1ef24251aaa113c12d81a250`. Rule list: `[]` (Enabled: `True`). {{(pid=88114) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:51:04.600715 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.fernet_utils [None req-8ca9780a-eb8d-41b9-93bd-bb0d33232578 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88114) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:51:04.630397 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:51:04.698599 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 1169/2336] 10.222.0.13 () {58 vars in 967 bytes} [Wed Aug 31 09:51:04 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 09:51:04.717596 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f01b63c-fecc-43d9-9eab-515b803a923b service placement] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:51:04.720294 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.fernet_utils [None req-3f01b63c-fecc-43d9-9eab-515b803a923b service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:51:04.856386 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f01b63c-fecc-43d9-9eab-515b803a923b service placement] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:51:04.863040 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b66c9299-0139-4927-9a98-25c473ed97a0 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '202bb9db7a8a4977ba788a290ed9a3fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '49548eb44d62415582af820ab66decee', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:51:04.863677 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66c9299-0139-4927-9a98-25c473ed97a0 service swift] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:51:04.863944 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66c9299-0139-4927-9a98-25c473ed97a0 service swift] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:51:04.864209 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66c9299-0139-4927-9a98-25c473ed97a0 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:51:04.874215 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b66c9299-0139-4927-9a98-25c473ed97a0 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:51:04.876567 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b66c9299-0139-4927-9a98-25c473ed97a0 service swift] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:51:04.894399 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 1168/2337] 10.222.0.13 () {60 vars in 1384 bytes} [Wed Aug 31 09:51:04 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 177 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 31 09:51:06.795197 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca9780a-eb8d-41b9-93bd-bb0d33232578 None None] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:51:06.796895 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.fernet_utils [None req-8ca9780a-eb8d-41b9-93bd-bb0d33232578 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88114) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:51:06.910603 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca9780a-eb8d-41b9-93bd-bb0d33232578 None None] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:51:06.916460 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be0f353c-49af-44fa-a69d-51e86615a6a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:51:06.916977 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0f353c-49af-44fa-a69d-51e86615a6a5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:51:06.917264 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0f353c-49af-44fa-a69d-51e86615a6a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:51:06.917626 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0f353c-49af-44fa-a69d-51e86615a6a5 admin admin] PATH_INFO: `/v3/users/9cd8955a84b04ca49eb79ad9f1e375eb` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:51:06.918844 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be0f353c-49af-44fa-a69d-51e86615a6a5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9cd8955a84b04ca49eb79ad9f1e375eb)` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:51:06.920881 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be0f353c-49af-44fa-a69d-51e86615a6a5 admin admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:51:07.026999 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.notifications [None req-be0f353c-49af-44fa-a69d-51e86615a6a5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9cd8955a84b04ca49eb79ad9f1e375eb'} {{(pid=88114) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:51:07.045418 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.notifications [None req-be0f353c-49af-44fa-a69d-51e86615a6a5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9cd8955a84b04ca49eb79ad9f1e375eb'} {{(pid=88114) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:51:07.059369 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.notifications [None req-be0f353c-49af-44fa-a69d-51e86615a6a5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9cd8955a84b04ca49eb79ad9f1e375eb'} {{(pid=88114) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:51:07.060492 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.notifications [None req-be0f353c-49af-44fa-a69d-51e86615a6a5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9cd8955a84b04ca49eb79ad9f1e375eb'} {{(pid=88114) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:51:07.090431 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.notifications [None req-be0f353c-49af-44fa-a69d-51e86615a6a5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9cd8955a84b04ca49eb79ad9f1e375eb'} {{(pid=88114) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:51:07.103842 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 1170/2338] 10.222.0.13 () {58 vars in 1224 bytes} [Wed Aug 31 09:51:06 2022] DELETE /identity/v3/users/9cd8955a84b04ca49eb79ad9f1e375eb => generated 0 bytes in 309 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:51:07.110269 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ee70de-35f0-430f-9641-db065d1ae3a4 None None] REQUEST_METHOD: `POST` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:51:07.111782 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ee70de-35f0-430f-9641-db065d1ae3a4 None None] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:51:07.112193 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ee70de-35f0-430f-9641-db065d1ae3a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:51:07.146292 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.auth.core [None req-96ee70de-35f0-430f-9641-db065d1ae3a4 None None] MFA Rules not processed for user `fd836cf590a145df9acf19cf3ae2ac56`. Rule list: `[]` (Enabled: `True`). {{(pid=88113) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:51:07.148507 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.fernet_utils [None req-96ee70de-35f0-430f-9641-db065d1ae3a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:51:07.233884 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=100 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:51:07.254516 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 1169/2339] 10.222.0.13 () {58 vars in 958 bytes} [Wed Aug 31 09:51:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 09:51:07.270375 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be0f353c-49af-44fa-a69d-51e86615a6a5 admin admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:51:07.271975 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.fernet_utils [None req-be0f353c-49af-44fa-a69d-51e86615a6a5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88114) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:51:07.353668 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be0f353c-49af-44fa-a69d-51e86615a6a5 admin admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:51:07.357867 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07971382-2635-429a-acf5-fbd3646e273f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9bdaacd597442c2bcde2c555f2cc140', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '49548eb44d62415582af820ab66decee', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:51:07.358703 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07971382-2635-429a-acf5-fbd3646e273f service neutron] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:51:07.359052 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07971382-2635-429a-acf5-fbd3646e273f service neutron] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:51:07.360161 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07971382-2635-429a-acf5-fbd3646e273f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:51:07.369025 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:51:07.378062 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07971382-2635-429a-acf5-fbd3646e273f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:51:07.382345 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07971382-2635-429a-acf5-fbd3646e273f service neutron] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:51:07.403203 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 1171/2340] 10.222.0.13 () {60 vars in 1365 bytes} [Wed Aug 31 09:51:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 09:51:07.738737 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ee70de-35f0-430f-9641-db065d1ae3a4 None None] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:51:07.742097 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.fernet_utils [None req-96ee70de-35f0-430f-9641-db065d1ae3a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:51:07.767621 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ee70de-35f0-430f-9641-db065d1ae3a4 None None] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:51:07.778924 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ca41b7-5931-4e97-a31a-47814ad9993f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:51:07.780389 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ca41b7-5931-4e97-a31a-47814ad9993f admin admin] REQUEST_METHOD: `DELETE` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:51:07.781183 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ca41b7-5931-4e97-a31a-47814ad9993f admin admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:51:07.782462 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ca41b7-5931-4e97-a31a-47814ad9993f admin admin] PATH_INFO: `/v3/users/e76879f5f5e649eab7ed7105537c0e97` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:51:07.785603 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8ca41b7-5931-4e97-a31a-47814ad9993f admin admin] RBAC: Authorizing `identity:delete_user(user_id=e76879f5f5e649eab7ed7105537c0e97)` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:51:07.789583 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8ca41b7-5931-4e97-a31a-47814ad9993f admin admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:51:07.852282 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07971382-2635-429a-acf5-fbd3646e273f service neutron] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:51:07.854982 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.fernet_utils [None req-07971382-2635-429a-acf5-fbd3646e273f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88114) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:51:07.902172 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07971382-2635-429a-acf5-fbd3646e273f service neutron] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:51:07.922629 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.notifications [None req-e8ca41b7-5931-4e97-a31a-47814ad9993f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e76879f5f5e649eab7ed7105537c0e97'} {{(pid=88113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:51:07.926193 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9246e5b-ba84-4245-a2b7-1955fb8209d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:51:07.927950 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9246e5b-ba84-4245-a2b7-1955fb8209d0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:51:07.928686 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9246e5b-ba84-4245-a2b7-1955fb8209d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:51:07.929461 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9246e5b-ba84-4245-a2b7-1955fb8209d0 admin admin] PATH_INFO: `/v3/projects/32966fbcfc754f8d87f390972b0756b5` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:51:07.932162 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9246e5b-ba84-4245-a2b7-1955fb8209d0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=32966fbcfc754f8d87f390972b0756b5)` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:51:07.936412 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9246e5b-ba84-4245-a2b7-1955fb8209d0 admin admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:51:07.936926 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.notifications [None req-e8ca41b7-5931-4e97-a31a-47814ad9993f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e76879f5f5e649eab7ed7105537c0e97'} {{(pid=88113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:51:07.955599 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.notifications [None req-e8ca41b7-5931-4e97-a31a-47814ad9993f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e76879f5f5e649eab7ed7105537c0e97'} {{(pid=88113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:51:07.956501 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.notifications [None req-e8ca41b7-5931-4e97-a31a-47814ad9993f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e76879f5f5e649eab7ed7105537c0e97'} {{(pid=88113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:51:07.976917 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.notifications [None req-e8ca41b7-5931-4e97-a31a-47814ad9993f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e76879f5f5e649eab7ed7105537c0e97'} {{(pid=88113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:51:07.986634 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 1170/2341] 10.222.0.13 () {58 vars in 1224 bytes} [Wed Aug 31 09:51:07 2022] DELETE /identity/v3/users/e76879f5f5e649eab7ed7105537c0e97 => generated 0 bytes in 249 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:51:07.987727 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ca41b7-5931-4e97-a31a-47814ad9993f admin admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:51:07.989240 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.fernet_utils [None req-e8ca41b7-5931-4e97-a31a-47814ad9993f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:51:08.038260 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.notifications [None req-a9246e5b-ba84-4245-a2b7-1955fb8209d0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '32966fbcfc754f8d87f390972b0756b5', 'request_id': 'req-a9246e5b-ba84-4245-a2b7-1955fb8209d0'} {{(pid=88114) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:51:08.045350 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.notifications [None req-a9246e5b-ba84-4245-a2b7-1955fb8209d0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '32966fbcfc754f8d87f390972b0756b5', 'request_id': 'req-a9246e5b-ba84-4245-a2b7-1955fb8209d0'} {{(pid=88114) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:51:08.053756 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.notifications [None req-a9246e5b-ba84-4245-a2b7-1955fb8209d0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '32966fbcfc754f8d87f390972b0756b5', 'request_id': 'req-a9246e5b-ba84-4245-a2b7-1955fb8209d0'} {{(pid=88114) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:51:08.073884 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.notifications [None req-a9246e5b-ba84-4245-a2b7-1955fb8209d0 admin admin] The token cache is being invalidate because project 32966fbcfc754f8d87f390972b0756b5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88114) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 09:51:08.074889 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.notifications [None req-a9246e5b-ba84-4245-a2b7-1955fb8209d0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88114) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:51:08.075951 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.notifications [None req-a9246e5b-ba84-4245-a2b7-1955fb8209d0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88114) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:51:08.078915 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 1172/2342] 10.222.0.13 () {58 vars in 1233 bytes} [Wed Aug 31 09:51:07 2022] DELETE /identity/v3/projects/32966fbcfc754f8d87f390972b0756b5 => generated 0 bytes in 229 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:51:08.080713 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910f25f9-5a1c-47d7-ba8e-ff406dd0e7fb None None] REQUEST_METHOD: `POST` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:51:08.081228 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910f25f9-5a1c-47d7-ba8e-ff406dd0e7fb None None] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:51:08.081771 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910f25f9-5a1c-47d7-ba8e-ff406dd0e7fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:51:08.095823 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ca41b7-5931-4e97-a31a-47814ad9993f admin admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:51:08.105172 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.auth.core [None req-910f25f9-5a1c-47d7-ba8e-ff406dd0e7fb None None] MFA Rules not processed for user `fd836cf590a145df9acf19cf3ae2ac56`. Rule list: `[]` (Enabled: `True`). {{(pid=88114) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:51:08.106466 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.fernet_utils [None req-910f25f9-5a1c-47d7-ba8e-ff406dd0e7fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88114) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:51:08.165982 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be0b1aba-36fb-4f6e-b1e8-f76f1ffb0236 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:51:08.166827 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0b1aba-36fb-4f6e-b1e8-f76f1ffb0236 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:51:08.166990 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0b1aba-36fb-4f6e-b1e8-f76f1ffb0236 admin admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:51:08.167182 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0b1aba-36fb-4f6e-b1e8-f76f1ffb0236 admin admin] PATH_INFO: `/v3/users/c475984de9e84ff18fb96c588a07c80b` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:51:08.168405 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be0b1aba-36fb-4f6e-b1e8-f76f1ffb0236 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c475984de9e84ff18fb96c588a07c80b)` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:51:08.170450 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be0b1aba-36fb-4f6e-b1e8-f76f1ffb0236 admin admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:51:08.198394 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 1173/2343] 10.222.0.13 () {58 vars in 958 bytes} [Wed Aug 31 09:51:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:51:08.199724 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea21e94-e166-4185-9c46-1780368dfe9e None None] REQUEST_METHOD: `POST` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:51:08.199993 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea21e94-e166-4185-9c46-1780368dfe9e None None] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:51:08.200264 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea21e94-e166-4185-9c46-1780368dfe9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:51:08.224071 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:51:08.228201 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.auth.core [None req-8ea21e94-e166-4185-9c46-1780368dfe9e None None] MFA Rules not processed for user `fd836cf590a145df9acf19cf3ae2ac56`. Rule list: `[]` (Enabled: `True`). {{(pid=88114) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:51:08.229517 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.fernet_utils [None req-8ea21e94-e166-4185-9c46-1780368dfe9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88114) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:51:08.249360 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 1174/2344] 10.222.0.13 () {58 vars in 958 bytes} [Wed Aug 31 09:51:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:51:08.251408 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ea21e94-e166-4185-9c46-1780368dfe9e None None] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:51:08.253109 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.fernet_utils [None req-8ea21e94-e166-4185-9c46-1780368dfe9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88114) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:51:08.284726 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.notifications [None req-be0b1aba-36fb-4f6e-b1e8-f76f1ffb0236 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c475984de9e84ff18fb96c588a07c80b'} {{(pid=88113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:51:08.294490 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.notifications [None req-be0b1aba-36fb-4f6e-b1e8-f76f1ffb0236 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c475984de9e84ff18fb96c588a07c80b'} {{(pid=88113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:51:08.310423 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.notifications [None req-be0b1aba-36fb-4f6e-b1e8-f76f1ffb0236 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c475984de9e84ff18fb96c588a07c80b'} {{(pid=88113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:51:08.311505 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.notifications [None req-be0b1aba-36fb-4f6e-b1e8-f76f1ffb0236 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c475984de9e84ff18fb96c588a07c80b'} {{(pid=88113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:51:08.322306 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.notifications [None req-be0b1aba-36fb-4f6e-b1e8-f76f1ffb0236 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c475984de9e84ff18fb96c588a07c80b'} {{(pid=88113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:51:08.330971 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 1171/2345] 10.222.0.13 () {58 vars in 1224 bytes} [Wed Aug 31 09:51:07 2022] DELETE /identity/v3/users/c475984de9e84ff18fb96c588a07c80b => generated 0 bytes in 344 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:51:08.332545 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b36dbe9-d42a-4006-802d-2f4dddccdbca None None] REQUEST_METHOD: `POST` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:51:08.333046 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b36dbe9-d42a-4006-802d-2f4dddccdbca None None] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:51:08.333316 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b36dbe9-d42a-4006-802d-2f4dddccdbca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:51:08.357135 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ea21e94-e166-4185-9c46-1780368dfe9e None None] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:51:08.367845 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.auth.core [None req-4b36dbe9-d42a-4006-802d-2f4dddccdbca None None] MFA Rules not processed for user `fd836cf590a145df9acf19cf3ae2ac56`. Rule list: `[]` (Enabled: `True`). {{(pid=88113) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:51:08.370256 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.fernet_utils [None req-4b36dbe9-d42a-4006-802d-2f4dddccdbca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:51:08.415468 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:51:08.464706 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 1172/2346] 10.222.0.13 () {58 vars in 958 bytes} [Wed Aug 31 09:51:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:51:08.470207 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b36dbe9-d42a-4006-802d-2f4dddccdbca None None] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:51:08.492587 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b36dbe9-d42a-4006-802d-2f4dddccdbca None None] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:51:08.496862 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f986011a-21ee-4b47-ba88-d817c0e42d57 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9bdaacd597442c2bcde2c555f2cc140', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '49548eb44d62415582af820ab66decee', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:51:08.497954 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f986011a-21ee-4b47-ba88-d817c0e42d57 service neutron] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:51:08.498690 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f986011a-21ee-4b47-ba88-d817c0e42d57 service neutron] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:51:08.499151 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f986011a-21ee-4b47-ba88-d817c0e42d57 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:51:08.499322 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d75a41-a8eb-411e-90d6-f6ce59468afd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:51:08.500936 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d75a41-a8eb-411e-90d6-f6ce59468afd admin admin] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:51:08.501596 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d75a41-a8eb-411e-90d6-f6ce59468afd admin admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:51:08.501950 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d75a41-a8eb-411e-90d6-f6ce59468afd admin admin] PATH_INFO: `/v3/domains` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:51:08.502637 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76d75a41-a8eb-411e-90d6-f6ce59468afd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88113) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:51:08.502970 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76d75a41-a8eb-411e-90d6-f6ce59468afd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:51:08.503818 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.fernet_utils [None req-f986011a-21ee-4b47-ba88-d817c0e42d57 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88114) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:51:08.504628 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76d75a41-a8eb-411e-90d6-f6ce59468afd admin admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:51:08.528512 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f986011a-21ee-4b47-ba88-d817c0e42d57 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:51:08.529448 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 1173/2347] 10.222.0.13 () {58 vars in 1153 bytes} [Wed Aug 31 09:51:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:51:08.531741 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f986011a-21ee-4b47-ba88-d817c0e42d57 service neutron] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:51:08.534665 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d75a41-a8eb-411e-90d6-f6ce59468afd admin admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:51:08.545426 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d75a41-a8eb-411e-90d6-f6ce59468afd admin admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:51:08.550297 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 1175/2348] 10.222.0.13 () {60 vars in 1365 bytes} [Wed Aug 31 09:51:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 300 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:51:08.552232 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a0cfbb8-7be0-4d69-8fdb-ec879181e333 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:51:08.552992 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0cfbb8-7be0-4d69-8fdb-ec879181e333 admin admin] REQUEST_METHOD: `POST` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:51:08.553411 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0cfbb8-7be0-4d69-8fdb-ec879181e333 admin admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:51:08.553792 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0cfbb8-7be0-4d69-8fdb-ec879181e333 admin admin] PATH_INFO: `/v3/projects` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:51:08.554759 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a0cfbb8-7be0-4d69-8fdb-ec879181e333 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:51:08.556930 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a0cfbb8-7be0-4d69-8fdb-ec879181e333 admin admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:51:08.601767 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 1174/2349] 10.222.0.13 () {60 vars in 1153 bytes} [Wed Aug 31 09:51:08 2022] POST /identity/v3/projects => generated 377 bytes in 67 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 09:51:08.605873 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f986011a-21ee-4b47-ba88-d817c0e42d57 service neutron] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:51:08.623741 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f986011a-21ee-4b47-ba88-d817c0e42d57 service neutron] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:51:08.706041 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a0cfbb8-7be0-4d69-8fdb-ec879181e333 admin admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:51:08.708979 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.fernet_utils [None req-3a0cfbb8-7be0-4d69-8fdb-ec879181e333 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:51:08.754313 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9778a68f-3d5a-4ba1-b603-4085c2ae4298 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:51:08.755316 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9778a68f-3d5a-4ba1-b603-4085c2ae4298 admin admin] REQUEST_METHOD: `POST` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:51:08.755778 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9778a68f-3d5a-4ba1-b603-4085c2ae4298 admin admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:51:08.756224 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9778a68f-3d5a-4ba1-b603-4085c2ae4298 admin admin] PATH_INFO: `/v3/users` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:51:08.757519 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9778a68f-3d5a-4ba1-b603-4085c2ae4298 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:51:08.761816 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9778a68f-3d5a-4ba1-b603-4085c2ae4298 admin admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:51:08.803865 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 1176/2350] 10.222.0.13 () {60 vars in 1144 bytes} [Wed Aug 31 09:51:08 2022] POST /identity/v3/users => generated 289 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 09:51:08.808041 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9778a68f-3d5a-4ba1-b603-4085c2ae4298 admin admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:51:08.835323 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9778a68f-3d5a-4ba1-b603-4085c2ae4298 admin admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:51:08.844701 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7755b44-b238-4a5a-93d2-5b78ebeaad5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:51:08.845682 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7755b44-b238-4a5a-93d2-5b78ebeaad5e admin admin] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:51:08.845682 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7755b44-b238-4a5a-93d2-5b78ebeaad5e admin admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:51:08.845964 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7755b44-b238-4a5a-93d2-5b78ebeaad5e admin admin] PATH_INFO: `/v3/roles` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:51:08.847201 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7755b44-b238-4a5a-93d2-5b78ebeaad5e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:51:08.848919 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7755b44-b238-4a5a-93d2-5b78ebeaad5e admin admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:51:08.864823 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 1177/2351] 10.222.0.13 () {58 vars in 1122 bytes} [Wed Aug 31 09:51:08 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 09:51:08.867099 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7755b44-b238-4a5a-93d2-5b78ebeaad5e admin admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:51:08.867424 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a0cfbb8-7be0-4d69-8fdb-ec879181e333 admin admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:51:08.873041 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e45fef0a-32d3-4858-8353-b9be32fab8ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:51:08.874283 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45fef0a-32d3-4858-8353-b9be32fab8ff admin admin] REQUEST_METHOD: `DELETE` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:51:08.874618 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45fef0a-32d3-4858-8353-b9be32fab8ff admin admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:51:08.875496 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45fef0a-32d3-4858-8353-b9be32fab8ff admin admin] PATH_INFO: `/v3/projects/0d7d52f95e5e452ca2ac11900fc0d931` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:51:08.878565 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e45fef0a-32d3-4858-8353-b9be32fab8ff admin admin] RBAC: Authorizing `identity:delete_project(project_id=0d7d52f95e5e452ca2ac11900fc0d931)` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:51:08.880267 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e45fef0a-32d3-4858-8353-b9be32fab8ff admin admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:51:08.881196 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7755b44-b238-4a5a-93d2-5b78ebeaad5e admin admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:51:08.883400 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:DELETE=8,keystone:SELECT=90,keystone:INSERT=2 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:51:08.889613 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b1bc06b-17d0-4154-a0ca-7cde9d1fc575 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:51:08.890461 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1bc06b-17d0-4154-a0ca-7cde9d1fc575 admin admin] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:51:08.892607 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1bc06b-17d0-4154-a0ca-7cde9d1fc575 admin admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:51:08.892607 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1bc06b-17d0-4154-a0ca-7cde9d1fc575 admin admin] PATH_INFO: `/v3/roles` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:51:08.893255 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b1bc06b-17d0-4154-a0ca-7cde9d1fc575 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:51:08.894695 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b1bc06b-17d0-4154-a0ca-7cde9d1fc575 admin admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:51:08.908437 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 1178/2352] 10.222.0.13 () {58 vars in 1122 bytes} [Wed Aug 31 09:51:08 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 09:51:08.913319 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b1bc06b-17d0-4154-a0ca-7cde9d1fc575 admin admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:51:08.928493 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b1bc06b-17d0-4154-a0ca-7cde9d1fc575 admin admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:51:08.940855 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a280b8cc-dc99-4ebb-8ee0-6828ac43eef7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:51:08.940855 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a280b8cc-dc99-4ebb-8ee0-6828ac43eef7 admin admin] REQUEST_METHOD: `PUT` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:51:08.941209 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a280b8cc-dc99-4ebb-8ee0-6828ac43eef7 admin admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:51:08.941209 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a280b8cc-dc99-4ebb-8ee0-6828ac43eef7 admin admin] PATH_INFO: `/v3/projects/6634173c9d534fca800d394d545a9d0c/users/1090c027746845239055973b44207556/roles/6c99f5d9f1f243eaa134d8540f21095d` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:51:08.985704 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a280b8cc-dc99-4ebb-8ee0-6828ac43eef7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6634173c9d534fca800d394d545a9d0c, user_id=1090c027746845239055973b44207556, role_id=6c99f5d9f1f243eaa134d8540f21095d)` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:51:08.988930 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a280b8cc-dc99-4ebb-8ee0-6828ac43eef7 admin admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:51:08.992119 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.notifications [None req-e45fef0a-32d3-4858-8353-b9be32fab8ff admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0d7d52f95e5e452ca2ac11900fc0d931', 'request_id': 'req-e45fef0a-32d3-4858-8353-b9be32fab8ff'} {{(pid=88113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:51:09.002101 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.notifications [None req-e45fef0a-32d3-4858-8353-b9be32fab8ff admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0d7d52f95e5e452ca2ac11900fc0d931', 'request_id': 'req-e45fef0a-32d3-4858-8353-b9be32fab8ff'} {{(pid=88113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:51:09.007447 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 1179/2353] 10.222.0.13 () {60 vars in 1483 bytes} [Wed Aug 31 09:51:08 2022] PUT /identity/v3/projects/6634173c9d534fca800d394d545a9d0c/users/1090c027746845239055973b44207556/roles/6c99f5d9f1f243eaa134d8540f21095d => generated 0 bytes in 95 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 09:51:09.008581 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a280b8cc-dc99-4ebb-8ee0-6828ac43eef7 admin admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:51:09.012150 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.fernet_utils [None req-a280b8cc-dc99-4ebb-8ee0-6828ac43eef7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88114) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:51:09.017224 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.notifications [None req-e45fef0a-32d3-4858-8353-b9be32fab8ff admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0d7d52f95e5e452ca2ac11900fc0d931', 'request_id': 'req-e45fef0a-32d3-4858-8353-b9be32fab8ff'} {{(pid=88113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:51:09.032105 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.notifications [None req-e45fef0a-32d3-4858-8353-b9be32fab8ff admin admin] The token cache is being invalidate because project 0d7d52f95e5e452ca2ac11900fc0d931 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88113) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 09:51:09.032415 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.notifications [None req-e45fef0a-32d3-4858-8353-b9be32fab8ff admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:51:09.033115 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.notifications [None req-e45fef0a-32d3-4858-8353-b9be32fab8ff admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:51:09.038214 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 1175/2354] 10.222.0.13 () {58 vars in 1233 bytes} [Wed Aug 31 09:51:08 2022] DELETE /identity/v3/projects/0d7d52f95e5e452ca2ac11900fc0d931 => generated 0 bytes in 330 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 09:51:09.038214 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbbe0614-bb26-4e5d-b343-3b8343d5744f None None] REQUEST_METHOD: `POST` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:51:09.038214 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbbe0614-bb26-4e5d-b343-3b8343d5744f None None] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:51:09.038214 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbbe0614-bb26-4e5d-b343-3b8343d5744f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:51:09.056932 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.auth.core [None req-bbbe0614-bb26-4e5d-b343-3b8343d5744f None None] MFA Rules not processed for user `fd836cf590a145df9acf19cf3ae2ac56`. Rule list: `[]` (Enabled: `True`). {{(pid=88113) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:51:09.058208 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.fernet_utils [None req-bbbe0614-bb26-4e5d-b343-3b8343d5744f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:51:09.107605 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a280b8cc-dc99-4ebb-8ee0-6828ac43eef7 admin admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:51:09.113817 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de7f051-afdb-4e18-87a1-43d7cda61238 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:51:09.114472 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de7f051-afdb-4e18-87a1-43d7cda61238 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:51:09.114749 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de7f051-afdb-4e18-87a1-43d7cda61238 admin admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:51:09.115024 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de7f051-afdb-4e18-87a1-43d7cda61238 admin admin] PATH_INFO: `/v3/projects/b7ce73b883774578b69c9267d2f80e41` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:51:09.116234 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9de7f051-afdb-4e18-87a1-43d7cda61238 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b7ce73b883774578b69c9267d2f80e41)` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:51:09.118239 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9de7f051-afdb-4e18-87a1-43d7cda61238 admin admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:51:09.131750 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 1176/2355] 10.222.0.13 () {58 vars in 958 bytes} [Wed Aug 31 09:51:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:51:09.133133 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586349a9-7675-4d36-b022-ac213369ac1a None None] REQUEST_METHOD: `POST` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:51:09.133457 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586349a9-7675-4d36-b022-ac213369ac1a None None] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:51:09.133761 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586349a9-7675-4d36-b022-ac213369ac1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:51:09.147592 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:51:09.154975 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.auth.core [None req-586349a9-7675-4d36-b022-ac213369ac1a None None] MFA Rules not processed for user `fd836cf590a145df9acf19cf3ae2ac56`. Rule list: `[]` (Enabled: `True`). {{(pid=88113) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:51:09.156747 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.fernet_utils [None req-586349a9-7675-4d36-b022-ac213369ac1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:51:09.168419 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 1177/2356] 10.222.0.13 () {58 vars in 958 bytes} [Wed Aug 31 09:51:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:51:09.169872 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-586349a9-7675-4d36-b022-ac213369ac1a None None] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:51:09.176922 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.fernet_utils [None req-586349a9-7675-4d36-b022-ac213369ac1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:51:09.201544 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.notifications [None req-9de7f051-afdb-4e18-87a1-43d7cda61238 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b7ce73b883774578b69c9267d2f80e41', 'request_id': 'req-9de7f051-afdb-4e18-87a1-43d7cda61238'} {{(pid=88114) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:51:09.214060 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.notifications [None req-9de7f051-afdb-4e18-87a1-43d7cda61238 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b7ce73b883774578b69c9267d2f80e41', 'request_id': 'req-9de7f051-afdb-4e18-87a1-43d7cda61238'} {{(pid=88114) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:51:09.224170 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.notifications [None req-9de7f051-afdb-4e18-87a1-43d7cda61238 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b7ce73b883774578b69c9267d2f80e41', 'request_id': 'req-9de7f051-afdb-4e18-87a1-43d7cda61238'} {{(pid=88114) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:51:09.241798 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.notifications [None req-9de7f051-afdb-4e18-87a1-43d7cda61238 admin admin] The token cache is being invalidate because project b7ce73b883774578b69c9267d2f80e41 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88114) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 09:51:09.242257 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.notifications [None req-9de7f051-afdb-4e18-87a1-43d7cda61238 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88114) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:51:09.243248 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.notifications [None req-9de7f051-afdb-4e18-87a1-43d7cda61238 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88114) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:51:09.245313 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 1180/2357] 10.222.0.13 () {58 vars in 1233 bytes} [Wed Aug 31 09:51:09 2022] DELETE /identity/v3/projects/b7ce73b883774578b69c9267d2f80e41 => generated 0 bytes in 237 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:51:09.247450 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9be969-23c3-44a0-bfde-9b47d43674bf None None] REQUEST_METHOD: `POST` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:51:09.247862 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9be969-23c3-44a0-bfde-9b47d43674bf None None] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:51:09.248491 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9be969-23c3-44a0-bfde-9b47d43674bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:51:09.252734 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-586349a9-7675-4d36-b022-ac213369ac1a None None] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:51:09.280131 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.auth.core [None req-2b9be969-23c3-44a0-bfde-9b47d43674bf None None] MFA Rules not processed for user `fd836cf590a145df9acf19cf3ae2ac56`. Rule list: `[]` (Enabled: `True`). {{(pid=88114) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:51:09.282782 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.fernet_utils [None req-2b9be969-23c3-44a0-bfde-9b47d43674bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88114) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:51:09.324965 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:51:09.344078 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5284f3ff-ce67-445c-8ba3-d3caf736d905 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9bdaacd597442c2bcde2c555f2cc140', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '49548eb44d62415582af820ab66decee', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:51:09.345264 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5284f3ff-ce67-445c-8ba3-d3caf736d905 service neutron] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:51:09.345710 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5284f3ff-ce67-445c-8ba3-d3caf736d905 service neutron] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:51:09.346128 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5284f3ff-ce67-445c-8ba3-d3caf736d905 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:51:09.349341 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.fernet_utils [None req-5284f3ff-ce67-445c-8ba3-d3caf736d905 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:51:09.380505 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 1181/2358] 10.222.0.13 () {58 vars in 958 bytes} [Wed Aug 31 09:51:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:51:09.380505 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c8fb3b-c0a7-4147-897a-80d4d7e231d5 None None] REQUEST_METHOD: `POST` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:51:09.380505 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c8fb3b-c0a7-4147-897a-80d4d7e231d5 None None] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:51:09.380505 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c8fb3b-c0a7-4147-897a-80d4d7e231d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:51:09.413097 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.auth.core [None req-c0c8fb3b-c0a7-4147-897a-80d4d7e231d5 None None] MFA Rules not processed for user `fd836cf590a145df9acf19cf3ae2ac56`. Rule list: `[]` (Enabled: `True`). {{(pid=88114) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:51:09.415181 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.fernet_utils [None req-c0c8fb3b-c0a7-4147-897a-80d4d7e231d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88114) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:51:09.430356 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 1182/2359] 10.222.0.13 () {58 vars in 958 bytes} [Wed Aug 31 09:51:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 53 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:51:09.431414 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5284f3ff-ce67-445c-8ba3-d3caf736d905 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:51:09.431864 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c8fb3b-c0a7-4147-897a-80d4d7e231d5 None None] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:51:09.433303 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5284f3ff-ce67-445c-8ba3-d3caf736d905 service neutron] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:51:09.445952 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 1178/2360] 10.222.0.13 () {60 vars in 1365 bytes} [Wed Aug 31 09:51:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 277 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:51:09.447304 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a1fd87-b29f-4335-9ffa-7eb3657d0fe9 None None] REQUEST_METHOD: `POST` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:51:09.447586 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a1fd87-b29f-4335-9ffa-7eb3657d0fe9 None None] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:51:09.447920 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a1fd87-b29f-4335-9ffa-7eb3657d0fe9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:51:09.450730 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c8fb3b-c0a7-4147-897a-80d4d7e231d5 None None] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:51:09.461122 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a75e20-ec76-4ddf-a798-269813fe2672 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:51:09.462099 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a75e20-ec76-4ddf-a798-269813fe2672 admin admin] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:51:09.462497 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a75e20-ec76-4ddf-a798-269813fe2672 admin admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:51:09.462916 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a75e20-ec76-4ddf-a798-269813fe2672 admin admin] PATH_INFO: `/v3/domains` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:51:09.463814 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14a75e20-ec76-4ddf-a798-269813fe2672 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88114) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:51:09.464217 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14a75e20-ec76-4ddf-a798-269813fe2672 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:51:09.466206 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14a75e20-ec76-4ddf-a798-269813fe2672 admin admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:51:09.486303 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.auth.core [None req-69a1fd87-b29f-4335-9ffa-7eb3657d0fe9 None None] MFA Rules not processed for user `fd836cf590a145df9acf19cf3ae2ac56`. Rule list: `[]` (Enabled: `True`). {{(pid=88113) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:51:09.487431 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.fernet_utils [None req-69a1fd87-b29f-4335-9ffa-7eb3657d0fe9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:51:09.489684 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 1183/2361] 10.222.0.13 () {58 vars in 1153 bytes} [Wed Aug 31 09:51:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:51:09.494685 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a75e20-ec76-4ddf-a798-269813fe2672 admin admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:51:09.505047 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 1179/2362] 10.222.0.13 () {58 vars in 958 bytes} [Wed Aug 31 09:51:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 59 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:51:09.510475 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a1fd87-b29f-4335-9ffa-7eb3657d0fe9 None None] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:51:09.520247 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a75e20-ec76-4ddf-a798-269813fe2672 admin admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:51:09.523165 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a1fd87-b29f-4335-9ffa-7eb3657d0fe9 None None] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:51:09.530435 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-989ec5f0-7717-4336-af18-4897fa475819 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:51:09.530827 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd689d3e-209c-45a3-85fa-45b2935f5d87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:51:09.531645 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989ec5f0-7717-4336-af18-4897fa475819 admin admin] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:51:09.531929 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd689d3e-209c-45a3-85fa-45b2935f5d87 admin admin] REQUEST_METHOD: `POST` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:51:09.532133 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989ec5f0-7717-4336-af18-4897fa475819 admin admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:51:09.532451 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd689d3e-209c-45a3-85fa-45b2935f5d87 admin admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:51:09.532616 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989ec5f0-7717-4336-af18-4897fa475819 admin admin] PATH_INFO: `/v3/domains` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:51:09.533134 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-989ec5f0-7717-4336-af18-4897fa475819 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88113) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:51:09.533301 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd689d3e-209c-45a3-85fa-45b2935f5d87 admin admin] PATH_INFO: `/v3/projects` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:51:09.533474 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-989ec5f0-7717-4336-af18-4897fa475819 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:51:09.534920 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd689d3e-209c-45a3-85fa-45b2935f5d87 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:51:09.535389 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-989ec5f0-7717-4336-af18-4897fa475819 admin admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:51:09.538889 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd689d3e-209c-45a3-85fa-45b2935f5d87 admin admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:51:09.557543 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 1180/2363] 10.222.0.13 () {58 vars in 1153 bytes} [Wed Aug 31 09:51:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 09:51:09.563194 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-989ec5f0-7717-4336-af18-4897fa475819 admin admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:51:09.582377 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-989ec5f0-7717-4336-af18-4897fa475819 admin admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:51:09.594123 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e7a7aea-b781-4da4-ad8b-ebd297c023f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:51:09.595759 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7a7aea-b781-4da4-ad8b-ebd297c023f5 admin admin] REQUEST_METHOD: `POST` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:51:09.596185 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7a7aea-b781-4da4-ad8b-ebd297c023f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:51:09.596748 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7a7aea-b781-4da4-ad8b-ebd297c023f5 admin admin] PATH_INFO: `/v3/projects` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:51:09.597818 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e7a7aea-b781-4da4-ad8b-ebd297c023f5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:51:09.602203 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e7a7aea-b781-4da4-ad8b-ebd297c023f5 admin admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:51:09.617738 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 1184/2364] 10.222.0.13 () {60 vars in 1153 bytes} [Wed Aug 31 09:51:09 2022] POST /identity/v3/projects => generated 371 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 09:51:09.631299 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd689d3e-209c-45a3-85fa-45b2935f5d87 admin admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:51:09.680231 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 1181/2365] 10.222.0.13 () {60 vars in 1153 bytes} [Wed Aug 31 09:51:09 2022] POST /identity/v3/projects => generated 373 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 09:51:09.686414 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e7a7aea-b781-4da4-ad8b-ebd297c023f5 admin admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:51:09.782559 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd689d3e-209c-45a3-85fa-45b2935f5d87 admin admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:51:09.789680 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:51:09.849874 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e7a7aea-b781-4da4-ad8b-ebd297c023f5 admin admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:51:09.858599 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb2c917-4fd1-45a7-9a39-4d4eab8497e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:51:09.860690 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb2c917-4fd1-45a7-9a39-4d4eab8497e6 admin admin] REQUEST_METHOD: `POST` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:51:09.861077 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb2c917-4fd1-45a7-9a39-4d4eab8497e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:51:09.861358 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb2c917-4fd1-45a7-9a39-4d4eab8497e6 admin admin] PATH_INFO: `/v3/users` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:51:09.862106 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecb2c917-4fd1-45a7-9a39-4d4eab8497e6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:51:09.864972 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecb2c917-4fd1-45a7-9a39-4d4eab8497e6 admin admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:51:09.901093 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 1182/2366] 10.222.0.13 () {60 vars in 1144 bytes} [Wed Aug 31 09:51:09 2022] POST /identity/v3/users => generated 287 bytes in 216 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 09:51:09.907501 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb2c917-4fd1-45a7-9a39-4d4eab8497e6 admin admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:51:09.914363 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1699235-7976-4e18-8499-143309047b41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:51:09.915239 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1699235-7976-4e18-8499-143309047b41 admin admin] REQUEST_METHOD: `POST` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:51:09.915239 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1699235-7976-4e18-8499-143309047b41 admin admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:51:09.915783 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1699235-7976-4e18-8499-143309047b41 admin admin] PATH_INFO: `/v3/users` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:51:09.916135 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1699235-7976-4e18-8499-143309047b41 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:51:09.918159 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1699235-7976-4e18-8499-143309047b41 admin admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:51:09.921519 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb2c917-4fd1-45a7-9a39-4d4eab8497e6 admin admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:51:09.928224 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f6cfb9-16c6-4374-9a78-66339337d407 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:51:09.929778 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f6cfb9-16c6-4374-9a78-66339337d407 admin admin] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:51:09.930311 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f6cfb9-16c6-4374-9a78-66339337d407 admin admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:51:09.930774 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f6cfb9-16c6-4374-9a78-66339337d407 admin admin] PATH_INFO: `/v3/roles` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:51:09.931678 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21f6cfb9-16c6-4374-9a78-66339337d407 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:51:09.933363 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21f6cfb9-16c6-4374-9a78-66339337d407 admin admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:51:09.943509 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 1185/2367] 10.222.0.13 () {60 vars in 1144 bytes} [Wed Aug 31 09:51:09 2022] POST /identity/v3/users => generated 286 bytes in 322 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 09:51:09.947798 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1699235-7976-4e18-8499-143309047b41 admin admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:51:09.955514 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 1183/2368] 10.222.0.13 () {58 vars in 1122 bytes} [Wed Aug 31 09:51:09 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 09:51:09.957636 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f6cfb9-16c6-4374-9a78-66339337d407 admin admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:51:09.966922 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1699235-7976-4e18-8499-143309047b41 admin admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:51:09.971065 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f6cfb9-16c6-4374-9a78-66339337d407 admin admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:51:09.979352 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9489a66-2a4f-4902-89ae-d3afecbb2a32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:51:09.979983 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9489a66-2a4f-4902-89ae-d3afecbb2a32 admin admin] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:51:09.980265 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9489a66-2a4f-4902-89ae-d3afecbb2a32 admin admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:51:09.980437 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07016d1f-f0d1-4123-a086-9c88898e8cf9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:51:09.980601 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9489a66-2a4f-4902-89ae-d3afecbb2a32 admin admin] PATH_INFO: `/v3/roles` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:51:09.981007 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07016d1f-f0d1-4123-a086-9c88898e8cf9 admin admin] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:51:09.981217 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9489a66-2a4f-4902-89ae-d3afecbb2a32 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:51:09.981357 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07016d1f-f0d1-4123-a086-9c88898e8cf9 admin admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:51:09.981625 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07016d1f-f0d1-4123-a086-9c88898e8cf9 admin admin] PATH_INFO: `/v3/roles` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:51:09.982247 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07016d1f-f0d1-4123-a086-9c88898e8cf9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:51:09.982929 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9489a66-2a4f-4902-89ae-d3afecbb2a32 admin admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:51:09.983869 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07016d1f-f0d1-4123-a086-9c88898e8cf9 admin admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:51:10.000416 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 1184/2369] 10.222.0.13 () {58 vars in 1122 bytes} [Wed Aug 31 09:51:09 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 09:51:10.001421 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 1186/2370] 10.222.0.13 () {58 vars in 1122 bytes} [Wed Aug 31 09:51:09 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 09:51:10.002787 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9489a66-2a4f-4902-89ae-d3afecbb2a32 admin admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:51:10.004328 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07016d1f-f0d1-4123-a086-9c88898e8cf9 admin admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:51:10.015100 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9489a66-2a4f-4902-89ae-d3afecbb2a32 admin admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:51:10.015905 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07016d1f-f0d1-4123-a086-9c88898e8cf9 admin admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:51:10.021101 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e7bfc64-e738-4e51-a408-bc52abd69e85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:51:10.021648 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1735f0e3-db8b-431d-818a-cb9dcb77b57f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:51:10.021857 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7bfc64-e738-4e51-a408-bc52abd69e85 admin admin] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:51:10.022006 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1735f0e3-db8b-431d-818a-cb9dcb77b57f admin admin] REQUEST_METHOD: `PUT` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:51:10.022182 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7bfc64-e738-4e51-a408-bc52abd69e85 admin admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:51:10.022319 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1735f0e3-db8b-431d-818a-cb9dcb77b57f admin admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:51:10.022466 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7bfc64-e738-4e51-a408-bc52abd69e85 admin admin] PATH_INFO: `/v3/roles` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:51:10.022648 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1735f0e3-db8b-431d-818a-cb9dcb77b57f admin admin] PATH_INFO: `/v3/projects/b058d5f34eee47a58629407e166f3499/users/c89dbe311b24408c925f102eefd6d8c3/roles/6c99f5d9f1f243eaa134d8540f21095d` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:51:10.023140 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e7bfc64-e738-4e51-a408-bc52abd69e85 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:51:10.025227 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e7bfc64-e738-4e51-a408-bc52abd69e85 admin admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:51:10.041781 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 1185/2371] 10.222.0.13 () {58 vars in 1122 bytes} [Wed Aug 31 09:51:10 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 09:51:10.042987 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e7bfc64-e738-4e51-a408-bc52abd69e85 admin admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:51:10.047835 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1735f0e3-db8b-431d-818a-cb9dcb77b57f admin admin] RBAC: Authorizing `identity:create_grant(project_id=b058d5f34eee47a58629407e166f3499, user_id=c89dbe311b24408c925f102eefd6d8c3, role_id=6c99f5d9f1f243eaa134d8540f21095d)` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:51:10.051650 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1735f0e3-db8b-431d-818a-cb9dcb77b57f admin admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:51:10.055580 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e7bfc64-e738-4e51-a408-bc52abd69e85 admin admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:51:10.061676 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af240491-8bac-418d-b97c-f28cdd057f23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:51:10.062370 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af240491-8bac-418d-b97c-f28cdd057f23 admin admin] REQUEST_METHOD: `PUT` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:51:10.062746 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af240491-8bac-418d-b97c-f28cdd057f23 admin admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:51:10.063120 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af240491-8bac-418d-b97c-f28cdd057f23 admin admin] PATH_INFO: `/v3/projects/1e493d3821e8406e96c47dfce3625ce4/users/65090dbeddc84d8cac3a1d58696f8cb6/roles/6c99f5d9f1f243eaa134d8540f21095d` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:51:10.066154 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:51:10.080342 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 1187/2372] 10.222.0.13 () {60 vars in 1483 bytes} [Wed Aug 31 09:51:10 2022] PUT /identity/v3/projects/b058d5f34eee47a58629407e166f3499/users/c89dbe311b24408c925f102eefd6d8c3/roles/6c99f5d9f1f243eaa134d8540f21095d => generated 0 bytes in 72 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 09:51:10.082875 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411e6918-a664-493d-afd7-a436bcef6430 None None] REQUEST_METHOD: `POST` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:51:10.083222 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411e6918-a664-493d-afd7-a436bcef6430 None None] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:51:10.083506 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411e6918-a664-493d-afd7-a436bcef6430 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:51:10.102180 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af240491-8bac-418d-b97c-f28cdd057f23 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1e493d3821e8406e96c47dfce3625ce4, user_id=65090dbeddc84d8cac3a1d58696f8cb6, role_id=6c99f5d9f1f243eaa134d8540f21095d)` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:51:10.107323 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af240491-8bac-418d-b97c-f28cdd057f23 admin admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:51:10.114062 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.auth.core [None req-411e6918-a664-493d-afd7-a436bcef6430 None None] MFA Rules not processed for user `fd836cf590a145df9acf19cf3ae2ac56`. Rule list: `[]` (Enabled: `True`). {{(pid=88114) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:51:10.115424 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.fernet_utils [None req-411e6918-a664-493d-afd7-a436bcef6430 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88114) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:51:10.129376 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 1186/2373] 10.222.0.13 () {60 vars in 1483 bytes} [Wed Aug 31 09:51:10 2022] PUT /identity/v3/projects/1e493d3821e8406e96c47dfce3625ce4/users/65090dbeddc84d8cac3a1d58696f8cb6/roles/6c99f5d9f1f243eaa134d8540f21095d => generated 0 bytes in 87 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:51:10.142994 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e1a3bd-a453-495e-a942-0e424c69b523 None None] REQUEST_METHOD: `POST` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:51:10.143279 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e1a3bd-a453-495e-a942-0e424c69b523 None None] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:51:10.143555 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e1a3bd-a453-495e-a942-0e424c69b523 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:51:10.187000 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.auth.core [None req-59e1a3bd-a453-495e-a942-0e424c69b523 None None] MFA Rules not processed for user `65090dbeddc84d8cac3a1d58696f8cb6`. Rule list: `[]` (Enabled: `True`). {{(pid=88113) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:51:10.190037 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.fernet_utils [None req-59e1a3bd-a453-495e-a942-0e424c69b523 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:51:10.227696 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 1188/2374] 10.222.0.13 () {58 vars in 958 bytes} [Wed Aug 31 09:51:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:51:10.246983 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-411e6918-a664-493d-afd7-a436bcef6430 None None] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:51:10.248850 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.fernet_utils [None req-411e6918-a664-493d-afd7-a436bcef6430 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88114) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:51:10.352564 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-411e6918-a664-493d-afd7-a436bcef6430 None None] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:51:10.377339 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 1187/2375] 10.222.0.13 () {58 vars in 958 bytes} [Wed Aug 31 09:51:10 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 235 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 09:51:10.396808 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:51:10.415439 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e1a3bd-a453-495e-a942-0e424c69b523 None None] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:51:10.417991 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.fernet_utils [None req-59e1a3bd-a453-495e-a942-0e424c69b523 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:51:10.494558 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ef935a-4177-48c1-8555-d7c2c9ee974a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9bdaacd597442c2bcde2c555f2cc140', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '49548eb44d62415582af820ab66decee', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:51:10.495876 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ef935a-4177-48c1-8555-d7c2c9ee974a service neutron] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:51:10.496966 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ef935a-4177-48c1-8555-d7c2c9ee974a service neutron] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:51:10.497977 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ef935a-4177-48c1-8555-d7c2c9ee974a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:51:10.607562 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e1a3bd-a453-495e-a942-0e424c69b523 None None] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:51:10.615198 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-950f5243-e5b3-42de-b05d-d4bc8770f38e service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be216d156b54452cbbf978c7734343aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '49548eb44d62415582af820ab66decee', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:51:10.616291 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950f5243-e5b3-42de-b05d-d4bc8770f38e service glance] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:51:10.616988 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950f5243-e5b3-42de-b05d-d4bc8770f38e service glance] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:51:10.617646 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950f5243-e5b3-42de-b05d-d4bc8770f38e service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:51:10.617987 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80ef935a-4177-48c1-8555-d7c2c9ee974a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:51:10.619626 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80ef935a-4177-48c1-8555-d7c2c9ee974a service neutron] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:51:10.635095 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-950f5243-e5b3-42de-b05d-d4bc8770f38e service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:51:10.637195 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 1189/2376] 10.222.0.13 () {60 vars in 1365 bytes} [Wed Aug 31 09:51:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 391 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 09:51:10.638578 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-950f5243-e5b3-42de-b05d-d4bc8770f38e service glance] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:51:10.669029 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 1188/2377] 10.222.0.13 () {60 vars in 1365 bytes} [Wed Aug 31 09:51:10 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 255 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 09:51:20.629865 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=41 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:51:20.649328 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=65,keystone:INSERT=1 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:51:20.658597 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4774d142-af0d-4b7b-87a4-68e01b6da554 None None] REQUEST_METHOD: `POST` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:51:20.658885 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4774d142-af0d-4b7b-87a4-68e01b6da554 None None] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:51:20.659162 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4774d142-af0d-4b7b-87a4-68e01b6da554 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:51:20.704059 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.auth.core [None req-4774d142-af0d-4b7b-87a4-68e01b6da554 None None] MFA Rules not processed for user `c89dbe311b24408c925f102eefd6d8c3`. Rule list: `[]` (Enabled: `True`). {{(pid=88114) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:51:20.705456 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.fernet_utils [None req-4774d142-af0d-4b7b-87a4-68e01b6da554 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88114) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:51:20.846712 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce23cef1-1a81-4a50-9a14-426aa0db200a None None] REQUEST_METHOD: `POST` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:51:20.846992 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce23cef1-1a81-4a50-9a14-426aa0db200a None None] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:51:20.848049 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce23cef1-1a81-4a50-9a14-426aa0db200a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:51:20.901728 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 1190/2378] 10.222.0.13 () {58 vars in 958 bytes} [Wed Aug 31 09:51:20 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 244 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:51:20.910511 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4774d142-af0d-4b7b-87a4-68e01b6da554 None None] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:51:20.933783 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.auth.core [None req-ce23cef1-1a81-4a50-9a14-426aa0db200a None None] MFA Rules not processed for user `1090c027746845239055973b44207556`. Rule list: `[]` (Enabled: `True`). {{(pid=88113) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:51:20.935076 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.fernet_utils [None req-ce23cef1-1a81-4a50-9a14-426aa0db200a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:51:20.938188 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4774d142-af0d-4b7b-87a4-68e01b6da554 None None] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:51:20.948331 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e1fff9-1008-4c30-872e-3edb5e9ef7a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:51:20.949440 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e1fff9-1008-4c30-872e-3edb5e9ef7a4 admin admin] REQUEST_METHOD: `POST` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:51:20.949882 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e1fff9-1008-4c30-872e-3edb5e9ef7a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:51:20.950331 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e1fff9-1008-4c30-872e-3edb5e9ef7a4 admin admin] PATH_INFO: `/v3/projects` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:51:20.951511 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6e1fff9-1008-4c30-872e-3edb5e9ef7a4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:51:20.954721 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6e1fff9-1008-4c30-872e-3edb5e9ef7a4 admin admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:51:21.060138 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 1191/2379] 10.222.0.13 () {60 vars in 1153 bytes} [Wed Aug 31 09:51:20 2022] POST /identity/v3/projects => generated 371 bytes in 145 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 09:51:21.062510 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e1fff9-1008-4c30-872e-3edb5e9ef7a4 admin admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:51:21.078665 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 1189/2380] 10.222.0.13 () {58 vars in 958 bytes} [Wed Aug 31 09:51:20 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 239 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 09:51:21.079819 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce23cef1-1a81-4a50-9a14-426aa0db200a None None] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:51:21.081704 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.fernet_utils [None req-ce23cef1-1a81-4a50-9a14-426aa0db200a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:51:21.099721 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce23cef1-1a81-4a50-9a14-426aa0db200a None None] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:51:21.210457 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11b1bc50-b45f-47f1-a701-25e8698b5267 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:51:21.211195 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b1bc50-b45f-47f1-a701-25e8698b5267 admin admin] REQUEST_METHOD: `POST` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:51:21.211532 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b1bc50-b45f-47f1-a701-25e8698b5267 admin admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:51:21.211867 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b1bc50-b45f-47f1-a701-25e8698b5267 admin admin] PATH_INFO: `/v3/projects` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:51:21.212596 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11b1bc50-b45f-47f1-a701-25e8698b5267 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:51:21.215990 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11b1bc50-b45f-47f1-a701-25e8698b5267 admin admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:51:21.234859 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e1fff9-1008-4c30-872e-3edb5e9ef7a4 admin admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:51:21.253282 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe95754e-e521-4fae-b0e9-59d9bf1088ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:51:21.254184 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe95754e-e521-4fae-b0e9-59d9bf1088ec admin admin] REQUEST_METHOD: `POST` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:51:21.254479 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe95754e-e521-4fae-b0e9-59d9bf1088ec admin admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:51:21.254749 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe95754e-e521-4fae-b0e9-59d9bf1088ec admin admin] PATH_INFO: `/v3/users` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:51:21.255449 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe95754e-e521-4fae-b0e9-59d9bf1088ec admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:51:21.258083 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe95754e-e521-4fae-b0e9-59d9bf1088ec admin admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:51:21.294287 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 1190/2381] 10.222.0.13 () {60 vars in 1153 bytes} [Wed Aug 31 09:51:21 2022] POST /identity/v3/projects => generated 377 bytes in 215 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 09:51:21.298196 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11b1bc50-b45f-47f1-a701-25e8698b5267 admin admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:51:21.299584 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 1192/2382] 10.222.0.13 () {60 vars in 1144 bytes} [Wed Aug 31 09:51:21 2022] POST /identity/v3/users => generated 292 bytes in 239 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 09:51:21.305188 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe95754e-e521-4fae-b0e9-59d9bf1088ec admin admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:51:21.328864 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe95754e-e521-4fae-b0e9-59d9bf1088ec admin admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:51:21.427244 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11b1bc50-b45f-47f1-a701-25e8698b5267 admin admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:51:21.436051 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-430e3057-87a7-435b-be4a-239b3c26af47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:51:21.438243 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430e3057-87a7-435b-be4a-239b3c26af47 admin admin] REQUEST_METHOD: `POST` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:51:21.438582 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430e3057-87a7-435b-be4a-239b3c26af47 admin admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:51:21.439462 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430e3057-87a7-435b-be4a-239b3c26af47 admin admin] PATH_INFO: `/v3/users` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:51:21.440248 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-430e3057-87a7-435b-be4a-239b3c26af47 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:51:21.441206 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:51:21.444811 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-430e3057-87a7-435b-be4a-239b3c26af47 admin admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:51:21.462657 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a09b20-c363-4791-9edd-583ee921b4e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:51:21.464094 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a09b20-c363-4791-9edd-583ee921b4e6 admin admin] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:51:21.464268 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a09b20-c363-4791-9edd-583ee921b4e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:51:21.464644 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a09b20-c363-4791-9edd-583ee921b4e6 admin admin] PATH_INFO: `/v3/roles` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:51:21.466442 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21a09b20-c363-4791-9edd-583ee921b4e6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:51:21.469972 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21a09b20-c363-4791-9edd-583ee921b4e6 admin admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:51:21.490890 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 1191/2383] 10.222.0.13 () {60 vars in 1144 bytes} [Wed Aug 31 09:51:21 2022] POST /identity/v3/users => generated 295 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 09:51:21.495582 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-430e3057-87a7-435b-be4a-239b3c26af47 admin admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:51:21.505365 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:51:21.511790 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-430e3057-87a7-435b-be4a-239b3c26af47 admin admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:51:21.523306 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 1193/2384] 10.222.0.13 () {58 vars in 1122 bytes} [Wed Aug 31 09:51:21 2022] GET /identity/v3/roles => generated 1420 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 09:51:21.525791 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a09b20-c363-4791-9edd-583ee921b4e6 admin admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:51:21.528727 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0966d4a7-008d-434e-933e-cf4601403e49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:51:21.529543 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0966d4a7-008d-434e-933e-cf4601403e49 admin admin] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:51:21.529836 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0966d4a7-008d-434e-933e-cf4601403e49 admin admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:51:21.530116 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0966d4a7-008d-434e-933e-cf4601403e49 admin admin] PATH_INFO: `/v3/roles` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:51:21.532377 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0966d4a7-008d-434e-933e-cf4601403e49 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:51:21.534005 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0966d4a7-008d-434e-933e-cf4601403e49 admin admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:51:21.557473 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 1192/2385] 10.222.0.13 () {58 vars in 1122 bytes} [Wed Aug 31 09:51:21 2022] GET /identity/v3/roles => generated 1420 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 09:51:21.558706 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0966d4a7-008d-434e-933e-cf4601403e49 admin admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:51:21.567515 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a09b20-c363-4791-9edd-583ee921b4e6 admin admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:51:21.573443 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0966d4a7-008d-434e-933e-cf4601403e49 admin admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:51:21.582592 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab18f47b-796a-431e-bb00-dc4262fd1854 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:51:21.584272 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab18f47b-796a-431e-bb00-dc4262fd1854 admin admin] REQUEST_METHOD: `PUT` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:51:21.584963 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab18f47b-796a-431e-bb00-dc4262fd1854 admin admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:51:21.585248 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab18f47b-796a-431e-bb00-dc4262fd1854 admin admin] PATH_INFO: `/v3/projects/061b65fc11484733bb6be383ff807570/users/bc507f26f0fd43d0aa1e594eeb86265f/roles/5d4ee63a3add47fc9a6b8b16d39e0a1f` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:51:21.587551 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f63a14-faba-48d8-ba85-ed7bb15194cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:51:21.588266 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f63a14-faba-48d8-ba85-ed7bb15194cb admin admin] REQUEST_METHOD: `PUT` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:51:21.588530 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f63a14-faba-48d8-ba85-ed7bb15194cb admin admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:51:21.589293 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f63a14-faba-48d8-ba85-ed7bb15194cb admin admin] PATH_INFO: `/v3/projects/c5f727a6b9094171b5fd34fe23f9cc5a/users/1e55ab86464d40dd85871673319b6986/roles/5d4ee63a3add47fc9a6b8b16d39e0a1f` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:51:21.628281 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1f63a14-faba-48d8-ba85-ed7bb15194cb admin admin] RBAC: Authorizing `identity:create_grant(project_id=c5f727a6b9094171b5fd34fe23f9cc5a, user_id=1e55ab86464d40dd85871673319b6986, role_id=5d4ee63a3add47fc9a6b8b16d39e0a1f)` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:51:21.631658 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1f63a14-faba-48d8-ba85-ed7bb15194cb admin admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:51:21.644974 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab18f47b-796a-431e-bb00-dc4262fd1854 admin admin] RBAC: Authorizing `identity:create_grant(project_id=061b65fc11484733bb6be383ff807570, user_id=bc507f26f0fd43d0aa1e594eeb86265f, role_id=5d4ee63a3add47fc9a6b8b16d39e0a1f)` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:51:21.651880 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab18f47b-796a-431e-bb00-dc4262fd1854 admin admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:51:21.675000 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 1193/2386] 10.222.0.13 () {60 vars in 1483 bytes} [Wed Aug 31 09:51:21 2022] PUT /identity/v3/projects/c5f727a6b9094171b5fd34fe23f9cc5a/users/1e55ab86464d40dd85871673319b6986/roles/5d4ee63a3add47fc9a6b8b16d39e0a1f => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:51:21.676292 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f63a14-faba-48d8-ba85-ed7bb15194cb admin admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:51:21.706472 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 1194/2387] 10.222.0.13 () {60 vars in 1483 bytes} [Wed Aug 31 09:51:21 2022] PUT /identity/v3/projects/061b65fc11484733bb6be383ff807570/users/bc507f26f0fd43d0aa1e594eeb86265f/roles/5d4ee63a3add47fc9a6b8b16d39e0a1f => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:51:21.713068 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab18f47b-796a-431e-bb00-dc4262fd1854 admin admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:51:21.788581 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f63a14-faba-48d8-ba85-ed7bb15194cb admin admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:51:21.895863 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab18f47b-796a-431e-bb00-dc4262fd1854 admin admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:51:21.905334 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cdc3155-a746-4ec9-8265-ebc927a9eda2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:51:21.906931 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e82bca-8b4c-4f38-9210-883dc48cf4a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:51:21.907596 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e82bca-8b4c-4f38-9210-883dc48cf4a9 admin admin] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:51:21.907877 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e82bca-8b4c-4f38-9210-883dc48cf4a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:51:21.908202 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e82bca-8b4c-4f38-9210-883dc48cf4a9 admin admin] PATH_INFO: `/v3/roles` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:51:21.908786 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1e82bca-8b4c-4f38-9210-883dc48cf4a9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:51:21.910761 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1e82bca-8b4c-4f38-9210-883dc48cf4a9 admin admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:51:21.911539 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cdc3155-a746-4ec9-8265-ebc927a9eda2 admin admin] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:51:21.911826 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cdc3155-a746-4ec9-8265-ebc927a9eda2 admin admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:51:21.912109 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cdc3155-a746-4ec9-8265-ebc927a9eda2 admin admin] PATH_INFO: `/v3/roles` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:51:21.913815 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cdc3155-a746-4ec9-8265-ebc927a9eda2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:51:21.916096 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cdc3155-a746-4ec9-8265-ebc927a9eda2 admin admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:51:21.935811 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 1195/2388] 10.222.0.13 () {58 vars in 1122 bytes} [Wed Aug 31 09:51:21 2022] GET /identity/v3/roles => generated 1420 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 09:51:21.938128 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cdc3155-a746-4ec9-8265-ebc927a9eda2 admin admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:51:21.941713 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 1194/2389] 10.222.0.13 () {58 vars in 1122 bytes} [Wed Aug 31 09:51:21 2022] GET /identity/v3/roles => generated 1420 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 09:51:21.950317 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e82bca-8b4c-4f38-9210-883dc48cf4a9 admin admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:51:21.982469 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e82bca-8b4c-4f38-9210-883dc48cf4a9 admin admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:51:21.982870 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cdc3155-a746-4ec9-8265-ebc927a9eda2 admin admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:51:22.000526 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b633d7a-5a70-4762-8c51-45a8f5f94ad5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:51:22.002749 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-453910ad-ac50-4c76-bd38-eed1db638042 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:51:22.003720 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453910ad-ac50-4c76-bd38-eed1db638042 admin admin] REQUEST_METHOD: `PUT` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:51:22.004633 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453910ad-ac50-4c76-bd38-eed1db638042 admin admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:51:22.006823 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b633d7a-5a70-4762-8c51-45a8f5f94ad5 admin admin] REQUEST_METHOD: `PUT` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:51:22.008257 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b633d7a-5a70-4762-8c51-45a8f5f94ad5 admin admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:51:22.010836 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b633d7a-5a70-4762-8c51-45a8f5f94ad5 admin admin] PATH_INFO: `/v3/projects/c5f727a6b9094171b5fd34fe23f9cc5a/users/1e55ab86464d40dd85871673319b6986/roles/5d4ee63a3add47fc9a6b8b16d39e0a1f` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:51:22.015830 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b633d7a-5a70-4762-8c51-45a8f5f94ad5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c5f727a6b9094171b5fd34fe23f9cc5a, user_id=1e55ab86464d40dd85871673319b6986, role_id=5d4ee63a3add47fc9a6b8b16d39e0a1f)` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:51:22.019392 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b633d7a-5a70-4762-8c51-45a8f5f94ad5 admin admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:51:22.026960 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453910ad-ac50-4c76-bd38-eed1db638042 admin admin] PATH_INFO: `/v3/projects/061b65fc11484733bb6be383ff807570/users/bc507f26f0fd43d0aa1e594eeb86265f/roles/5d4ee63a3add47fc9a6b8b16d39e0a1f` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:51:22.031365 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-453910ad-ac50-4c76-bd38-eed1db638042 admin admin] RBAC: Authorizing `identity:create_grant(project_id=061b65fc11484733bb6be383ff807570, user_id=bc507f26f0fd43d0aa1e594eeb86265f, role_id=5d4ee63a3add47fc9a6b8b16d39e0a1f)` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:51:22.035251 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-453910ad-ac50-4c76-bd38-eed1db638042 admin admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:51:22.068200 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 1196/2390] 10.222.0.13 () {60 vars in 1483 bytes} [Wed Aug 31 09:51:21 2022] PUT /identity/v3/projects/061b65fc11484733bb6be383ff807570/users/bc507f26f0fd43d0aa1e594eeb86265f/roles/5d4ee63a3add47fc9a6b8b16d39e0a1f => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:51:22.075037 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 1195/2391] 10.222.0.13 () {60 vars in 1483 bytes} [Wed Aug 31 09:51:21 2022] PUT /identity/v3/projects/c5f727a6b9094171b5fd34fe23f9cc5a/users/1e55ab86464d40dd85871673319b6986/roles/5d4ee63a3add47fc9a6b8b16d39e0a1f => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:51:29.939476 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a9fec8-557e-4c68-a722-590e981a53c0 None None] REQUEST_METHOD: `POST` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:51:29.940375 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a9fec8-557e-4c68-a722-590e981a53c0 None None] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:51:29.940375 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a9fec8-557e-4c68-a722-590e981a53c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:51:30.036874 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.auth.core [None req-82a9fec8-557e-4c68-a722-590e981a53c0 None None] MFA Rules not processed for user `1e55ab86464d40dd85871673319b6986`. Rule list: `[]` (Enabled: `True`). {{(pid=88114) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:51:30.060319 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.fernet_utils [None req-82a9fec8-557e-4c68-a722-590e981a53c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88114) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:51:30.238160 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 1197/2392] 10.222.0.13 () {58 vars in 958 bytes} [Wed Aug 31 09:51:29 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 301 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:51:30.278366 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b633d7a-5a70-4762-8c51-45a8f5f94ad5 admin admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:51:30.281593 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.fernet_utils [None req-0b633d7a-5a70-4762-8c51-45a8f5f94ad5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:51:30.515601 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:51:30.576449 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b633d7a-5a70-4762-8c51-45a8f5f94ad5 admin admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:51:30.731074 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb1d01bc-6650-44b8-b481-638ea8a91234 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc076493239d4ccdbdc37c0369c49529', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '49548eb44d62415582af820ab66decee', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:51:30.731074 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1d01bc-6650-44b8-b481-638ea8a91234 service nova] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:51:30.731074 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1d01bc-6650-44b8-b481-638ea8a91234 service nova] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:51:30.731074 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1d01bc-6650-44b8-b481-638ea8a91234 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:51:30.797700 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088a035d-4683-464e-9694-d2dd3c921de3 None None] REQUEST_METHOD: `POST` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:51:30.798219 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088a035d-4683-464e-9694-d2dd3c921de3 None None] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:51:30.798688 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088a035d-4683-464e-9694-d2dd3c921de3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:51:30.854844 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb1d01bc-6650-44b8-b481-638ea8a91234 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:51:30.857142 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb1d01bc-6650-44b8-b481-638ea8a91234 service nova] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:51:30.863812 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.auth.core [None req-088a035d-4683-464e-9694-d2dd3c921de3 None None] MFA Rules not processed for user `bc507f26f0fd43d0aa1e594eeb86265f`. Rule list: `[]` (Enabled: `True`). {{(pid=88114) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:51:30.865606 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.fernet_utils [None req-088a035d-4683-464e-9694-d2dd3c921de3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88114) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:51:30.872425 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 1196/2393] 10.222.0.13 () {60 vars in 1365 bytes} [Wed Aug 31 09:51:30 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 594 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:51:30.916875 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:51:31.009974 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 1198/2394] 10.222.0.13 () {58 vars in 958 bytes} [Wed Aug 31 09:51:30 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 214 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:51:31.029259 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb1d01bc-6650-44b8-b481-638ea8a91234 service nova] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:51:31.032038 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.fernet_utils [None req-cb1d01bc-6650-44b8-b481-638ea8a91234 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:51:31.072773 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb1d01bc-6650-44b8-b481-638ea8a91234 service nova] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:51:31.083468 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf92f4a8-b25d-4a0f-8d87-7c4fb03e5b98 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc076493239d4ccdbdc37c0369c49529', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '49548eb44d62415582af820ab66decee', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:51:31.084416 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf92f4a8-b25d-4a0f-8d87-7c4fb03e5b98 service nova] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:51:31.084871 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf92f4a8-b25d-4a0f-8d87-7c4fb03e5b98 service nova] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:51:31.085343 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf92f4a8-b25d-4a0f-8d87-7c4fb03e5b98 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:51:31.173151 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf92f4a8-b25d-4a0f-8d87-7c4fb03e5b98 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:51:31.175031 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf92f4a8-b25d-4a0f-8d87-7c4fb03e5b98 service nova] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:51:31.213680 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 1197/2395] 10.222.0.13 () {60 vars in 1365 bytes} [Wed Aug 31 09:51:31 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 186 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:51:32.825813 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817c7910-80a5-45a3-b002-2afa9343dafd None None] REQUEST_METHOD: `POST` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:51:32.826281 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817c7910-80a5-45a3-b002-2afa9343dafd None None] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:51:32.827259 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817c7910-80a5-45a3-b002-2afa9343dafd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:51:32.861580 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.auth.core [None req-817c7910-80a5-45a3-b002-2afa9343dafd None None] MFA Rules not processed for user `c89dbe311b24408c925f102eefd6d8c3`. Rule list: `[]` (Enabled: `True`). {{(pid=88114) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:51:32.863811 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.fernet_utils [None req-817c7910-80a5-45a3-b002-2afa9343dafd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88114) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:51:32.876203 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 1199/2396] 10.222.0.13 () {58 vars in 958 bytes} [Wed Aug 31 09:51:32 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:51:41.197353 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=85 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:51:42.846224 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=20 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:51:54.666136 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf92f4a8-b25d-4a0f-8d87-7c4fb03e5b98 service nova] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:51:54.731315 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf92f4a8-b25d-4a0f-8d87-7c4fb03e5b98 service nova] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:51:54.736057 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab711a90-6ef5-4632-9c06-375ce371986d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:51:54.736642 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab711a90-6ef5-4632-9c06-375ce371986d admin admin] REQUEST_METHOD: `DELETE` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:51:54.736920 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab711a90-6ef5-4632-9c06-375ce371986d admin admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:51:54.737196 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab711a90-6ef5-4632-9c06-375ce371986d admin admin] PATH_INFO: `/v3/users/65090dbeddc84d8cac3a1d58696f8cb6` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:51:54.738375 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab711a90-6ef5-4632-9c06-375ce371986d admin admin] RBAC: Authorizing `identity:delete_user(user_id=65090dbeddc84d8cac3a1d58696f8cb6)` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:51:54.740319 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab711a90-6ef5-4632-9c06-375ce371986d admin admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:51:54.797026 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.notifications [None req-ab711a90-6ef5-4632-9c06-375ce371986d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '65090dbeddc84d8cac3a1d58696f8cb6'} {{(pid=88113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:51:54.804884 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.notifications [None req-ab711a90-6ef5-4632-9c06-375ce371986d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '65090dbeddc84d8cac3a1d58696f8cb6'} {{(pid=88113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:51:54.819493 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.notifications [None req-ab711a90-6ef5-4632-9c06-375ce371986d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '65090dbeddc84d8cac3a1d58696f8cb6'} {{(pid=88113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:51:54.820251 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.notifications [None req-ab711a90-6ef5-4632-9c06-375ce371986d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '65090dbeddc84d8cac3a1d58696f8cb6'} {{(pid=88113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:51:54.829155 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.notifications [None req-ab711a90-6ef5-4632-9c06-375ce371986d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '65090dbeddc84d8cac3a1d58696f8cb6'} {{(pid=88113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:51:54.836716 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 1198/2397] 10.222.0.13 () {58 vars in 1224 bytes} [Wed Aug 31 09:51:54 2022] DELETE /identity/v3/users/65090dbeddc84d8cac3a1d58696f8cb6 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:51:54.846125 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f37e322-a92d-43fb-afae-85740b694417 None None] REQUEST_METHOD: `POST` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:51:54.846125 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f37e322-a92d-43fb-afae-85740b694417 None None] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:51:54.846125 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f37e322-a92d-43fb-afae-85740b694417 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:51:54.862057 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.auth.core [None req-6f37e322-a92d-43fb-afae-85740b694417 None None] MFA Rules not processed for user `fd836cf590a145df9acf19cf3ae2ac56`. Rule list: `[]` (Enabled: `True`). {{(pid=88114) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:51:54.863209 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.fernet_utils [None req-6f37e322-a92d-43fb-afae-85740b694417 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88114) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:51:54.932292 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 1200/2398] 10.222.0.13 () {58 vars in 958 bytes} [Wed Aug 31 09:51:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 09:51:54.944840 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab711a90-6ef5-4632-9c06-375ce371986d admin admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:51:54.946284 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.fernet_utils [None req-ab711a90-6ef5-4632-9c06-375ce371986d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:51:55.009707 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab711a90-6ef5-4632-9c06-375ce371986d admin admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:51:55.013447 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed77f887-c4e9-4f05-b49d-a6b966be517e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9bdaacd597442c2bcde2c555f2cc140', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '49548eb44d62415582af820ab66decee', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:51:55.014050 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed77f887-c4e9-4f05-b49d-a6b966be517e service neutron] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:51:55.014322 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed77f887-c4e9-4f05-b49d-a6b966be517e service neutron] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:51:55.014591 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed77f887-c4e9-4f05-b49d-a6b966be517e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:51:55.025058 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed77f887-c4e9-4f05-b49d-a6b966be517e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:51:55.026530 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed77f887-c4e9-4f05-b49d-a6b966be517e service neutron] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:51:55.036636 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 1199/2399] 10.222.0.13 () {60 vars in 1365 bytes} [Wed Aug 31 09:51:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:51:55.317920 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f37e322-a92d-43fb-afae-85740b694417 None None] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:51:55.320423 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.fernet_utils [None req-6f37e322-a92d-43fb-afae-85740b694417 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88114) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:51:55.335480 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f37e322-a92d-43fb-afae-85740b694417 None None] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:51:55.346132 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c0820cf-6427-4c55-a567-9fb55c2ab3ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:51:55.346922 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0820cf-6427-4c55-a567-9fb55c2ab3ac admin admin] REQUEST_METHOD: `DELETE` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:51:55.347109 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0820cf-6427-4c55-a567-9fb55c2ab3ac admin admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:51:55.347382 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0820cf-6427-4c55-a567-9fb55c2ab3ac admin admin] PATH_INFO: `/v3/projects/1e493d3821e8406e96c47dfce3625ce4` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:51:55.348619 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c0820cf-6427-4c55-a567-9fb55c2ab3ac admin admin] RBAC: Authorizing `identity:delete_project(project_id=1e493d3821e8406e96c47dfce3625ce4)` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:51:55.350588 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c0820cf-6427-4c55-a567-9fb55c2ab3ac admin admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:51:55.400727 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.notifications [None req-0c0820cf-6427-4c55-a567-9fb55c2ab3ac admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1e493d3821e8406e96c47dfce3625ce4', 'request_id': 'req-0c0820cf-6427-4c55-a567-9fb55c2ab3ac'} {{(pid=88114) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:51:55.407235 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.notifications [None req-0c0820cf-6427-4c55-a567-9fb55c2ab3ac admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1e493d3821e8406e96c47dfce3625ce4', 'request_id': 'req-0c0820cf-6427-4c55-a567-9fb55c2ab3ac'} {{(pid=88114) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:51:55.413323 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.notifications [None req-0c0820cf-6427-4c55-a567-9fb55c2ab3ac admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1e493d3821e8406e96c47dfce3625ce4', 'request_id': 'req-0c0820cf-6427-4c55-a567-9fb55c2ab3ac'} {{(pid=88114) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:51:55.426528 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.notifications [None req-0c0820cf-6427-4c55-a567-9fb55c2ab3ac admin admin] The token cache is being invalidate because project 1e493d3821e8406e96c47dfce3625ce4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88114) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 09:51:55.426818 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.notifications [None req-0c0820cf-6427-4c55-a567-9fb55c2ab3ac admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88114) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:51:55.427440 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.notifications [None req-0c0820cf-6427-4c55-a567-9fb55c2ab3ac admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88114) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:51:55.428973 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 1201/2400] 10.222.0.13 () {58 vars in 1233 bytes} [Wed Aug 31 09:51:55 2022] DELETE /identity/v3/projects/1e493d3821e8406e96c47dfce3625ce4 => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:51:55.434018 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c77a72-9c3f-4574-a841-440f8ca629e4 None None] REQUEST_METHOD: `POST` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:51:55.434307 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c77a72-9c3f-4574-a841-440f8ca629e4 None None] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:51:55.434585 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c77a72-9c3f-4574-a841-440f8ca629e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:51:55.455553 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.auth.core [None req-25c77a72-9c3f-4574-a841-440f8ca629e4 None None] MFA Rules not processed for user `fd836cf590a145df9acf19cf3ae2ac56`. Rule list: `[]` (Enabled: `True`). {{(pid=88113) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:51:55.456847 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.fernet_utils [None req-25c77a72-9c3f-4574-a841-440f8ca629e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:51:55.475277 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:51:55.524058 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 1200/2401] 10.222.0.13 () {58 vars in 958 bytes} [Wed Aug 31 09:51:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:51:55.533779 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf26fad-bd85-4a1a-9980-13128546f03d None None] REQUEST_METHOD: `POST` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:51:55.534061 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf26fad-bd85-4a1a-9980-13128546f03d None None] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:51:55.534338 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf26fad-bd85-4a1a-9980-13128546f03d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:51:55.555144 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.auth.core [None req-faf26fad-bd85-4a1a-9980-13128546f03d None None] MFA Rules not processed for user `fd836cf590a145df9acf19cf3ae2ac56`. Rule list: `[]` (Enabled: `True`). {{(pid=88114) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:51:55.556367 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.fernet_utils [None req-faf26fad-bd85-4a1a-9980-13128546f03d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88114) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:51:55.564879 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 1202/2402] 10.222.0.13 () {58 vars in 958 bytes} [Wed Aug 31 09:51:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:51:55.569356 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c77a72-9c3f-4574-a841-440f8ca629e4 None None] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:51:55.579062 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c77a72-9c3f-4574-a841-440f8ca629e4 None None] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:51:55.584518 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-357840dc-6184-4b3f-a88a-6442ce4d1ab8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:51:55.585134 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357840dc-6184-4b3f-a88a-6442ce4d1ab8 admin admin] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:51:55.585434 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357840dc-6184-4b3f-a88a-6442ce4d1ab8 admin admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:51:55.585709 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357840dc-6184-4b3f-a88a-6442ce4d1ab8 admin admin] PATH_INFO: `/v3/domains` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:51:55.586438 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-357840dc-6184-4b3f-a88a-6442ce4d1ab8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88113) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:51:55.586741 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-357840dc-6184-4b3f-a88a-6442ce4d1ab8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:51:55.588447 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-357840dc-6184-4b3f-a88a-6442ce4d1ab8 admin admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:51:55.606845 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 1201/2403] 10.222.0.13 () {58 vars in 1153 bytes} [Wed Aug 31 09:51:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:51:55.612065 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf26fad-bd85-4a1a-9980-13128546f03d None None] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:51:55.621422 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf26fad-bd85-4a1a-9980-13128546f03d None None] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:51:55.626946 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6923265-64c7-4632-9774-3a43634e6cfb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:51:55.627545 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6923265-64c7-4632-9774-3a43634e6cfb admin admin] REQUEST_METHOD: `POST` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:51:55.627827 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6923265-64c7-4632-9774-3a43634e6cfb admin admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:51:55.628101 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6923265-64c7-4632-9774-3a43634e6cfb admin admin] PATH_INFO: `/v3/projects` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:51:55.628736 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6923265-64c7-4632-9774-3a43634e6cfb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:51:55.630677 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6923265-64c7-4632-9774-3a43634e6cfb admin admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:51:55.659917 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 1203/2404] 10.222.0.13 () {60 vars in 1153 bytes} [Wed Aug 31 09:51:55 2022] POST /identity/v3/projects => generated 373 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 09:51:55.664294 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-357840dc-6184-4b3f-a88a-6442ce4d1ab8 admin admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:51:55.675386 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-357840dc-6184-4b3f-a88a-6442ce4d1ab8 admin admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:51:55.733405 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa1bfed2-472b-4232-ac37-dcdb50696cd6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:51:55.734144 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1bfed2-472b-4232-ac37-dcdb50696cd6 admin admin] REQUEST_METHOD: `POST` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:51:55.734382 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1bfed2-472b-4232-ac37-dcdb50696cd6 admin admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:51:55.734588 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1bfed2-472b-4232-ac37-dcdb50696cd6 admin admin] PATH_INFO: `/v3/users` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:51:55.735240 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa1bfed2-472b-4232-ac37-dcdb50696cd6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:51:55.737903 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa1bfed2-472b-4232-ac37-dcdb50696cd6 admin admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:51:55.773109 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 1202/2405] 10.222.0.13 () {60 vars in 1144 bytes} [Wed Aug 31 09:51:55 2022] POST /identity/v3/users => generated 287 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 09:51:55.777504 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6923265-64c7-4632-9774-3a43634e6cfb admin admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:51:55.788726 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6923265-64c7-4632-9774-3a43634e6cfb admin admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:51:55.794066 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06531cc8-3f32-4b77-a059-e5ccf8d252f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:51:55.794674 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06531cc8-3f32-4b77-a059-e5ccf8d252f5 admin admin] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:51:55.794996 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06531cc8-3f32-4b77-a059-e5ccf8d252f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:51:55.795216 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06531cc8-3f32-4b77-a059-e5ccf8d252f5 admin admin] PATH_INFO: `/v3/roles` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:51:55.795897 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06531cc8-3f32-4b77-a059-e5ccf8d252f5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:51:55.797486 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06531cc8-3f32-4b77-a059-e5ccf8d252f5 admin admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:51:55.808227 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 1204/2406] 10.222.0.13 () {58 vars in 1122 bytes} [Wed Aug 31 09:51:55 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 09:51:55.812456 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa1bfed2-472b-4232-ac37-dcdb50696cd6 admin admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:51:55.822898 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa1bfed2-472b-4232-ac37-dcdb50696cd6 admin admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:51:55.827934 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f47aa14-744e-405a-9b30-1e27baaae0e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:51:55.828529 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f47aa14-744e-405a-9b30-1e27baaae0e1 admin admin] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:51:55.828809 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f47aa14-744e-405a-9b30-1e27baaae0e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:51:55.829082 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f47aa14-744e-405a-9b30-1e27baaae0e1 admin admin] PATH_INFO: `/v3/roles` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:51:55.830504 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f47aa14-744e-405a-9b30-1e27baaae0e1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:51:55.831906 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f47aa14-744e-405a-9b30-1e27baaae0e1 admin admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:51:55.842880 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 1203/2407] 10.222.0.13 () {58 vars in 1122 bytes} [Wed Aug 31 09:51:55 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 09:51:55.847805 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06531cc8-3f32-4b77-a059-e5ccf8d252f5 admin admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:51:55.857976 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06531cc8-3f32-4b77-a059-e5ccf8d252f5 admin admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:51:55.862670 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10675e2a-20ff-4f4e-9d14-b1ead024b68f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:51:55.863344 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10675e2a-20ff-4f4e-9d14-b1ead024b68f admin admin] REQUEST_METHOD: `PUT` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:51:55.863619 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10675e2a-20ff-4f4e-9d14-b1ead024b68f admin admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:51:55.863888 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10675e2a-20ff-4f4e-9d14-b1ead024b68f admin admin] PATH_INFO: `/v3/projects/bd65e77f9ead40a6aae26c91a2a629fd/users/e36bee012a48479cbc84d96d316bec95/roles/6c99f5d9f1f243eaa134d8540f21095d` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:51:55.878900 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10675e2a-20ff-4f4e-9d14-b1ead024b68f admin admin] RBAC: Authorizing `identity:create_grant(project_id=bd65e77f9ead40a6aae26c91a2a629fd, user_id=e36bee012a48479cbc84d96d316bec95, role_id=6c99f5d9f1f243eaa134d8540f21095d)` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:51:55.881590 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10675e2a-20ff-4f4e-9d14-b1ead024b68f admin admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:51:55.898657 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 1205/2408] 10.222.0.13 () {60 vars in 1483 bytes} [Wed Aug 31 09:51:55 2022] PUT /identity/v3/projects/bd65e77f9ead40a6aae26c91a2a629fd/users/e36bee012a48479cbc84d96d316bec95/roles/6c99f5d9f1f243eaa134d8540f21095d => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:51:55.909915 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b00387-b975-4d16-a138-7bf8adc336b2 None None] REQUEST_METHOD: `POST` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:51:55.910204 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b00387-b975-4d16-a138-7bf8adc336b2 None None] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:51:55.910480 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b00387-b975-4d16-a138-7bf8adc336b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:51:55.940189 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.auth.core [None req-70b00387-b975-4d16-a138-7bf8adc336b2 None None] MFA Rules not processed for user `e36bee012a48479cbc84d96d316bec95`. Rule list: `[]` (Enabled: `True`). {{(pid=88113) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:51:55.941454 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.fernet_utils [None req-70b00387-b975-4d16-a138-7bf8adc336b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:51:56.019534 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 1204/2409] 10.222.0.13 () {58 vars in 958 bytes} [Wed Aug 31 09:51:55 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:51:56.031100 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10675e2a-20ff-4f4e-9d14-b1ead024b68f admin admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:51:56.034024 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.fernet_utils [None req-10675e2a-20ff-4f4e-9d14-b1ead024b68f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88114) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:51:56.062671 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:51:56.102916 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10675e2a-20ff-4f4e-9d14-b1ead024b68f admin admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:51:56.106677 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cef7068-0543-4bd8-aee3-73592e1aac9e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d52b69c8e94441dacfcff944326f169', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '49548eb44d62415582af820ab66decee', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:51:56.107326 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cef7068-0543-4bd8-aee3-73592e1aac9e service cinder] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:51:56.107566 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cef7068-0543-4bd8-aee3-73592e1aac9e service cinder] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:51:56.107837 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cef7068-0543-4bd8-aee3-73592e1aac9e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:51:56.117981 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cef7068-0543-4bd8-aee3-73592e1aac9e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:51:56.119425 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cef7068-0543-4bd8-aee3-73592e1aac9e service cinder] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:51:56.129140 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 1206/2410] 10.222.0.13 () {60 vars in 1365 bytes} [Wed Aug 31 09:51:56 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:51:56.154512 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b00387-b975-4d16-a138-7bf8adc336b2 None None] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:51:56.176088 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:51:56.225603 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b00387-b975-4d16-a138-7bf8adc336b2 None None] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:51:56.230382 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c63171-e6a4-44b3-8be1-7c85599c4b89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:51:56.230962 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c63171-e6a4-44b3-8be1-7c85599c4b89 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:51:56.231239 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c63171-e6a4-44b3-8be1-7c85599c4b89 admin admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:51:56.231503 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c63171-e6a4-44b3-8be1-7c85599c4b89 admin admin] PATH_INFO: `/v3/users/e36bee012a48479cbc84d96d316bec95` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:51:56.232660 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99c63171-e6a4-44b3-8be1-7c85599c4b89 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e36bee012a48479cbc84d96d316bec95)` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:51:56.234668 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99c63171-e6a4-44b3-8be1-7c85599c4b89 admin admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:51:56.295028 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.notifications [None req-99c63171-e6a4-44b3-8be1-7c85599c4b89 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e36bee012a48479cbc84d96d316bec95'} {{(pid=88113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:51:56.304185 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.notifications [None req-99c63171-e6a4-44b3-8be1-7c85599c4b89 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e36bee012a48479cbc84d96d316bec95'} {{(pid=88113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:51:56.317465 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.notifications [None req-99c63171-e6a4-44b3-8be1-7c85599c4b89 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e36bee012a48479cbc84d96d316bec95'} {{(pid=88113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:51:56.318209 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.notifications [None req-99c63171-e6a4-44b3-8be1-7c85599c4b89 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e36bee012a48479cbc84d96d316bec95'} {{(pid=88113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:51:56.327021 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.notifications [None req-99c63171-e6a4-44b3-8be1-7c85599c4b89 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e36bee012a48479cbc84d96d316bec95'} {{(pid=88113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:51:56.334070 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 1205/2411] 10.222.0.13 () {58 vars in 1224 bytes} [Wed Aug 31 09:51:56 2022] DELETE /identity/v3/users/e36bee012a48479cbc84d96d316bec95 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:51:56.340270 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32cbf3e0-c2b2-486f-bba5-8f396545481e None None] REQUEST_METHOD: `POST` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:51:56.340877 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32cbf3e0-c2b2-486f-bba5-8f396545481e None None] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:51:56.341579 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32cbf3e0-c2b2-486f-bba5-8f396545481e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:51:56.379477 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.auth.core [None req-32cbf3e0-c2b2-486f-bba5-8f396545481e None None] MFA Rules not processed for user `fd836cf590a145df9acf19cf3ae2ac56`. Rule list: `[]` (Enabled: `True`). {{(pid=88114) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:51:56.381485 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.fernet_utils [None req-32cbf3e0-c2b2-486f-bba5-8f396545481e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88114) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:51:56.453098 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 1207/2412] 10.222.0.13 () {58 vars in 958 bytes} [Wed Aug 31 09:51:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:51:56.463928 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c63171-e6a4-44b3-8be1-7c85599c4b89 admin admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:51:56.465500 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.fernet_utils [None req-99c63171-e6a4-44b3-8be1-7c85599c4b89 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:51:56.558519 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c63171-e6a4-44b3-8be1-7c85599c4b89 admin admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:51:56.562734 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fde99f2-83a3-4fe4-a1ce-c68c7d7ed06e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9bdaacd597442c2bcde2c555f2cc140', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '49548eb44d62415582af820ab66decee', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:51:56.563358 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fde99f2-83a3-4fe4-a1ce-c68c7d7ed06e service neutron] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:51:56.563684 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fde99f2-83a3-4fe4-a1ce-c68c7d7ed06e service neutron] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:51:56.564419 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fde99f2-83a3-4fe4-a1ce-c68c7d7ed06e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:51:56.574896 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fde99f2-83a3-4fe4-a1ce-c68c7d7ed06e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:51:56.577128 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fde99f2-83a3-4fe4-a1ce-c68c7d7ed06e service neutron] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:51:56.591755 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 1206/2413] 10.222.0.13 () {60 vars in 1365 bytes} [Wed Aug 31 09:51:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:51:56.889223 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32cbf3e0-c2b2-486f-bba5-8f396545481e None None] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:51:56.890936 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.fernet_utils [None req-32cbf3e0-c2b2-486f-bba5-8f396545481e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88114) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:51:56.904257 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32cbf3e0-c2b2-486f-bba5-8f396545481e None None] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:51:56.909275 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d8e2b2c-108a-4369-8f41-8e6ad05a5c5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:51:56.910032 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8e2b2c-108a-4369-8f41-8e6ad05a5c5b admin admin] REQUEST_METHOD: `DELETE` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:51:56.910385 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8e2b2c-108a-4369-8f41-8e6ad05a5c5b admin admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:51:56.911274 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8e2b2c-108a-4369-8f41-8e6ad05a5c5b admin admin] PATH_INFO: `/v3/projects/bd65e77f9ead40a6aae26c91a2a629fd` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:51:56.912047 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d8e2b2c-108a-4369-8f41-8e6ad05a5c5b admin admin] RBAC: Authorizing `identity:delete_project(project_id=bd65e77f9ead40a6aae26c91a2a629fd)` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:51:56.921239 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d8e2b2c-108a-4369-8f41-8e6ad05a5c5b admin admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:51:56.978799 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.notifications [None req-3d8e2b2c-108a-4369-8f41-8e6ad05a5c5b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bd65e77f9ead40a6aae26c91a2a629fd', 'request_id': 'req-3d8e2b2c-108a-4369-8f41-8e6ad05a5c5b'} {{(pid=88114) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:51:56.990775 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.notifications [None req-3d8e2b2c-108a-4369-8f41-8e6ad05a5c5b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bd65e77f9ead40a6aae26c91a2a629fd', 'request_id': 'req-3d8e2b2c-108a-4369-8f41-8e6ad05a5c5b'} {{(pid=88114) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:51:56.997309 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.notifications [None req-3d8e2b2c-108a-4369-8f41-8e6ad05a5c5b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bd65e77f9ead40a6aae26c91a2a629fd', 'request_id': 'req-3d8e2b2c-108a-4369-8f41-8e6ad05a5c5b'} {{(pid=88114) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:51:57.012127 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.notifications [None req-3d8e2b2c-108a-4369-8f41-8e6ad05a5c5b admin admin] The token cache is being invalidate because project bd65e77f9ead40a6aae26c91a2a629fd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88114) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 09:51:57.012412 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.notifications [None req-3d8e2b2c-108a-4369-8f41-8e6ad05a5c5b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88114) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:51:57.013034 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.notifications [None req-3d8e2b2c-108a-4369-8f41-8e6ad05a5c5b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88114) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:51:57.014785 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 1208/2414] 10.222.0.13 () {58 vars in 1233 bytes} [Wed Aug 31 09:51:56 2022] DELETE /identity/v3/projects/bd65e77f9ead40a6aae26c91a2a629fd => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:51:57.020702 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf52460-2c85-4266-8b84-029082e1808a None None] REQUEST_METHOD: `POST` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:51:57.021214 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf52460-2c85-4266-8b84-029082e1808a None None] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:51:57.021336 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf52460-2c85-4266-8b84-029082e1808a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:51:57.041962 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.auth.core [None req-2bf52460-2c85-4266-8b84-029082e1808a None None] MFA Rules not processed for user `fd836cf590a145df9acf19cf3ae2ac56`. Rule list: `[]` (Enabled: `True`). {{(pid=88113) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:51:57.043192 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.fernet_utils [None req-2bf52460-2c85-4266-8b84-029082e1808a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:51:57.079724 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:51:57.110075 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 1207/2415] 10.222.0.13 () {58 vars in 958 bytes} [Wed Aug 31 09:51:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:51:57.119461 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69dba1ea-c0e0-4184-b416-68bf1cda7733 None None] REQUEST_METHOD: `POST` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:51:57.119751 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69dba1ea-c0e0-4184-b416-68bf1cda7733 None None] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:51:57.120029 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69dba1ea-c0e0-4184-b416-68bf1cda7733 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:51:57.146709 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.auth.core [None req-69dba1ea-c0e0-4184-b416-68bf1cda7733 None None] MFA Rules not processed for user `fd836cf590a145df9acf19cf3ae2ac56`. Rule list: `[]` (Enabled: `True`). {{(pid=88114) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:51:57.147931 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.fernet_utils [None req-69dba1ea-c0e0-4184-b416-68bf1cda7733 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88114) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:51:57.155924 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 1209/2416] 10.222.0.13 () {58 vars in 958 bytes} [Wed Aug 31 09:51:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:51:57.161141 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf52460-2c85-4266-8b84-029082e1808a None None] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:51:57.171203 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf52460-2c85-4266-8b84-029082e1808a None None] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:51:57.176242 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf9d255d-a2db-4aa6-ad03-e21f617aab3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:51:57.176856 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9d255d-a2db-4aa6-ad03-e21f617aab3c admin admin] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:51:57.177132 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9d255d-a2db-4aa6-ad03-e21f617aab3c admin admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:51:57.177426 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9d255d-a2db-4aa6-ad03-e21f617aab3c admin admin] PATH_INFO: `/v3/domains` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:51:57.178092 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf9d255d-a2db-4aa6-ad03-e21f617aab3c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88113) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:51:57.178393 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf9d255d-a2db-4aa6-ad03-e21f617aab3c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:51:57.179997 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf9d255d-a2db-4aa6-ad03-e21f617aab3c admin admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:51:57.192508 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 1208/2417] 10.222.0.13 () {58 vars in 1153 bytes} [Wed Aug 31 09:51:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:51:57.202972 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69dba1ea-c0e0-4184-b416-68bf1cda7733 None None] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:51:57.213060 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69dba1ea-c0e0-4184-b416-68bf1cda7733 None None] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:51:57.217961 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda6fe82-9f1d-4ca1-aa0c-435617b4fc80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:51:57.218581 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda6fe82-9f1d-4ca1-aa0c-435617b4fc80 admin admin] REQUEST_METHOD: `POST` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:51:57.218859 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda6fe82-9f1d-4ca1-aa0c-435617b4fc80 admin admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:51:57.219125 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda6fe82-9f1d-4ca1-aa0c-435617b4fc80 admin admin] PATH_INFO: `/v3/projects` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:51:57.219755 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bda6fe82-9f1d-4ca1-aa0c-435617b4fc80 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:51:57.222459 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bda6fe82-9f1d-4ca1-aa0c-435617b4fc80 admin admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:51:57.252606 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 1210/2418] 10.222.0.13 () {60 vars in 1153 bytes} [Wed Aug 31 09:51:57 2022] POST /identity/v3/projects => generated 381 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 09:51:57.257089 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf9d255d-a2db-4aa6-ad03-e21f617aab3c admin admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:51:57.271888 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf9d255d-a2db-4aa6-ad03-e21f617aab3c admin admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:51:57.335441 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8858443-2989-4e16-85b0-ab40a2055b92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:51:57.336047 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8858443-2989-4e16-85b0-ab40a2055b92 admin admin] REQUEST_METHOD: `POST` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:51:57.336324 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8858443-2989-4e16-85b0-ab40a2055b92 admin admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:51:57.336597 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8858443-2989-4e16-85b0-ab40a2055b92 admin admin] PATH_INFO: `/v3/users` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:51:57.337236 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8858443-2989-4e16-85b0-ab40a2055b92 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:51:57.339233 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8858443-2989-4e16-85b0-ab40a2055b92 admin admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:51:57.361051 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 1209/2419] 10.222.0.13 () {60 vars in 1144 bytes} [Wed Aug 31 09:51:57 2022] POST /identity/v3/users => generated 291 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 09:51:57.365661 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda6fe82-9f1d-4ca1-aa0c-435617b4fc80 admin admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:51:57.376604 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda6fe82-9f1d-4ca1-aa0c-435617b4fc80 admin admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:51:57.381210 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ac5169-a761-4b3f-a948-6149eae0143e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:51:57.381833 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ac5169-a761-4b3f-a948-6149eae0143e admin admin] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:51:57.382109 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ac5169-a761-4b3f-a948-6149eae0143e admin admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:51:57.382372 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ac5169-a761-4b3f-a948-6149eae0143e admin admin] PATH_INFO: `/v3/roles` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:51:57.383009 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30ac5169-a761-4b3f-a948-6149eae0143e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:51:57.384589 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30ac5169-a761-4b3f-a948-6149eae0143e admin admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:51:57.395673 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 1211/2420] 10.222.0.13 () {58 vars in 1122 bytes} [Wed Aug 31 09:51:57 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 09:51:57.400079 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8858443-2989-4e16-85b0-ab40a2055b92 admin admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:51:57.412206 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8858443-2989-4e16-85b0-ab40a2055b92 admin admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:51:57.417051 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce05d54-b7f3-47f3-a5dc-9eef2bd6329c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:51:57.417676 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce05d54-b7f3-47f3-a5dc-9eef2bd6329c admin admin] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:51:57.417951 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce05d54-b7f3-47f3-a5dc-9eef2bd6329c admin admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:51:57.418219 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce05d54-b7f3-47f3-a5dc-9eef2bd6329c admin admin] PATH_INFO: `/v3/roles` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:51:57.418849 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ce05d54-b7f3-47f3-a5dc-9eef2bd6329c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:51:57.420452 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ce05d54-b7f3-47f3-a5dc-9eef2bd6329c admin admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:51:57.432613 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 1210/2421] 10.222.0.13 () {58 vars in 1122 bytes} [Wed Aug 31 09:51:57 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 09:51:57.436889 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ac5169-a761-4b3f-a948-6149eae0143e admin admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:51:57.450080 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ac5169-a761-4b3f-a948-6149eae0143e admin admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:51:57.456289 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ba8239-2048-466f-b115-f7786c419fe8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:51:57.457173 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ba8239-2048-466f-b115-f7786c419fe8 admin admin] REQUEST_METHOD: `PUT` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:51:57.457486 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ba8239-2048-466f-b115-f7786c419fe8 admin admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:51:57.457755 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ba8239-2048-466f-b115-f7786c419fe8 admin admin] PATH_INFO: `/v3/projects/f8336bd8ac0c4600a25261e1af19a7df/users/f62009b582dd480198ceca6f41e255c2/roles/6c99f5d9f1f243eaa134d8540f21095d` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:51:57.460842 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:51:57.478564 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32ba8239-2048-466f-b115-f7786c419fe8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f8336bd8ac0c4600a25261e1af19a7df, user_id=f62009b582dd480198ceca6f41e255c2, role_id=6c99f5d9f1f243eaa134d8540f21095d)` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:51:57.481225 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32ba8239-2048-466f-b115-f7786c419fe8 admin admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:51:57.495973 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 1212/2422] 10.222.0.13 () {60 vars in 1483 bytes} [Wed Aug 31 09:51:57 2022] PUT /identity/v3/projects/f8336bd8ac0c4600a25261e1af19a7df/users/f62009b582dd480198ceca6f41e255c2/roles/6c99f5d9f1f243eaa134d8540f21095d => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:51:57.501484 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9851782-38e5-4b89-a858-3936f0fe67d7 None None] REQUEST_METHOD: `POST` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:51:57.501836 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9851782-38e5-4b89-a858-3936f0fe67d7 None None] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:51:57.502033 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9851782-38e5-4b89-a858-3936f0fe67d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:51:57.522295 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.auth.core [None req-b9851782-38e5-4b89-a858-3936f0fe67d7 None None] MFA Rules not processed for user `fd836cf590a145df9acf19cf3ae2ac56`. Rule list: `[]` (Enabled: `True`). {{(pid=88113) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:51:57.523371 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.fernet_utils [None req-b9851782-38e5-4b89-a858-3936f0fe67d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:51:57.609561 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 1211/2423] 10.222.0.13 () {58 vars in 958 bytes} [Wed Aug 31 09:51:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:51:57.620136 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ba8239-2048-466f-b115-f7786c419fe8 admin admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:51:57.622400 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.fernet_utils [None req-32ba8239-2048-466f-b115-f7786c419fe8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88114) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:51:57.692685 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ba8239-2048-466f-b115-f7786c419fe8 admin admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:51:57.696966 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81fbb89e-a622-4c19-b386-b914641e32ee service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9bdaacd597442c2bcde2c555f2cc140', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '49548eb44d62415582af820ab66decee', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:51:57.697623 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81fbb89e-a622-4c19-b386-b914641e32ee service neutron] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:51:57.697902 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81fbb89e-a622-4c19-b386-b914641e32ee service neutron] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:51:57.698170 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81fbb89e-a622-4c19-b386-b914641e32ee service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:51:57.708796 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81fbb89e-a622-4c19-b386-b914641e32ee service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:51:57.710289 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81fbb89e-a622-4c19-b386-b914641e32ee service neutron] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:51:57.721701 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 1213/2424] 10.222.0.13 () {60 vars in 1365 bytes} [Wed Aug 31 09:51:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:51:58.509687 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693d8ca2-2ab0-41f6-bb25-9ff3aa51b7dc None None] REQUEST_METHOD: `POST` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:51:58.510227 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693d8ca2-2ab0-41f6-bb25-9ff3aa51b7dc None None] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:51:58.510227 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693d8ca2-2ab0-41f6-bb25-9ff3aa51b7dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:51:58.550719 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.auth.core [None req-693d8ca2-2ab0-41f6-bb25-9ff3aa51b7dc None None] MFA Rules not processed for user `f62009b582dd480198ceca6f41e255c2`. Rule list: `[]` (Enabled: `True`). {{(pid=88113) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:51:58.551982 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.fernet_utils [None req-693d8ca2-2ab0-41f6-bb25-9ff3aa51b7dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:51:58.608742 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:51:58.672977 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 1212/2425] 10.222.0.13 () {58 vars in 958 bytes} [Wed Aug 31 09:51:58 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:51:58.690950 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81fbb89e-a622-4c19-b386-b914641e32ee service neutron] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:51:58.693261 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.fernet_utils [None req-81fbb89e-a622-4c19-b386-b914641e32ee service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88114) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:51:58.762927 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81fbb89e-a622-4c19-b386-b914641e32ee service neutron] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:51:58.770353 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be9d42d-86c3-489a-bc60-4eedf42f0f61 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d52b69c8e94441dacfcff944326f169', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '49548eb44d62415582af820ab66decee', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:51:58.773295 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be9d42d-86c3-489a-bc60-4eedf42f0f61 service cinder] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:51:58.774859 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be9d42d-86c3-489a-bc60-4eedf42f0f61 service cinder] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:51:58.775440 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be9d42d-86c3-489a-bc60-4eedf42f0f61 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:51:58.800676 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4be9d42d-86c3-489a-bc60-4eedf42f0f61 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:51:58.803735 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4be9d42d-86c3-489a-bc60-4eedf42f0f61 service cinder] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:51:58.816574 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 1214/2426] 10.222.0.13 () {60 vars in 1365 bytes} [Wed Aug 31 09:51:58 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:52:01.395473 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec36781-b113-4607-b95f-b6a9069d9009 None None] REQUEST_METHOD: `POST` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:52:01.395999 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec36781-b113-4607-b95f-b6a9069d9009 None None] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:52:01.396120 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec36781-b113-4607-b95f-b6a9069d9009 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:52:01.416048 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.auth.core [None req-fec36781-b113-4607-b95f-b6a9069d9009 None None] MFA Rules not processed for user `f62009b582dd480198ceca6f41e255c2`. Rule list: `[]` (Enabled: `True`). {{(pid=88113) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:52:01.417282 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.fernet_utils [None req-fec36781-b113-4607-b95f-b6a9069d9009 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:52:01.425182 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 1213/2427] 10.222.0.13 () {58 vars in 958 bytes} [Wed Aug 31 09:52:01 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:52:08.811251 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=59,keystone:INSERT=1 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:52:11.410342 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=14 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:52:16.837962 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def5cc90-15e0-4927-9345-e8c5341c6b1c None None] REQUEST_METHOD: `POST` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:52:16.838540 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def5cc90-15e0-4927-9345-e8c5341c6b1c None None] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:52:16.838540 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def5cc90-15e0-4927-9345-e8c5341c6b1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:52:16.865982 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.auth.core [None req-def5cc90-15e0-4927-9345-e8c5341c6b1c None None] MFA Rules not processed for user `1090c027746845239055973b44207556`. Rule list: `[]` (Enabled: `True`). {{(pid=88114) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:52:16.868030 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.fernet_utils [None req-def5cc90-15e0-4927-9345-e8c5341c6b1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88114) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:52:16.938369 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 1215/2428] 10.222.0.13 () {58 vars in 958 bytes} [Wed Aug 31 09:52:16 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:52:26.928735 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=25 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:52:35.811630 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fec36781-b113-4607-b95f-b6a9069d9009 None None] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:52:35.827704 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fec36781-b113-4607-b95f-b6a9069d9009 None None] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:52:35.831916 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba21323-05d6-41cf-9f6f-bd026170971a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9bdaacd597442c2bcde2c555f2cc140', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '49548eb44d62415582af820ab66decee', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:52:35.832559 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba21323-05d6-41cf-9f6f-bd026170971a service neutron] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:52:35.832837 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba21323-05d6-41cf-9f6f-bd026170971a service neutron] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:52:35.833108 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba21323-05d6-41cf-9f6f-bd026170971a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:52:35.834940 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.fernet_utils [None req-bba21323-05d6-41cf-9f6f-bd026170971a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:52:35.916485 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bba21323-05d6-41cf-9f6f-bd026170971a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:52:35.918007 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bba21323-05d6-41cf-9f6f-bd026170971a service neutron] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:52:35.929235 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 1214/2429] 10.222.0.13 () {60 vars in 1365 bytes} [Wed Aug 31 09:52:35 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:52:43.028914 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def5cc90-15e0-4927-9345-e8c5341c6b1c None None] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:52:43.031751 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.fernet_utils [None req-def5cc90-15e0-4927-9345-e8c5341c6b1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88114) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:52:43.055168 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def5cc90-15e0-4927-9345-e8c5341c6b1c None None] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:52:43.061726 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4419a98-5b5b-47dc-923b-bd5bc2f627de service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9bdaacd597442c2bcde2c555f2cc140', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '49548eb44d62415582af820ab66decee', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:52:43.063614 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4419a98-5b5b-47dc-923b-bd5bc2f627de service neutron] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:52:43.064106 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4419a98-5b5b-47dc-923b-bd5bc2f627de service neutron] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:52:43.064559 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4419a98-5b5b-47dc-923b-bd5bc2f627de service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:52:43.067683 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.fernet_utils [None req-b4419a98-5b5b-47dc-923b-bd5bc2f627de service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88114) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:52:43.147397 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4419a98-5b5b-47dc-923b-bd5bc2f627de service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:52:43.148850 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4419a98-5b5b-47dc-923b-bd5bc2f627de service neutron] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:52:43.159934 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 1216/2430] 10.222.0.13 () {60 vars in 1365 bytes} [Wed Aug 31 09:52:43 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:52:45.925164 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=30 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:52:46.933236 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba21323-05d6-41cf-9f6f-bd026170971a service neutron] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:52:46.935164 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.fernet_utils [None req-bba21323-05d6-41cf-9f6f-bd026170971a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:52:46.948804 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba21323-05d6-41cf-9f6f-bd026170971a service neutron] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:52:46.955249 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7008113-85c0-46ea-90c7-808312206776 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d52b69c8e94441dacfcff944326f169', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '49548eb44d62415582af820ab66decee', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:52:46.956370 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7008113-85c0-46ea-90c7-808312206776 service cinder] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:52:46.956760 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7008113-85c0-46ea-90c7-808312206776 service cinder] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:52:46.957171 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7008113-85c0-46ea-90c7-808312206776 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:52:46.960127 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.fernet_utils [None req-d7008113-85c0-46ea-90c7-808312206776 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:52:47.074863 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7008113-85c0-46ea-90c7-808312206776 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:52:47.076572 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7008113-85c0-46ea-90c7-808312206776 service cinder] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:52:47.091684 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 1215/2431] 10.222.0.13 () {60 vars in 1365 bytes} [Wed Aug 31 09:52:46 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 159 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:52:47.231427 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42591169-02ed-45da-9a7f-5e08b9e28193 None None] REQUEST_METHOD: `POST` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:52:47.231999 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42591169-02ed-45da-9a7f-5e08b9e28193 None None] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:52:47.232121 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42591169-02ed-45da-9a7f-5e08b9e28193 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:52:47.255334 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.auth.core [None req-42591169-02ed-45da-9a7f-5e08b9e28193 None None] MFA Rules not processed for user `25b4233e5f0e41bf99b3fb3f5f6229c9`. Rule list: `[]` (Enabled: `True`). {{(pid=88114) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:52:47.256564 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.fernet_utils [None req-42591169-02ed-45da-9a7f-5e08b9e28193 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88114) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:52:47.331306 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 1217/2432] 10.222.0.13 () {58 vars in 958 bytes} [Wed Aug 31 09:52:47 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 09:52:47.378698 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7008113-85c0-46ea-90c7-808312206776 service cinder] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:52:47.380804 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.fernet_utils [None req-d7008113-85c0-46ea-90c7-808312206776 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:52:47.471697 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7008113-85c0-46ea-90c7-808312206776 service cinder] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:52:47.476325 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d6eea2-2fff-4cbd-aca4-969df2d9f154 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be216d156b54452cbbf978c7734343aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '49548eb44d62415582af820ab66decee', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:52:47.476996 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d6eea2-2fff-4cbd-aca4-969df2d9f154 service glance] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:52:47.477315 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d6eea2-2fff-4cbd-aca4-969df2d9f154 service glance] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:52:47.477710 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d6eea2-2fff-4cbd-aca4-969df2d9f154 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:52:47.479663 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.fernet_utils [None req-22d6eea2-2fff-4cbd-aca4-969df2d9f154 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:52:47.496113 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22d6eea2-2fff-4cbd-aca4-969df2d9f154 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:52:47.497617 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22d6eea2-2fff-4cbd-aca4-969df2d9f154 service glance] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:52:47.511286 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 1216/2433] 10.222.0.13 () {60 vars in 1365 bytes} [Wed Aug 31 09:52:47 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:52:57.323539 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=55 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:52:57.506270 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=60 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:53:16.973249 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42591169-02ed-45da-9a7f-5e08b9e28193 None None] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:53:16.985471 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42591169-02ed-45da-9a7f-5e08b9e28193 None None] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:53:16.989749 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-320376af-2d54-43cc-b6be-ce3959d60466 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9bdaacd597442c2bcde2c555f2cc140', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '49548eb44d62415582af820ab66decee', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:53:16.990351 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320376af-2d54-43cc-b6be-ce3959d60466 service neutron] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:53:16.990622 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320376af-2d54-43cc-b6be-ce3959d60466 service neutron] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:53:16.990889 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320376af-2d54-43cc-b6be-ce3959d60466 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:53:16.992643 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.fernet_utils [None req-320376af-2d54-43cc-b6be-ce3959d60466 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88114) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:53:17.002848 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-320376af-2d54-43cc-b6be-ce3959d60466 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:53:17.004375 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-320376af-2d54-43cc-b6be-ce3959d60466 service neutron] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:53:17.012280 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 1218/2434] 10.222.0.13 () {60 vars in 1365 bytes} [Wed Aug 31 09:53:16 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:53:27.008828 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=6 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:53:27.734484 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d6eea2-2fff-4cbd-aca4-969df2d9f154 service glance] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:53:27.736105 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.fernet_utils [None req-22d6eea2-2fff-4cbd-aca4-969df2d9f154 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:53:27.751349 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d6eea2-2fff-4cbd-aca4-969df2d9f154 service glance] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:53:27.758157 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd1f5ace-25a9-41bb-8d0b-4ca735d87df4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc076493239d4ccdbdc37c0369c49529', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '49548eb44d62415582af820ab66decee', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:53:27.759164 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1f5ace-25a9-41bb-8d0b-4ca735d87df4 service nova] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:53:27.759582 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1f5ace-25a9-41bb-8d0b-4ca735d87df4 service nova] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:53:27.760175 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1f5ace-25a9-41bb-8d0b-4ca735d87df4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:53:27.762973 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.fernet_utils [None req-fd1f5ace-25a9-41bb-8d0b-4ca735d87df4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:53:27.777879 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd1f5ace-25a9-41bb-8d0b-4ca735d87df4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:53:27.779519 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd1f5ace-25a9-41bb-8d0b-4ca735d87df4 service nova] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:53:27.793230 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 1217/2435] 10.222.0.13 () {60 vars in 1365 bytes} [Wed Aug 31 09:53:27 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 59 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:53:35.655859 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-320376af-2d54-43cc-b6be-ce3959d60466 service neutron] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:53:35.672189 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-320376af-2d54-43cc-b6be-ce3959d60466 service neutron] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:53:35.680477 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca2a85b-aad5-41d7-9b39-8c3e75cb5235 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc076493239d4ccdbdc37c0369c49529', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '49548eb44d62415582af820ab66decee', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:53:35.681119 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca2a85b-aad5-41d7-9b39-8c3e75cb5235 service nova] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:53:35.681453 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca2a85b-aad5-41d7-9b39-8c3e75cb5235 service nova] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:53:35.681727 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca2a85b-aad5-41d7-9b39-8c3e75cb5235 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:53:35.683476 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.fernet_utils [None req-fca2a85b-aad5-41d7-9b39-8c3e75cb5235 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88114) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:53:35.752137 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fca2a85b-aad5-41d7-9b39-8c3e75cb5235 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:53:35.753947 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fca2a85b-aad5-41d7-9b39-8c3e75cb5235 service nova] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:53:35.764234 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 1219/2436] 10.222.0.13 () {60 vars in 1365 bytes} [Wed Aug 31 09:53:35 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:53:37.788102 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=6 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:53:40.836168 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3cd7e2-8402-4a8b-84ba-f8dc48ca2797 None None] REQUEST_METHOD: `POST` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:53:40.836765 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3cd7e2-8402-4a8b-84ba-f8dc48ca2797 None None] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:53:40.836765 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3cd7e2-8402-4a8b-84ba-f8dc48ca2797 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:53:40.857401 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.auth.core [None req-cc3cd7e2-8402-4a8b-84ba-f8dc48ca2797 None None] MFA Rules not processed for user `f62009b582dd480198ceca6f41e255c2`. Rule list: `[]` (Enabled: `True`). {{(pid=88113) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:53:40.858751 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.fernet_utils [None req-cc3cd7e2-8402-4a8b-84ba-f8dc48ca2797 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:53:40.866762 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 1218/2437] 10.222.0.13 () {58 vars in 958 bytes} [Wed Aug 31 09:53:40 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:53:41.116523 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca2a85b-aad5-41d7-9b39-8c3e75cb5235 service nova] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:53:41.131386 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca2a85b-aad5-41d7-9b39-8c3e75cb5235 service nova] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:53:41.137658 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d297ac6c-2cd1-439d-8412-97a18df77534 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9bdaacd597442c2bcde2c555f2cc140', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '49548eb44d62415582af820ab66decee', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:53:41.138735 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d297ac6c-2cd1-439d-8412-97a18df77534 service neutron] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:53:41.139230 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d297ac6c-2cd1-439d-8412-97a18df77534 service neutron] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:53:41.139684 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d297ac6c-2cd1-439d-8412-97a18df77534 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:53:41.142747 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.fernet_utils [None req-d297ac6c-2cd1-439d-8412-97a18df77534 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88114) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:53:41.158795 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d297ac6c-2cd1-439d-8412-97a18df77534 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:53:41.160849 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d297ac6c-2cd1-439d-8412-97a18df77534 service neutron] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:53:41.169635 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 1220/2438] 10.222.0.13 () {60 vars in 1365 bytes} [Wed Aug 31 09:53:41 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:53:41.520590 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc3cd7e2-8402-4a8b-84ba-f8dc48ca2797 None None] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:53:41.522988 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.fernet_utils [None req-cc3cd7e2-8402-4a8b-84ba-f8dc48ca2797 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:53:41.616444 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc3cd7e2-8402-4a8b-84ba-f8dc48ca2797 None None] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:53:41.622052 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04485e10-8320-4bdb-b249-90e8b9fa5ec0 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ff3aff5ef444910be6dbb989d0e3194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '49548eb44d62415582af820ab66decee', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:53:41.622695 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04485e10-8320-4bdb-b249-90e8b9fa5ec0 service placement] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:53:41.622969 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04485e10-8320-4bdb-b249-90e8b9fa5ec0 service placement] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:53:41.623239 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04485e10-8320-4bdb-b249-90e8b9fa5ec0 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:53:41.625031 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.fernet_utils [None req-04485e10-8320-4bdb-b249-90e8b9fa5ec0 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:53:41.642167 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04485e10-8320-4bdb-b249-90e8b9fa5ec0 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:53:41.643995 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04485e10-8320-4bdb-b249-90e8b9fa5ec0 service placement] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:53:41.655705 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 1219/2439] 10.222.0.13 () {60 vars in 1365 bytes} [Wed Aug 31 09:53:41 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:53:51.165881 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=34 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:53:51.650868 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=35 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:53:52.415478 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d297ac6c-2cd1-439d-8412-97a18df77534 service neutron] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:53:52.416934 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.fernet_utils [None req-d297ac6c-2cd1-439d-8412-97a18df77534 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88114) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:53:52.434571 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d297ac6c-2cd1-439d-8412-97a18df77534 service neutron] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:53:52.440422 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed9a654-f705-459b-9328-167e78eba8eb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc076493239d4ccdbdc37c0369c49529', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '49548eb44d62415582af820ab66decee', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:53:52.441030 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed9a654-f705-459b-9328-167e78eba8eb service nova] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:53:52.441303 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed9a654-f705-459b-9328-167e78eba8eb service nova] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:53:52.441602 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed9a654-f705-459b-9328-167e78eba8eb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:53:52.443304 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.fernet_utils [None req-1ed9a654-f705-459b-9328-167e78eba8eb service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88114) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:53:52.456199 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ed9a654-f705-459b-9328-167e78eba8eb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:53:52.457798 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ed9a654-f705-459b-9328-167e78eba8eb service nova] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:53:52.468672 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 1221/2440] 10.222.0.13 () {60 vars in 1365 bytes} [Wed Aug 31 09:53:52 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:54:02.464106 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=6 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:54:22.576360 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3dc9ac-349c-48ba-8e66-a4b4fc8f6689 None None] REQUEST_METHOD: `POST` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:54:22.576950 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3dc9ac-349c-48ba-8e66-a4b4fc8f6689 None None] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:54:22.577113 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3dc9ac-349c-48ba-8e66-a4b4fc8f6689 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:54:22.598864 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.auth.core [None req-0a3dc9ac-349c-48ba-8e66-a4b4fc8f6689 None None] MFA Rules not processed for user `886e40cf192d46c2b2e7eb2a95ec62e1`. Rule list: `[]` (Enabled: `True`). {{(pid=88113) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:54:22.600092 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.fernet_utils [None req-0a3dc9ac-349c-48ba-8e66-a4b4fc8f6689 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:54:22.607582 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 1220/2441] 10.222.0.13 () {58 vars in 958 bytes} [Wed Aug 31 09:54:22 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:54:29.392668 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed9a654-f705-459b-9328-167e78eba8eb service nova] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:54:29.407955 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed9a654-f705-459b-9328-167e78eba8eb service nova] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:54:29.415511 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ed5145-f3f0-4bd2-8d8e-ae00a90c6b68 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc076493239d4ccdbdc37c0369c49529', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '49548eb44d62415582af820ab66decee', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:54:29.416414 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ed5145-f3f0-4bd2-8d8e-ae00a90c6b68 service nova] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:54:29.416810 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ed5145-f3f0-4bd2-8d8e-ae00a90c6b68 service nova] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:54:29.417197 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ed5145-f3f0-4bd2-8d8e-ae00a90c6b68 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:54:29.419271 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.fernet_utils [None req-71ed5145-f3f0-4bd2-8d8e-ae00a90c6b68 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88114) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:54:29.523158 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71ed5145-f3f0-4bd2-8d8e-ae00a90c6b68 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:54:29.525351 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71ed5145-f3f0-4bd2-8d8e-ae00a90c6b68 service nova] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:54:29.539111 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 1222/2442] 10.222.0.13 () {60 vars in 1365 bytes} [Wed Aug 31 09:54:29 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:54:31.756251 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a3dc9ac-349c-48ba-8e66-a4b4fc8f6689 None None] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:54:31.772418 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a3dc9ac-349c-48ba-8e66-a4b4fc8f6689 None None] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:54:31.778356 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dddc3e08-a67c-4f7c-aeab-ecce1f756eef service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc076493239d4ccdbdc37c0369c49529', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '49548eb44d62415582af820ab66decee', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:54:31.779020 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dddc3e08-a67c-4f7c-aeab-ecce1f756eef service nova] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:54:31.779298 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dddc3e08-a67c-4f7c-aeab-ecce1f756eef service nova] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:54:31.779569 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dddc3e08-a67c-4f7c-aeab-ecce1f756eef service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:54:31.781555 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.fernet_utils [None req-dddc3e08-a67c-4f7c-aeab-ecce1f756eef service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:54:31.796868 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dddc3e08-a67c-4f7c-aeab-ecce1f756eef service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:54:31.798941 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dddc3e08-a67c-4f7c-aeab-ecce1f756eef service nova] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:54:31.814333 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 1221/2443] 10.222.0.13 () {60 vars in 1365 bytes} [Wed Aug 31 09:54:31 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 58 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:54:39.532130 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=28 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:54:41.809268 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=9 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:55:00.810469 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ed5145-f3f0-4bd2-8d8e-ae00a90c6b68 service nova] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:55:00.824065 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ed5145-f3f0-4bd2-8d8e-ae00a90c6b68 service nova] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:55:00.829626 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af1ac31f-5920-4b60-9919-f426dff4b1c6 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ff3aff5ef444910be6dbb989d0e3194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '49548eb44d62415582af820ab66decee', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:55:00.830299 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1ac31f-5920-4b60-9919-f426dff4b1c6 service placement] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:55:00.830657 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1ac31f-5920-4b60-9919-f426dff4b1c6 service placement] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:55:00.830903 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1ac31f-5920-4b60-9919-f426dff4b1c6 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:55:00.832743 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.fernet_utils [None req-af1ac31f-5920-4b60-9919-f426dff4b1c6 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88114) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:55:00.846211 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af1ac31f-5920-4b60-9919-f426dff4b1c6 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:55:00.847854 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af1ac31f-5920-4b60-9919-f426dff4b1c6 service placement] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:55:00.859640 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 1223/2444] 10.222.0.13 () {60 vars in 1365 bytes} [Wed Aug 31 09:55:00 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:55:00.892219 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dddc3e08-a67c-4f7c-aeab-ecce1f756eef service nova] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:55:00.903781 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dddc3e08-a67c-4f7c-aeab-ecce1f756eef service nova] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:55:00.908779 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4267f1c5-2039-49c6-a07c-905eb4543322 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:55:00.909274 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4267f1c5-2039-49c6-a07c-905eb4543322 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:55:00.909616 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4267f1c5-2039-49c6-a07c-905eb4543322 admin admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:55:00.909912 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4267f1c5-2039-49c6-a07c-905eb4543322 admin admin] PATH_INFO: `/v3/users/f62009b582dd480198ceca6f41e255c2` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:55:00.911095 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4267f1c5-2039-49c6-a07c-905eb4543322 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f62009b582dd480198ceca6f41e255c2)` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:55:00.913042 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4267f1c5-2039-49c6-a07c-905eb4543322 admin admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:55:00.968266 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.notifications [None req-4267f1c5-2039-49c6-a07c-905eb4543322 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f62009b582dd480198ceca6f41e255c2'} {{(pid=88113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:55:00.978932 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.notifications [None req-4267f1c5-2039-49c6-a07c-905eb4543322 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f62009b582dd480198ceca6f41e255c2'} {{(pid=88113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:55:00.994835 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.notifications [None req-4267f1c5-2039-49c6-a07c-905eb4543322 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f62009b582dd480198ceca6f41e255c2'} {{(pid=88113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:55:00.995738 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.notifications [None req-4267f1c5-2039-49c6-a07c-905eb4543322 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f62009b582dd480198ceca6f41e255c2'} {{(pid=88113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:55:01.005798 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.notifications [None req-4267f1c5-2039-49c6-a07c-905eb4543322 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f62009b582dd480198ceca6f41e255c2'} {{(pid=88113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:55:01.014971 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 1222/2445] 10.222.0.13 () {58 vars in 1224 bytes} [Wed Aug 31 09:55:00 2022] DELETE /identity/v3/users/f62009b582dd480198ceca6f41e255c2 => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:55:01.206899 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af1ac31f-5920-4b60-9919-f426dff4b1c6 service placement] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:55:01.222807 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af1ac31f-5920-4b60-9919-f426dff4b1c6 service placement] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:55:01.326220 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ef4484-6f3e-4471-8147-b4391aed2d96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:55:01.327204 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ef4484-6f3e-4471-8147-b4391aed2d96 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:55:01.327699 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ef4484-6f3e-4471-8147-b4391aed2d96 admin admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:55:01.328189 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ef4484-6f3e-4471-8147-b4391aed2d96 admin admin] PATH_INFO: `/v3/projects/f8336bd8ac0c4600a25261e1af19a7df` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:55:01.330132 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34ef4484-6f3e-4471-8147-b4391aed2d96 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f8336bd8ac0c4600a25261e1af19a7df)` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:55:01.334005 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34ef4484-6f3e-4471-8147-b4391aed2d96 admin admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:55:01.391238 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.notifications [None req-34ef4484-6f3e-4471-8147-b4391aed2d96 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f8336bd8ac0c4600a25261e1af19a7df', 'request_id': 'req-34ef4484-6f3e-4471-8147-b4391aed2d96'} {{(pid=88114) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:55:01.399443 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.notifications [None req-34ef4484-6f3e-4471-8147-b4391aed2d96 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f8336bd8ac0c4600a25261e1af19a7df', 'request_id': 'req-34ef4484-6f3e-4471-8147-b4391aed2d96'} {{(pid=88114) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:55:01.406222 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.notifications [None req-34ef4484-6f3e-4471-8147-b4391aed2d96 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f8336bd8ac0c4600a25261e1af19a7df', 'request_id': 'req-34ef4484-6f3e-4471-8147-b4391aed2d96'} {{(pid=88114) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:55:01.420533 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.notifications [None req-34ef4484-6f3e-4471-8147-b4391aed2d96 admin admin] The token cache is being invalidate because project f8336bd8ac0c4600a25261e1af19a7df was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88114) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 09:55:01.420821 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.notifications [None req-34ef4484-6f3e-4471-8147-b4391aed2d96 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88114) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:55:01.421501 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.notifications [None req-34ef4484-6f3e-4471-8147-b4391aed2d96 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88114) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:55:01.423057 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 1224/2446] 10.222.0.13 () {58 vars in 1233 bytes} [Wed Aug 31 09:55:01 2022] DELETE /identity/v3/projects/f8336bd8ac0c4600a25261e1af19a7df => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:55:11.009637 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=29,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:55:11.417077 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=57,keystone:DELETE=7,keystone:INSERT=1 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:55:18.986087 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4267f1c5-2039-49c6-a07c-905eb4543322 admin admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:55:18.988178 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.fernet_utils [None req-4267f1c5-2039-49c6-a07c-905eb4543322 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:55:19.071290 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4267f1c5-2039-49c6-a07c-905eb4543322 admin admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:55:19.140029 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4826b140-d62c-4ab6-9848-8b39cca7bbf5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc076493239d4ccdbdc37c0369c49529', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '49548eb44d62415582af820ab66decee', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:55:19.140654 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4826b140-d62c-4ab6-9848-8b39cca7bbf5 service nova] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:55:19.140933 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4826b140-d62c-4ab6-9848-8b39cca7bbf5 service nova] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:55:19.141202 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4826b140-d62c-4ab6-9848-8b39cca7bbf5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:55:19.142965 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.fernet_utils [None req-4826b140-d62c-4ab6-9848-8b39cca7bbf5 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:55:19.206075 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4826b140-d62c-4ab6-9848-8b39cca7bbf5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:55:19.208084 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4826b140-d62c-4ab6-9848-8b39cca7bbf5 service nova] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:55:19.217732 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 1223/2447] 10.222.0.13 () {60 vars in 1365 bytes} [Wed Aug 31 09:55:18 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 232 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:55:29.213338 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=84 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:55:35.377312 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64bb465d-2381-48be-90e3-820aafa3b035 None None] REQUEST_METHOD: `POST` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:55:35.378798 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64bb465d-2381-48be-90e3-820aafa3b035 None None] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:55:35.378798 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64bb465d-2381-48be-90e3-820aafa3b035 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:55:35.404360 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.auth.core [None req-64bb465d-2381-48be-90e3-820aafa3b035 None None] MFA Rules not processed for user `25b4233e5f0e41bf99b3fb3f5f6229c9`. Rule list: `[]` (Enabled: `True`). {{(pid=88114) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:55:35.405727 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.fernet_utils [None req-64bb465d-2381-48be-90e3-820aafa3b035 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88114) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:55:35.415724 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 1225/2448] 10.222.0.13 () {58 vars in 958 bytes} [Wed Aug 31 09:55:35 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 09:55:45.393847 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=3 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:56:00.612545 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4826b140-d62c-4ab6-9848-8b39cca7bbf5 service nova] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:56:00.614118 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.fernet_utils [None req-4826b140-d62c-4ab6-9848-8b39cca7bbf5 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:56:00.741227 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4826b140-d62c-4ab6-9848-8b39cca7bbf5 service nova] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:56:00.745251 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec55e23-6bb9-496f-bbad-348db46cad0e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9bdaacd597442c2bcde2c555f2cc140', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '49548eb44d62415582af820ab66decee', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:56:00.745935 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec55e23-6bb9-496f-bbad-348db46cad0e service neutron] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:56:00.746216 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec55e23-6bb9-496f-bbad-348db46cad0e service neutron] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:56:00.746566 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec55e23-6bb9-496f-bbad-348db46cad0e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:56:00.749241 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.fernet_utils [None req-bec55e23-6bb9-496f-bbad-348db46cad0e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:56:00.932996 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bec55e23-6bb9-496f-bbad-348db46cad0e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:56:00.936433 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bec55e23-6bb9-496f-bbad-348db46cad0e service neutron] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:56:00.967196 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 1224/2449] 10.222.0.13 () {60 vars in 1365 bytes} [Wed Aug 31 09:56:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 355 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:56:05.415801 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beaea69a-0466-469d-a2ba-fa803b7a4ded None None] REQUEST_METHOD: `POST` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:56:05.417000 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beaea69a-0466-469d-a2ba-fa803b7a4ded None None] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:56:05.417000 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beaea69a-0466-469d-a2ba-fa803b7a4ded None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:56:05.438019 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.auth.core [None req-beaea69a-0466-469d-a2ba-fa803b7a4ded None None] MFA Rules not processed for user `4961545b43f74fdb87d01b78600d54ed`. Rule list: `[]` (Enabled: `True`). {{(pid=88114) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:56:05.439953 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.fernet_utils [None req-beaea69a-0466-469d-a2ba-fa803b7a4ded None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88114) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:56:05.502788 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 1226/2450] 10.222.0.13 () {58 vars in 958 bytes} [Wed Aug 31 09:56:05 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:56:05.655078 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec55e23-6bb9-496f-bbad-348db46cad0e service neutron] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:56:05.656525 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.fernet_utils [None req-bec55e23-6bb9-496f-bbad-348db46cad0e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:56:05.670471 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec55e23-6bb9-496f-bbad-348db46cad0e service neutron] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:56:05.676566 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f1e4324-8b52-4df0-a086-6f08bcc3fa12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:56:05.677195 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1e4324-8b52-4df0-a086-6f08bcc3fa12 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:56:05.677500 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1e4324-8b52-4df0-a086-6f08bcc3fa12 admin admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:56:05.677797 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1e4324-8b52-4df0-a086-6f08bcc3fa12 admin admin] PATH_INFO: `/v3/users/886e40cf192d46c2b2e7eb2a95ec62e1` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:56:05.679103 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f1e4324-8b52-4df0-a086-6f08bcc3fa12 admin admin] RBAC: Authorizing `identity:delete_user(user_id=886e40cf192d46c2b2e7eb2a95ec62e1)` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:56:05.681078 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f1e4324-8b52-4df0-a086-6f08bcc3fa12 admin admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:56:05.714076 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:56:05.740755 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.notifications [None req-3f1e4324-8b52-4df0-a086-6f08bcc3fa12 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '886e40cf192d46c2b2e7eb2a95ec62e1'} {{(pid=88113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:56:05.760122 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.notifications [None req-3f1e4324-8b52-4df0-a086-6f08bcc3fa12 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '886e40cf192d46c2b2e7eb2a95ec62e1'} {{(pid=88113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:56:05.772841 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.notifications [None req-3f1e4324-8b52-4df0-a086-6f08bcc3fa12 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '886e40cf192d46c2b2e7eb2a95ec62e1'} {{(pid=88113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:56:05.773600 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.notifications [None req-3f1e4324-8b52-4df0-a086-6f08bcc3fa12 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '886e40cf192d46c2b2e7eb2a95ec62e1'} {{(pid=88113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:56:05.781610 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.notifications [None req-3f1e4324-8b52-4df0-a086-6f08bcc3fa12 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '886e40cf192d46c2b2e7eb2a95ec62e1'} {{(pid=88113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:56:05.788442 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 1225/2451] 10.222.0.13 () {58 vars in 1224 bytes} [Wed Aug 31 09:56:05 2022] DELETE /identity/v3/users/886e40cf192d46c2b2e7eb2a95ec62e1 => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:56:05.984864 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beaea69a-0466-469d-a2ba-fa803b7a4ded None None] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:56:05.998574 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beaea69a-0466-469d-a2ba-fa803b7a4ded None None] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:56:06.066813 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c68c4b3-a512-4607-bb43-fabb251e98b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:56:06.067464 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c68c4b3-a512-4607-bb43-fabb251e98b0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:56:06.067739 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c68c4b3-a512-4607-bb43-fabb251e98b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:56:06.068005 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c68c4b3-a512-4607-bb43-fabb251e98b0 admin admin] PATH_INFO: `/v3/projects/c0fc6660b5494415915cdf2b325e5bd6` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:56:06.069165 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c68c4b3-a512-4607-bb43-fabb251e98b0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c0fc6660b5494415915cdf2b325e5bd6)` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:56:06.071170 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c68c4b3-a512-4607-bb43-fabb251e98b0 admin admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:56:06.128099 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.notifications [None req-1c68c4b3-a512-4607-bb43-fabb251e98b0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c0fc6660b5494415915cdf2b325e5bd6', 'request_id': 'req-1c68c4b3-a512-4607-bb43-fabb251e98b0'} {{(pid=88114) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:56:06.138129 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.notifications [None req-1c68c4b3-a512-4607-bb43-fabb251e98b0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c0fc6660b5494415915cdf2b325e5bd6', 'request_id': 'req-1c68c4b3-a512-4607-bb43-fabb251e98b0'} {{(pid=88114) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:56:06.150602 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.notifications [None req-1c68c4b3-a512-4607-bb43-fabb251e98b0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c0fc6660b5494415915cdf2b325e5bd6', 'request_id': 'req-1c68c4b3-a512-4607-bb43-fabb251e98b0'} {{(pid=88114) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:56:06.165991 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.notifications [None req-1c68c4b3-a512-4607-bb43-fabb251e98b0 admin admin] The token cache is being invalidate because project c0fc6660b5494415915cdf2b325e5bd6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88114) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 09:56:06.166399 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.notifications [None req-1c68c4b3-a512-4607-bb43-fabb251e98b0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88114) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:56:06.167157 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.notifications [None req-1c68c4b3-a512-4607-bb43-fabb251e98b0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88114) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:56:06.168897 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 1227/2452] 10.222.0.13 () {58 vars in 1233 bytes} [Wed Aug 31 09:56:05 2022] DELETE /identity/v3/projects/c0fc6660b5494415915cdf2b325e5bd6 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 09:56:06.174645 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213dbe00-1779-4541-8307-7c53e190a954 None None] REQUEST_METHOD: `POST` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:56:06.174942 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213dbe00-1779-4541-8307-7c53e190a954 None None] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:56:06.175290 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213dbe00-1779-4541-8307-7c53e190a954 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:56:06.247954 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.auth.core [None req-213dbe00-1779-4541-8307-7c53e190a954 None None] MFA Rules not processed for user `fd836cf590a145df9acf19cf3ae2ac56`. Rule list: `[]` (Enabled: `True`). {{(pid=88113) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:56:06.249693 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.fernet_utils [None req-213dbe00-1779-4541-8307-7c53e190a954 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:56:06.317229 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 1226/2453] 10.222.0.13 () {58 vars in 958 bytes} [Wed Aug 31 09:56:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 09:56:06.324758 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de06a7c-0cc9-4495-8d59-1b119b883086 None None] REQUEST_METHOD: `POST` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:56:06.325036 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de06a7c-0cc9-4495-8d59-1b119b883086 None None] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:56:06.325303 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de06a7c-0cc9-4495-8d59-1b119b883086 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:56:06.345455 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.auth.core [None req-1de06a7c-0cc9-4495-8d59-1b119b883086 None None] MFA Rules not processed for user `fd836cf590a145df9acf19cf3ae2ac56`. Rule list: `[]` (Enabled: `True`). {{(pid=88114) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:56:06.346736 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.fernet_utils [None req-1de06a7c-0cc9-4495-8d59-1b119b883086 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88114) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:56:06.354842 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 1228/2454] 10.222.0.13 () {58 vars in 958 bytes} [Wed Aug 31 09:56:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:56:06.359040 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-213dbe00-1779-4541-8307-7c53e190a954 None None] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:56:06.370240 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-213dbe00-1779-4541-8307-7c53e190a954 None None] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:56:06.374902 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-719d26f5-0273-437b-85f0-d8b9e6899a0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:56:06.375499 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719d26f5-0273-437b-85f0-d8b9e6899a0f admin admin] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:56:06.375778 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719d26f5-0273-437b-85f0-d8b9e6899a0f admin admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:56:06.376434 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719d26f5-0273-437b-85f0-d8b9e6899a0f admin admin] PATH_INFO: `/v3/domains` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:56:06.377317 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-719d26f5-0273-437b-85f0-d8b9e6899a0f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88113) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:56:06.377662 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-719d26f5-0273-437b-85f0-d8b9e6899a0f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:56:06.379261 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-719d26f5-0273-437b-85f0-d8b9e6899a0f admin admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:56:06.390999 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 1227/2455] 10.222.0.13 () {58 vars in 1153 bytes} [Wed Aug 31 09:56:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:56:06.395285 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de06a7c-0cc9-4495-8d59-1b119b883086 None None] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:56:06.404547 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de06a7c-0cc9-4495-8d59-1b119b883086 None None] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:56:06.410532 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8928c087-fa80-40ed-9036-6577522874ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:56:06.411134 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8928c087-fa80-40ed-9036-6577522874ed admin admin] REQUEST_METHOD: `POST` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:56:06.411406 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8928c087-fa80-40ed-9036-6577522874ed admin admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:56:06.411670 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8928c087-fa80-40ed-9036-6577522874ed admin admin] PATH_INFO: `/v3/projects` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:56:06.412294 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8928c087-fa80-40ed-9036-6577522874ed admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:56:06.414209 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8928c087-fa80-40ed-9036-6577522874ed admin admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:56:06.450333 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 1229/2456] 10.222.0.13 () {60 vars in 1153 bytes} [Wed Aug 31 09:56:06 2022] POST /identity/v3/projects => generated 389 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 09:56:06.455154 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-719d26f5-0273-437b-85f0-d8b9e6899a0f admin admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:56:06.464998 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-719d26f5-0273-437b-85f0-d8b9e6899a0f admin admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:56:06.522340 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d9189c3-5ada-4cb6-8b9d-764bd2f9dcbc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:56:06.522939 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9189c3-5ada-4cb6-8b9d-764bd2f9dcbc admin admin] REQUEST_METHOD: `POST` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:56:06.523213 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9189c3-5ada-4cb6-8b9d-764bd2f9dcbc admin admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:56:06.523480 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9189c3-5ada-4cb6-8b9d-764bd2f9dcbc admin admin] PATH_INFO: `/v3/users` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:56:06.524112 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d9189c3-5ada-4cb6-8b9d-764bd2f9dcbc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:56:06.526063 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d9189c3-5ada-4cb6-8b9d-764bd2f9dcbc admin admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:56:06.535072 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=93,keystone:DELETE=5,keystone:INSERT=2 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:56:06.548836 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 1228/2457] 10.222.0.13 () {60 vars in 1144 bytes} [Wed Aug 31 09:56:06 2022] POST /identity/v3/users => generated 295 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 09:56:06.553337 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8928c087-fa80-40ed-9036-6577522874ed admin admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:56:06.564064 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8928c087-fa80-40ed-9036-6577522874ed admin admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:56:06.569328 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c25c8f-ad34-44b7-b31a-c72d2fc467b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:56:06.569960 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c25c8f-ad34-44b7-b31a-c72d2fc467b6 admin admin] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:56:06.570233 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c25c8f-ad34-44b7-b31a-c72d2fc467b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:56:06.570500 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c25c8f-ad34-44b7-b31a-c72d2fc467b6 admin admin] PATH_INFO: `/v3/roles` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:56:06.571135 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19c25c8f-ad34-44b7-b31a-c72d2fc467b6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:56:06.572716 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19c25c8f-ad34-44b7-b31a-c72d2fc467b6 admin admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:56:06.574141 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:56:06.585057 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 1230/2458] 10.222.0.13 () {58 vars in 1122 bytes} [Wed Aug 31 09:56:06 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 09:56:06.589265 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d9189c3-5ada-4cb6-8b9d-764bd2f9dcbc admin admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:56:06.600082 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d9189c3-5ada-4cb6-8b9d-764bd2f9dcbc admin admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:56:06.605082 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a8ce18-60a2-4cbb-a70c-7528198f3580 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:56:06.605744 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a8ce18-60a2-4cbb-a70c-7528198f3580 admin admin] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:56:06.606063 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a8ce18-60a2-4cbb-a70c-7528198f3580 admin admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:56:06.606338 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a8ce18-60a2-4cbb-a70c-7528198f3580 admin admin] PATH_INFO: `/v3/roles` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:56:06.606997 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9a8ce18-60a2-4cbb-a70c-7528198f3580 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:56:06.608688 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9a8ce18-60a2-4cbb-a70c-7528198f3580 admin admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:56:06.621991 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 1229/2459] 10.222.0.13 () {58 vars in 1122 bytes} [Wed Aug 31 09:56:06 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 09:56:06.627205 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c25c8f-ad34-44b7-b31a-c72d2fc467b6 admin admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:56:06.637475 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c25c8f-ad34-44b7-b31a-c72d2fc467b6 admin admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:56:06.642712 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d21e0234-daf6-49b2-a27a-571032dfe344 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:56:06.643381 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21e0234-daf6-49b2-a27a-571032dfe344 admin admin] REQUEST_METHOD: `PUT` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:56:06.643652 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21e0234-daf6-49b2-a27a-571032dfe344 admin admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:56:06.643923 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21e0234-daf6-49b2-a27a-571032dfe344 admin admin] PATH_INFO: `/v3/projects/a68bf51dcd404f559e9ae2efeb185712/users/6dc6f70f3b1b40d9be1f9d01da0de1f1/roles/6c99f5d9f1f243eaa134d8540f21095d` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:56:06.660864 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d21e0234-daf6-49b2-a27a-571032dfe344 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a68bf51dcd404f559e9ae2efeb185712, user_id=6dc6f70f3b1b40d9be1f9d01da0de1f1, role_id=6c99f5d9f1f243eaa134d8540f21095d)` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:56:06.663634 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d21e0234-daf6-49b2-a27a-571032dfe344 admin admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:56:06.676885 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 1231/2460] 10.222.0.13 () {60 vars in 1483 bytes} [Wed Aug 31 09:56:06 2022] PUT /identity/v3/projects/a68bf51dcd404f559e9ae2efeb185712/users/6dc6f70f3b1b40d9be1f9d01da0de1f1/roles/6c99f5d9f1f243eaa134d8540f21095d => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 09:56:06.682174 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dbd2d6d-441f-4a43-b194-353f0c92c4cc None None] REQUEST_METHOD: `POST` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:56:06.682662 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dbd2d6d-441f-4a43-b194-353f0c92c4cc None None] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:56:06.682759 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dbd2d6d-441f-4a43-b194-353f0c92c4cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:56:06.702351 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.auth.core [None req-5dbd2d6d-441f-4a43-b194-353f0c92c4cc None None] MFA Rules not processed for user `fd836cf590a145df9acf19cf3ae2ac56`. Rule list: `[]` (Enabled: `True`). {{(pid=88113) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:56:06.703456 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.fernet_utils [None req-5dbd2d6d-441f-4a43-b194-353f0c92c4cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:56:06.793824 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 1230/2461] 10.222.0.13 () {58 vars in 958 bytes} [Wed Aug 31 09:56:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:56:06.803815 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d21e0234-daf6-49b2-a27a-571032dfe344 admin admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:56:06.805582 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.fernet_utils [None req-d21e0234-daf6-49b2-a27a-571032dfe344 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88114) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:56:06.895486 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d21e0234-daf6-49b2-a27a-571032dfe344 admin admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:56:06.899226 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a53cec-e4b2-4a4a-8971-0ce43861d8d9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9bdaacd597442c2bcde2c555f2cc140', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '49548eb44d62415582af820ab66decee', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:56:06.899826 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a53cec-e4b2-4a4a-8971-0ce43861d8d9 service neutron] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:56:06.900103 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a53cec-e4b2-4a4a-8971-0ce43861d8d9 service neutron] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:56:06.900369 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a53cec-e4b2-4a4a-8971-0ce43861d8d9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:56:06.910187 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12a53cec-e4b2-4a4a-8971-0ce43861d8d9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:56:06.911601 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12a53cec-e4b2-4a4a-8971-0ce43861d8d9 service neutron] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:56:06.920969 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 1232/2462] 10.222.0.13 () {60 vars in 1365 bytes} [Wed Aug 31 09:56:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:56:11.113156 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dbd2d6d-441f-4a43-b194-353f0c92c4cc None None] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:56:11.116061 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.fernet_utils [None req-5dbd2d6d-441f-4a43-b194-353f0c92c4cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:56:11.386870 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dbd2d6d-441f-4a43-b194-353f0c92c4cc None None] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:56:11.398898 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b570f2b6-3728-44d3-8a49-79bdeaf0ef1f service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ff3aff5ef444910be6dbb989d0e3194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '49548eb44d62415582af820ab66decee', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:56:11.399495 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b570f2b6-3728-44d3-8a49-79bdeaf0ef1f service placement] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:56:11.399854 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b570f2b6-3728-44d3-8a49-79bdeaf0ef1f service placement] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:56:11.400280 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b570f2b6-3728-44d3-8a49-79bdeaf0ef1f service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:56:11.402691 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.fernet_utils [None req-b570f2b6-3728-44d3-8a49-79bdeaf0ef1f service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:56:11.419392 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b570f2b6-3728-44d3-8a49-79bdeaf0ef1f service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:56:11.420265 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b570f2b6-3728-44d3-8a49-79bdeaf0ef1f service placement] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:56:11.436127 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 1231/2463] 10.222.0.13 () {60 vars in 1365 bytes} [Wed Aug 31 09:56:11 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:56:11.698843 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f022b903-6db3-4abc-b4c7-4f86758507d0 None None] REQUEST_METHOD: `POST` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:56:11.699692 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f022b903-6db3-4abc-b4c7-4f86758507d0 None None] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:56:11.699692 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f022b903-6db3-4abc-b4c7-4f86758507d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:56:11.766489 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.auth.core [None req-f022b903-6db3-4abc-b4c7-4f86758507d0 None None] MFA Rules not processed for user `6dc6f70f3b1b40d9be1f9d01da0de1f1`. Rule list: `[]` (Enabled: `True`). {{(pid=88114) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:56:11.768157 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.fernet_utils [None req-f022b903-6db3-4abc-b4c7-4f86758507d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88114) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:56:11.938272 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 1233/2464] 10.222.0.13 () {58 vars in 958 bytes} [Wed Aug 31 09:56:11 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 240 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:56:11.948442 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b570f2b6-3728-44d3-8a49-79bdeaf0ef1f service placement] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:56:11.950696 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.fernet_utils [None req-b570f2b6-3728-44d3-8a49-79bdeaf0ef1f service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:56:12.042590 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:56:12.122836 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b570f2b6-3728-44d3-8a49-79bdeaf0ef1f service placement] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:56:12.129015 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4af29e1-9834-4790-bd59-4febb5856469 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc076493239d4ccdbdc37c0369c49529', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '49548eb44d62415582af820ab66decee', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:56:12.129755 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4af29e1-9834-4790-bd59-4febb5856469 service nova] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:56:12.130035 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4af29e1-9834-4790-bd59-4febb5856469 service nova] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:56:12.130304 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4af29e1-9834-4790-bd59-4febb5856469 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:56:12.143710 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4af29e1-9834-4790-bd59-4febb5856469 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:56:12.145415 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4af29e1-9834-4790-bd59-4febb5856469 service nova] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:56:12.159299 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 1232/2465] 10.222.0.13 () {60 vars in 1365 bytes} [Wed Aug 31 09:56:11 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 211 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 09:56:13.919642 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03990b52-0eaa-4647-9587-8646cd2e654b None None] REQUEST_METHOD: `POST` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:56:13.920166 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03990b52-0eaa-4647-9587-8646cd2e654b None None] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:56:13.920166 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03990b52-0eaa-4647-9587-8646cd2e654b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:56:13.942544 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.auth.core [None req-03990b52-0eaa-4647-9587-8646cd2e654b None None] MFA Rules not processed for user `6dc6f70f3b1b40d9be1f9d01da0de1f1`. Rule list: `[]` (Enabled: `True`). {{(pid=88114) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:56:13.944940 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.fernet_utils [None req-03990b52-0eaa-4647-9587-8646cd2e654b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88114) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:56:13.958449 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 1234/2466] 10.222.0.13 () {58 vars in 958 bytes} [Wed Aug 31 09:56:13 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:56:14.337070 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4af29e1-9834-4790-bd59-4febb5856469 service nova] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:56:14.351373 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4af29e1-9834-4790-bd59-4febb5856469 service nova] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:56:14.357093 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a833f30-d895-4166-abb3-3dcaed576a29 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9bdaacd597442c2bcde2c555f2cc140', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '49548eb44d62415582af820ab66decee', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:56:14.358586 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a833f30-d895-4166-abb3-3dcaed576a29 service neutron] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:56:14.359228 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a833f30-d895-4166-abb3-3dcaed576a29 service neutron] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:56:14.359588 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a833f30-d895-4166-abb3-3dcaed576a29 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:56:14.362844 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.fernet_utils [None req-8a833f30-d895-4166-abb3-3dcaed576a29 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:56:14.382868 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a833f30-d895-4166-abb3-3dcaed576a29 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:56:14.385837 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a833f30-d895-4166-abb3-3dcaed576a29 service neutron] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:56:14.409445 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 1233/2467] 10.222.0.13 () {60 vars in 1365 bytes} [Wed Aug 31 09:56:14 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 67 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:56:18.758268 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117127b4-0894-42d1-9b7e-2c109718d3fb None None] REQUEST_METHOD: `POST` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:56:18.758785 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117127b4-0894-42d1-9b7e-2c109718d3fb None None] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:56:18.758892 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117127b4-0894-42d1-9b7e-2c109718d3fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:56:18.784278 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.auth.core [None req-117127b4-0894-42d1-9b7e-2c109718d3fb None None] MFA Rules not processed for user `1090c027746845239055973b44207556`. Rule list: `[]` (Enabled: `True`). {{(pid=88114) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:56:18.785651 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.fernet_utils [None req-117127b4-0894-42d1-9b7e-2c109718d3fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88114) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:56:18.828309 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:56:18.874831 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 1235/2468] 10.222.0.13 () {58 vars in 958 bytes} [Wed Aug 31 09:56:18 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:56:24.395362 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=29 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:56:28.864826 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=8 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:56:29.844330 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a833f30-d895-4166-abb3-3dcaed576a29 service neutron] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:56:29.857798 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a833f30-d895-4166-abb3-3dcaed576a29 service neutron] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:56:29.863671 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2fce875-6627-416e-8e08-f6ac55dff576 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc076493239d4ccdbdc37c0369c49529', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '49548eb44d62415582af820ab66decee', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:56:29.864286 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2fce875-6627-416e-8e08-f6ac55dff576 service nova] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:56:29.864553 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2fce875-6627-416e-8e08-f6ac55dff576 service nova] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:56:29.864856 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2fce875-6627-416e-8e08-f6ac55dff576 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:56:29.866890 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.fernet_utils [None req-b2fce875-6627-416e-8e08-f6ac55dff576 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:56:29.929059 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2fce875-6627-416e-8e08-f6ac55dff576 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:56:29.930960 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2fce875-6627-416e-8e08-f6ac55dff576 service nova] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:56:29.942591 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 1234/2469] 10.222.0.13 () {60 vars in 1365 bytes} [Wed Aug 31 09:56:29 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:56:30.131211 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf9529a-440e-4b02-93bb-f4ba204b4d4a None None] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:56:30.132210 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf9529a-440e-4b02-93bb-f4ba204b4d4a None None] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:56:30.132730 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf9529a-440e-4b02-93bb-f4ba204b4d4a None None] PATH_INFO: `/` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:56:30.134145 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 1236/2470] 10.222.0.13 () {54 vars in 921 bytes} [Wed Aug 31 09:56:30 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 09:56:30.141232 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcedab5a-662c-4960-8f4c-2ee98f14e812 None None] REQUEST_METHOD: `POST` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:56:30.141562 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcedab5a-662c-4960-8f4c-2ee98f14e812 None None] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:56:30.141836 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcedab5a-662c-4960-8f4c-2ee98f14e812 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:56:30.175769 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.auth.core [None req-bcedab5a-662c-4960-8f4c-2ee98f14e812 None None] MFA Rules not processed for user `a9bdaacd597442c2bcde2c555f2cc140`. Rule list: `[]` (Enabled: `True`). {{(pid=88113) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:56:30.177147 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.fernet_utils [None req-bcedab5a-662c-4960-8f4c-2ee98f14e812 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:56:30.184719 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 1235/2471] 10.222.0.13 () {58 vars in 1018 bytes} [Wed Aug 31 09:56:30 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:56:30.197609 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf9529a-440e-4b02-93bb-f4ba204b4d4a None None] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:56:30.199407 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.fernet_utils [None req-ddf9529a-440e-4b02-93bb-f4ba204b4d4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88114) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:56:30.220080 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf9529a-440e-4b02-93bb-f4ba204b4d4a None None] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:56:30.224505 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33233ac1-62f9-4ce7-aafd-0766c1e48c5f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9bdaacd597442c2bcde2c555f2cc140', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '49548eb44d62415582af820ab66decee', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:56:30.225209 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33233ac1-62f9-4ce7-aafd-0766c1e48c5f service neutron] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:56:30.225557 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33233ac1-62f9-4ce7-aafd-0766c1e48c5f service neutron] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:56:30.226104 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33233ac1-62f9-4ce7-aafd-0766c1e48c5f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:56:30.234614 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33233ac1-62f9-4ce7-aafd-0766c1e48c5f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:56:30.236373 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33233ac1-62f9-4ce7-aafd-0766c1e48c5f service neutron] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:56:30.245029 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 1237/2472] 10.222.0.13 () {60 vars in 1365 bytes} [Wed Aug 31 09:56:30 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:56:30.360261 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da619d25-78f3-4a5d-a098-61eacdd05332 None None] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:56:30.360629 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da619d25-78f3-4a5d-a098-61eacdd05332 None None] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:56:30.360979 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da619d25-78f3-4a5d-a098-61eacdd05332 None None] PATH_INFO: `/` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:56:30.363505 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 1236/2473] 10.222.0.13 () {54 vars in 921 bytes} [Wed Aug 31 09:56:30 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 31 09:56:30.369959 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9e2b7a-60ee-4d39-a97b-7c77eaf052e7 None None] REQUEST_METHOD: `POST` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:56:30.370240 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9e2b7a-60ee-4d39-a97b-7c77eaf052e7 None None] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:56:30.370517 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9e2b7a-60ee-4d39-a97b-7c77eaf052e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:56:30.398358 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.auth.core [None req-2b9e2b7a-60ee-4d39-a97b-7c77eaf052e7 None None] MFA Rules not processed for user `dc076493239d4ccdbdc37c0369c49529`. Rule list: `[]` (Enabled: `True`). {{(pid=88114) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:56:30.399873 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.fernet_utils [None req-2b9e2b7a-60ee-4d39-a97b-7c77eaf052e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88114) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:56:30.419236 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 1238/2474] 10.222.0.13 () {58 vars in 1018 bytes} [Wed Aug 31 09:56:30 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:56:30.430684 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da619d25-78f3-4a5d-a098-61eacdd05332 None None] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:56:30.445792 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da619d25-78f3-4a5d-a098-61eacdd05332 None None] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:56:30.451860 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-433c2fc3-10b8-46ac-a48e-d57240b251cf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9bdaacd597442c2bcde2c555f2cc140', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '49548eb44d62415582af820ab66decee', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:56:30.452258 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433c2fc3-10b8-46ac-a48e-d57240b251cf service neutron] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:56:30.452417 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433c2fc3-10b8-46ac-a48e-d57240b251cf service neutron] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:56:30.452578 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433c2fc3-10b8-46ac-a48e-d57240b251cf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:56:30.464642 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-433c2fc3-10b8-46ac-a48e-d57240b251cf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:56:30.466183 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-433c2fc3-10b8-46ac-a48e-d57240b251cf service neutron] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:56:30.479577 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 1237/2475] 10.222.0.13 () {60 vars in 1365 bytes} [Wed Aug 31 09:56:30 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 09:56:30.666492 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b9e2b7a-60ee-4d39-a97b-7c77eaf052e7 None None] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:56:30.682035 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b9e2b7a-60ee-4d39-a97b-7c77eaf052e7 None None] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:56:30.690166 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d350c105-54ff-4b8a-9388-fd8590bed903 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc076493239d4ccdbdc37c0369c49529', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '49548eb44d62415582af820ab66decee', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:56:30.690795 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d350c105-54ff-4b8a-9388-fd8590bed903 service nova] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:56:30.691258 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d350c105-54ff-4b8a-9388-fd8590bed903 service nova] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:56:30.691906 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d350c105-54ff-4b8a-9388-fd8590bed903 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:56:30.694586 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.fernet_utils [None req-d350c105-54ff-4b8a-9388-fd8590bed903 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88114) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:56:30.727500 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d350c105-54ff-4b8a-9388-fd8590bed903 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:56:30.729428 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d350c105-54ff-4b8a-9388-fd8590bed903 service nova] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:56:30.748811 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 1239/2476] 10.222.0.13 () {60 vars in 1365 bytes} [Wed Aug 31 09:56:30 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:56:40.474498 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=41 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:56:40.739529 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=15 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:57:38.766989 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4acbb749-c802-481f-9795-79267b7d7f93 None None] REQUEST_METHOD: `POST` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:57:38.767632 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4acbb749-c802-481f-9795-79267b7d7f93 None None] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:57:38.767632 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4acbb749-c802-481f-9795-79267b7d7f93 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:57:38.790560 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.auth.core [None req-4acbb749-c802-481f-9795-79267b7d7f93 None None] MFA Rules not processed for user `25b4233e5f0e41bf99b3fb3f5f6229c9`. Rule list: `[]` (Enabled: `True`). {{(pid=88113) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:57:38.791838 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.fernet_utils [None req-4acbb749-c802-481f-9795-79267b7d7f93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:57:38.863171 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 1238/2477] 10.222.0.13 () {58 vars in 958 bytes} [Wed Aug 31 09:57:38 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:57:48.024742 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d350c105-54ff-4b8a-9388-fd8590bed903 service nova] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:57:48.034327 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d350c105-54ff-4b8a-9388-fd8590bed903 service nova] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:57:48.037822 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b8b683-dbcb-4b1d-ae1a-4fb547b0ba85 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9bdaacd597442c2bcde2c555f2cc140', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '49548eb44d62415582af820ab66decee', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:57:48.038412 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b8b683-dbcb-4b1d-ae1a-4fb547b0ba85 service neutron] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:57:48.038740 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b8b683-dbcb-4b1d-ae1a-4fb547b0ba85 service neutron] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:57:48.038980 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b8b683-dbcb-4b1d-ae1a-4fb547b0ba85 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:57:48.040652 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.fernet_utils [None req-67b8b683-dbcb-4b1d-ae1a-4fb547b0ba85 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88114) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:57:48.054972 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67b8b683-dbcb-4b1d-ae1a-4fb547b0ba85 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:57:48.056415 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67b8b683-dbcb-4b1d-ae1a-4fb547b0ba85 service neutron] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:57:48.069038 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 1240/2478] 10.222.0.13 () {60 vars in 1365 bytes} [Wed Aug 31 09:57:48 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:57:48.855748 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=25 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:57:49.256620 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4acbb749-c802-481f-9795-79267b7d7f93 None None] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:57:49.266721 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4acbb749-c802-481f-9795-79267b7d7f93 None None] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:57:49.271347 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49dee887-d4c7-4052-9efe-ffde0dfb921f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9bdaacd597442c2bcde2c555f2cc140', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '49548eb44d62415582af820ab66decee', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:57:49.271949 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49dee887-d4c7-4052-9efe-ffde0dfb921f service neutron] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:57:49.272225 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49dee887-d4c7-4052-9efe-ffde0dfb921f service neutron] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:57:49.272490 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49dee887-d4c7-4052-9efe-ffde0dfb921f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:57:49.274241 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.fernet_utils [None req-49dee887-d4c7-4052-9efe-ffde0dfb921f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:57:49.288566 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49dee887-d4c7-4052-9efe-ffde0dfb921f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:57:49.290336 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49dee887-d4c7-4052-9efe-ffde0dfb921f service neutron] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:57:49.301355 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 1239/2479] 10.222.0.13 () {60 vars in 1365 bytes} [Wed Aug 31 09:57:49 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:57:58.062823 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=6 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:57:59.296284 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=6 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:58:23.265452 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64e6773-646f-4063-bad3-0c4f8fcd69d8 None None] REQUEST_METHOD: `POST` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:58:23.266019 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64e6773-646f-4063-bad3-0c4f8fcd69d8 None None] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:58:23.266019 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64e6773-646f-4063-bad3-0c4f8fcd69d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:58:23.323211 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.auth.core [None req-b64e6773-646f-4063-bad3-0c4f8fcd69d8 None None] MFA Rules not processed for user `768cb67c1ef24251aaa113c12d81a250`. Rule list: `[]` (Enabled: `True`). {{(pid=88114) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:58:23.324461 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.fernet_utils [None req-b64e6773-646f-4063-bad3-0c4f8fcd69d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88114) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:58:23.396686 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 1241/2480] 10.222.0.13 () {58 vars in 1009 bytes} [Wed Aug 31 09:58:23 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 09:58:23.454662 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49dee887-d4c7-4052-9efe-ffde0dfb921f service neutron] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:58:23.455979 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.fernet_utils [None req-49dee887-d4c7-4052-9efe-ffde0dfb921f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:58:23.569026 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49dee887-d4c7-4052-9efe-ffde0dfb921f service neutron] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:58:23.574364 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a46f1155-e222-4a0e-800f-640b0fd8d106 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '202bb9db7a8a4977ba788a290ed9a3fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '49548eb44d62415582af820ab66decee', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:58:23.574991 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46f1155-e222-4a0e-800f-640b0fd8d106 service swift] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:58:23.575918 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46f1155-e222-4a0e-800f-640b0fd8d106 service swift] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:58:23.576199 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46f1155-e222-4a0e-800f-640b0fd8d106 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:58:23.586442 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a46f1155-e222-4a0e-800f-640b0fd8d106 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:58:23.588066 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a46f1155-e222-4a0e-800f-640b0fd8d106 service swift] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:58:23.597443 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 1240/2481] 10.222.0.13 () {60 vars in 1384 bytes} [Wed Aug 31 09:58:23 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 31 09:58:33.271916 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b64e6773-646f-4063-bad3-0c4f8fcd69d8 None None] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:58:33.287913 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b64e6773-646f-4063-bad3-0c4f8fcd69d8 None None] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:58:33.296439 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb2c06e-83ec-43e1-addd-35c0687a68c4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9bdaacd597442c2bcde2c555f2cc140', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '49548eb44d62415582af820ab66decee', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:58:33.297482 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb2c06e-83ec-43e1-addd-35c0687a68c4 service neutron] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:58:33.297975 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb2c06e-83ec-43e1-addd-35c0687a68c4 service neutron] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:58:33.298444 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb2c06e-83ec-43e1-addd-35c0687a68c4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:58:33.301248 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.fernet_utils [None req-cbb2c06e-83ec-43e1-addd-35c0687a68c4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88114) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:58:33.328812 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbb2c06e-83ec-43e1-addd-35c0687a68c4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:58:33.331703 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbb2c06e-83ec-43e1-addd-35c0687a68c4 service neutron] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:58:33.348188 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 1242/2482] 10.222.0.13 () {60 vars in 1365 bytes} [Wed Aug 31 09:58:33 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 78 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 09:58:33.593464 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=41 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:58:35.687553 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a46f1155-e222-4a0e-800f-640b0fd8d106 service swift] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:58:35.698553 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a46f1155-e222-4a0e-800f-640b0fd8d106 service swift] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:58:35.702591 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37885e92-430c-45ce-bda1-1b7ba1a6010e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9bdaacd597442c2bcde2c555f2cc140', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '49548eb44d62415582af820ab66decee', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:58:35.703409 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37885e92-430c-45ce-bda1-1b7ba1a6010e service neutron] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:58:35.703991 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37885e92-430c-45ce-bda1-1b7ba1a6010e service neutron] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:58:35.704282 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37885e92-430c-45ce-bda1-1b7ba1a6010e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:58:35.707730 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.fernet_utils [None req-37885e92-430c-45ce-bda1-1b7ba1a6010e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:58:35.725417 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37885e92-430c-45ce-bda1-1b7ba1a6010e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:58:35.729496 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37885e92-430c-45ce-bda1-1b7ba1a6010e service neutron] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:58:35.748240 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 1241/2483] 10.222.0.13 () {60 vars in 1365 bytes} [Wed Aug 31 09:58:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 61 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:58:39.980473 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb2c06e-83ec-43e1-addd-35c0687a68c4 service neutron] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:58:39.980473 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.fernet_utils [None req-cbb2c06e-83ec-43e1-addd-35c0687a68c4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88114) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:58:40.008056 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37885e92-430c-45ce-bda1-1b7ba1a6010e service neutron] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:58:40.015325 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb2c06e-83ec-43e1-addd-35c0687a68c4 service neutron] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:58:40.021471 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37885e92-430c-45ce-bda1-1b7ba1a6010e service neutron] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:58:40.024696 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e154ff-755c-4636-9d27-7d3d724d16f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:58:40.025368 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e154ff-755c-4636-9d27-7d3d724d16f3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:58:40.026736 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e154ff-755c-4636-9d27-7d3d724d16f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:58:40.027527 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e154ff-755c-4636-9d27-7d3d724d16f3 admin admin] PATH_INFO: `/v3/users/4961545b43f74fdb87d01b78600d54ed` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:58:40.029113 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f7293c5-6669-4dc4-8328-efd9569f3eae service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9bdaacd597442c2bcde2c555f2cc140', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '49548eb44d62415582af820ab66decee', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:58:40.030096 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7293c5-6669-4dc4-8328-efd9569f3eae service neutron] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:58:40.030447 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7293c5-6669-4dc4-8328-efd9569f3eae service neutron] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:58:40.030795 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7293c5-6669-4dc4-8328-efd9569f3eae service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:58:40.032788 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.fernet_utils [None req-4f7293c5-6669-4dc4-8328-efd9569f3eae service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:58:40.053919 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f7293c5-6669-4dc4-8328-efd9569f3eae service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:58:40.056935 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f7293c5-6669-4dc4-8328-efd9569f3eae service neutron] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:58:40.059966 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63e154ff-755c-4636-9d27-7d3d724d16f3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4961545b43f74fdb87d01b78600d54ed)` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:58:40.069181 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63e154ff-755c-4636-9d27-7d3d724d16f3 admin admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:58:40.077895 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 1242/2484] 10.222.0.13 () {60 vars in 1365 bytes} [Wed Aug 31 09:58:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 70 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:58:40.187523 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.notifications [None req-63e154ff-755c-4636-9d27-7d3d724d16f3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4961545b43f74fdb87d01b78600d54ed'} {{(pid=88114) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:58:40.199459 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.notifications [None req-63e154ff-755c-4636-9d27-7d3d724d16f3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4961545b43f74fdb87d01b78600d54ed'} {{(pid=88114) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:58:40.213039 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.notifications [None req-63e154ff-755c-4636-9d27-7d3d724d16f3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4961545b43f74fdb87d01b78600d54ed'} {{(pid=88114) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:58:40.214171 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.notifications [None req-63e154ff-755c-4636-9d27-7d3d724d16f3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4961545b43f74fdb87d01b78600d54ed'} {{(pid=88114) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:58:40.234787 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.notifications [None req-63e154ff-755c-4636-9d27-7d3d724d16f3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4961545b43f74fdb87d01b78600d54ed'} {{(pid=88114) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:58:40.244280 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 1243/2485] 10.222.0.13 () {58 vars in 1224 bytes} [Wed Aug 31 09:58:39 2022] DELETE /identity/v3/users/4961545b43f74fdb87d01b78600d54ed => generated 0 bytes in 270 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:58:40.464423 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f7293c5-6669-4dc4-8328-efd9569f3eae service neutron] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:58:40.481941 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f7293c5-6669-4dc4-8328-efd9569f3eae service neutron] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:58:40.593188 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90faa2dd-0e50-4d6f-9f91-336d2ce1c1e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:58:40.594300 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90faa2dd-0e50-4d6f-9f91-336d2ce1c1e3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:58:40.594766 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90faa2dd-0e50-4d6f-9f91-336d2ce1c1e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:58:40.595192 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90faa2dd-0e50-4d6f-9f91-336d2ce1c1e3 admin admin] PATH_INFO: `/v3/projects/05be916a558445ee8550559d491efbd0` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:58:40.615597 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90faa2dd-0e50-4d6f-9f91-336d2ce1c1e3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=05be916a558445ee8550559d491efbd0)` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:58:40.619072 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90faa2dd-0e50-4d6f-9f91-336d2ce1c1e3 admin admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:58:40.708182 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.notifications [None req-90faa2dd-0e50-4d6f-9f91-336d2ce1c1e3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '05be916a558445ee8550559d491efbd0', 'request_id': 'req-90faa2dd-0e50-4d6f-9f91-336d2ce1c1e3'} {{(pid=88113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:58:40.717612 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.notifications [None req-90faa2dd-0e50-4d6f-9f91-336d2ce1c1e3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '05be916a558445ee8550559d491efbd0', 'request_id': 'req-90faa2dd-0e50-4d6f-9f91-336d2ce1c1e3'} {{(pid=88113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:58:40.730481 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.notifications [None req-90faa2dd-0e50-4d6f-9f91-336d2ce1c1e3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '05be916a558445ee8550559d491efbd0', 'request_id': 'req-90faa2dd-0e50-4d6f-9f91-336d2ce1c1e3'} {{(pid=88113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:58:40.756148 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.notifications [None req-90faa2dd-0e50-4d6f-9f91-336d2ce1c1e3 admin admin] The token cache is being invalidate because project 05be916a558445ee8550559d491efbd0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88113) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 09:58:40.756652 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.notifications [None req-90faa2dd-0e50-4d6f-9f91-336d2ce1c1e3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:58:40.757817 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.notifications [None req-90faa2dd-0e50-4d6f-9f91-336d2ce1c1e3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:58:40.760636 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 1243/2486] 10.222.0.13 () {58 vars in 1233 bytes} [Wed Aug 31 09:58:40 2022] DELETE /identity/v3/projects/05be916a558445ee8550559d491efbd0 => generated 0 bytes in 297 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:58:40.766544 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07dd00f3-405f-4b3c-9cfa-a40a0f20de4b None None] REQUEST_METHOD: `POST` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:58:40.766863 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07dd00f3-405f-4b3c-9cfa-a40a0f20de4b None None] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:58:40.767087 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07dd00f3-405f-4b3c-9cfa-a40a0f20de4b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:58:40.795696 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.auth.core [None req-07dd00f3-405f-4b3c-9cfa-a40a0f20de4b None None] MFA Rules not processed for user `fd836cf590a145df9acf19cf3ae2ac56`. Rule list: `[]` (Enabled: `True`). {{(pid=88114) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:58:40.796987 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.fernet_utils [None req-07dd00f3-405f-4b3c-9cfa-a40a0f20de4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88114) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:58:40.830207 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:58:40.952979 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 1244/2487] 10.222.0.13 () {58 vars in 958 bytes} [Wed Aug 31 09:58:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:58:40.965782 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66b241c-3475-4169-b065-2fc7eddffcea None None] REQUEST_METHOD: `POST` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:58:40.966101 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66b241c-3475-4169-b065-2fc7eddffcea None None] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:58:40.966356 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66b241c-3475-4169-b065-2fc7eddffcea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:58:40.998367 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.auth.core [None req-e66b241c-3475-4169-b065-2fc7eddffcea None None] MFA Rules not processed for user `fd836cf590a145df9acf19cf3ae2ac56`. Rule list: `[]` (Enabled: `True`). {{(pid=88113) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:58:41.001343 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.fernet_utils [None req-e66b241c-3475-4169-b065-2fc7eddffcea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:58:41.018210 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 1244/2488] 10.222.0.13 () {58 vars in 958 bytes} [Wed Aug 31 09:58:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 53 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:58:41.019356 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07dd00f3-405f-4b3c-9cfa-a40a0f20de4b None None] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:58:41.033191 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07dd00f3-405f-4b3c-9cfa-a40a0f20de4b None None] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:58:41.040646 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a82e6a79-32c1-407d-ad53-c73d48c324cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:58:41.041662 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82e6a79-32c1-407d-ad53-c73d48c324cf admin admin] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:58:41.041943 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82e6a79-32c1-407d-ad53-c73d48c324cf admin admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:58:41.042209 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82e6a79-32c1-407d-ad53-c73d48c324cf admin admin] PATH_INFO: `/v3/domains` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:58:41.042883 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a82e6a79-32c1-407d-ad53-c73d48c324cf admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88114) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:58:41.043341 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a82e6a79-32c1-407d-ad53-c73d48c324cf admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:58:41.045106 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a82e6a79-32c1-407d-ad53-c73d48c324cf admin admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:58:41.061424 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 1245/2489] 10.222.0.13 () {58 vars in 1153 bytes} [Wed Aug 31 09:58:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:58:41.068227 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e66b241c-3475-4169-b065-2fc7eddffcea None None] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:58:41.081633 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e66b241c-3475-4169-b065-2fc7eddffcea None None] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:58:41.091653 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-721733f4-80cb-4147-ba13-15c4527cdd9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:58:41.092377 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721733f4-80cb-4147-ba13-15c4527cdd9c admin admin] REQUEST_METHOD: `POST` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:58:41.092655 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721733f4-80cb-4147-ba13-15c4527cdd9c admin admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:58:41.092925 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721733f4-80cb-4147-ba13-15c4527cdd9c admin admin] PATH_INFO: `/v3/projects` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:58:41.093613 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-721733f4-80cb-4147-ba13-15c4527cdd9c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:58:41.096900 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-721733f4-80cb-4147-ba13-15c4527cdd9c admin admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:58:41.159623 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 1245/2490] 10.222.0.13 () {60 vars in 1153 bytes} [Wed Aug 31 09:58:41 2022] POST /identity/v3/projects => generated 369 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 09:58:41.167330 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a82e6a79-32c1-407d-ad53-c73d48c324cf admin admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:58:41.186692 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a82e6a79-32c1-407d-ad53-c73d48c324cf admin admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:58:41.290378 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d4bc6e2-2cc9-4e84-a640-4c6af2709e80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:58:41.291017 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4bc6e2-2cc9-4e84-a640-4c6af2709e80 admin admin] REQUEST_METHOD: `POST` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:58:41.291293 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4bc6e2-2cc9-4e84-a640-4c6af2709e80 admin admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:58:41.291564 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4bc6e2-2cc9-4e84-a640-4c6af2709e80 admin admin] PATH_INFO: `/v3/users` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:58:41.292213 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d4bc6e2-2cc9-4e84-a640-4c6af2709e80 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:58:41.294214 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d4bc6e2-2cc9-4e84-a640-4c6af2709e80 admin admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:58:41.325062 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 1246/2491] 10.222.0.13 () {60 vars in 1144 bytes} [Wed Aug 31 09:58:41 2022] POST /identity/v3/users => generated 285 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 09:58:41.331320 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-721733f4-80cb-4147-ba13-15c4527cdd9c admin admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:58:41.343999 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-721733f4-80cb-4147-ba13-15c4527cdd9c admin admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:58:41.350809 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae839d71-3745-4fc3-b210-2e0e15a8b39c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:58:41.351445 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae839d71-3745-4fc3-b210-2e0e15a8b39c admin admin] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:58:41.351793 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae839d71-3745-4fc3-b210-2e0e15a8b39c admin admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:58:41.352020 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae839d71-3745-4fc3-b210-2e0e15a8b39c admin admin] PATH_INFO: `/v3/roles` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:58:41.352719 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae839d71-3745-4fc3-b210-2e0e15a8b39c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:58:41.354839 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae839d71-3745-4fc3-b210-2e0e15a8b39c admin admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:58:41.382480 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 1246/2492] 10.222.0.13 () {58 vars in 1122 bytes} [Wed Aug 31 09:58:41 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 09:58:41.383344 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d4bc6e2-2cc9-4e84-a640-4c6af2709e80 admin admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:58:41.396436 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d4bc6e2-2cc9-4e84-a640-4c6af2709e80 admin admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:58:41.403200 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0388234-0c75-4d73-8b78-71b232b155b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:58:41.403996 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0388234-0c75-4d73-8b78-71b232b155b5 admin admin] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:58:41.404262 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0388234-0c75-4d73-8b78-71b232b155b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:58:41.404565 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0388234-0c75-4d73-8b78-71b232b155b5 admin admin] PATH_INFO: `/v3/roles` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:58:41.405438 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0388234-0c75-4d73-8b78-71b232b155b5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:58:41.407356 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0388234-0c75-4d73-8b78-71b232b155b5 admin admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:58:41.426040 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 1247/2493] 10.222.0.13 () {58 vars in 1122 bytes} [Wed Aug 31 09:58:41 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 09:58:41.432174 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae839d71-3745-4fc3-b210-2e0e15a8b39c admin admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:58:41.449518 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae839d71-3745-4fc3-b210-2e0e15a8b39c admin admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:58:41.458639 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d80a6c6a-8f09-4096-b4c2-1d7eaba1d6a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:58:41.459347 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80a6c6a-8f09-4096-b4c2-1d7eaba1d6a1 admin admin] REQUEST_METHOD: `PUT` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:58:41.459627 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80a6c6a-8f09-4096-b4c2-1d7eaba1d6a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:58:41.459901 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80a6c6a-8f09-4096-b4c2-1d7eaba1d6a1 admin admin] PATH_INFO: `/v3/projects/37bd863e37e049eb839a7b6286d37090/users/47a02b245543433babd23d2bb41347f8/roles/6c99f5d9f1f243eaa134d8540f21095d` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:58:41.465957 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:58:41.485445 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d80a6c6a-8f09-4096-b4c2-1d7eaba1d6a1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=37bd863e37e049eb839a7b6286d37090, user_id=47a02b245543433babd23d2bb41347f8, role_id=6c99f5d9f1f243eaa134d8540f21095d)` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:58:41.489744 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d80a6c6a-8f09-4096-b4c2-1d7eaba1d6a1 admin admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:58:41.509454 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 1247/2494] 10.222.0.13 () {60 vars in 1483 bytes} [Wed Aug 31 09:58:41 2022] PUT /identity/v3/projects/37bd863e37e049eb839a7b6286d37090/users/47a02b245543433babd23d2bb41347f8/roles/6c99f5d9f1f243eaa134d8540f21095d => generated 0 bytes in 78 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:58:41.529788 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a820e0b5-d843-4ba5-bc22-55d0e0c0bacc None None] REQUEST_METHOD: `POST` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:58:41.530210 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a820e0b5-d843-4ba5-bc22-55d0e0c0bacc None None] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:58:41.530622 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a820e0b5-d843-4ba5-bc22-55d0e0c0bacc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:58:41.577982 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.auth.core [None req-a820e0b5-d843-4ba5-bc22-55d0e0c0bacc None None] MFA Rules not processed for user `47a02b245543433babd23d2bb41347f8`. Rule list: `[]` (Enabled: `True`). {{(pid=88114) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:58:41.580774 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.fernet_utils [None req-a820e0b5-d843-4ba5-bc22-55d0e0c0bacc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88114) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:58:41.691309 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 1248/2495] 10.222.0.13 () {58 vars in 958 bytes} [Wed Aug 31 09:58:41 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:58:41.697196 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d80a6c6a-8f09-4096-b4c2-1d7eaba1d6a1 admin admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:58:41.800479 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d80a6c6a-8f09-4096-b4c2-1d7eaba1d6a1 admin admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:58:41.814173 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de4b00a-443e-479e-a800-900ca9be8740 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:58:41.815115 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de4b00a-443e-479e-a800-900ca9be8740 admin admin] REQUEST_METHOD: `POST` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:58:41.815592 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de4b00a-443e-479e-a800-900ca9be8740 admin admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:58:41.816837 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de4b00a-443e-479e-a800-900ca9be8740 admin admin] PATH_INFO: `/v3/projects` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:58:41.817849 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1de4b00a-443e-479e-a800-900ca9be8740 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:58:41.820241 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1de4b00a-443e-479e-a800-900ca9be8740 admin admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:58:41.877103 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 1248/2496] 10.222.0.13 () {60 vars in 1153 bytes} [Wed Aug 31 09:58:41 2022] POST /identity/v3/projects => generated 367 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 09:58:41.885150 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a820e0b5-d843-4ba5-bc22-55d0e0c0bacc None None] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:58:41.890328 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f292b7-5962-4a8a-a964-6299f6e8105c None None] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:58:41.890771 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f292b7-5962-4a8a-a964-6299f6e8105c None None] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:58:41.891206 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f292b7-5962-4a8a-a964-6299f6e8105c None None] PATH_INFO: `/` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:58:41.893061 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 1249/2497] 10.222.0.13 () {54 vars in 921 bytes} [Wed Aug 31 09:58:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 31 09:58:41.904802 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b963d9d6-98e7-4a8a-9f72-d152924fd1a9 None None] REQUEST_METHOD: `POST` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:58:41.905638 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b963d9d6-98e7-4a8a-9f72-d152924fd1a9 None None] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:58:41.906196 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b963d9d6-98e7-4a8a-9f72-d152924fd1a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:58:41.909535 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a820e0b5-d843-4ba5-bc22-55d0e0c0bacc None None] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:58:41.935536 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:58:41.979356 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.auth.core [None req-b963d9d6-98e7-4a8a-9f72-d152924fd1a9 None None] MFA Rules not processed for user `dc076493239d4ccdbdc37c0369c49529`. Rule list: `[]` (Enabled: `True`). {{(pid=88113) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:58:41.980785 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.fernet_utils [None req-b963d9d6-98e7-4a8a-9f72-d152924fd1a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:58:42.058790 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef9459a-1b47-4975-839e-5b71520ac032 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:58:42.060711 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef9459a-1b47-4975-839e-5b71520ac032 admin admin] REQUEST_METHOD: `POST` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:58:42.062042 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef9459a-1b47-4975-839e-5b71520ac032 admin admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:58:42.062781 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef9459a-1b47-4975-839e-5b71520ac032 admin admin] PATH_INFO: `/v3/users` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:58:42.063902 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fef9459a-1b47-4975-839e-5b71520ac032 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:58:42.067276 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fef9459a-1b47-4975-839e-5b71520ac032 admin admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:58:42.107899 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 1249/2498] 10.222.0.13 () {60 vars in 1144 bytes} [Wed Aug 31 09:58:41 2022] POST /identity/v3/users => generated 290 bytes in 223 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 09:58:42.109799 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef9459a-1b47-4975-839e-5b71520ac032 admin admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:58:42.122891 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef9459a-1b47-4975-839e-5b71520ac032 admin admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:58:42.130504 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b77c91-7ac8-4e5f-941d-4ebc6e17304f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:58:42.131133 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b77c91-7ac8-4e5f-941d-4ebc6e17304f admin admin] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:58:42.131412 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b77c91-7ac8-4e5f-941d-4ebc6e17304f admin admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:58:42.131684 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b77c91-7ac8-4e5f-941d-4ebc6e17304f admin admin] PATH_INFO: `/v3/roles` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:58:42.132741 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40b77c91-7ac8-4e5f-941d-4ebc6e17304f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:58:42.134762 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40b77c91-7ac8-4e5f-941d-4ebc6e17304f admin admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:58:42.139827 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 1250/2499] 10.222.0.13 () {58 vars in 1018 bytes} [Wed Aug 31 09:58:41 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 237 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 09:58:42.148209 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b963d9d6-98e7-4a8a-9f72-d152924fd1a9 None None] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:58:42.148396 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 1250/2500] 10.222.0.13 () {58 vars in 1122 bytes} [Wed Aug 31 09:58:42 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 09:58:42.151660 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.fernet_utils [None req-b963d9d6-98e7-4a8a-9f72-d152924fd1a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:58:42.153850 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b77c91-7ac8-4e5f-941d-4ebc6e17304f admin admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:58:42.167586 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b77c91-7ac8-4e5f-941d-4ebc6e17304f admin admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:58:42.174135 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a406faae-8b8b-4afa-b343-b884019db6e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:58:42.174979 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a406faae-8b8b-4afa-b343-b884019db6e9 admin admin] REQUEST_METHOD: `PUT` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:58:42.175326 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a406faae-8b8b-4afa-b343-b884019db6e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:58:42.176187 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a406faae-8b8b-4afa-b343-b884019db6e9 admin admin] PATH_INFO: `/v3/projects/39a44ffc3d9e422c80bdbf19bd970be4/users/b069d8e2f0f5471484fa189202590464/roles/5d4ee63a3add47fc9a6b8b16d39e0a1f` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:58:42.195037 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a406faae-8b8b-4afa-b343-b884019db6e9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=39a44ffc3d9e422c80bdbf19bd970be4, user_id=b069d8e2f0f5471484fa189202590464, role_id=5d4ee63a3add47fc9a6b8b16d39e0a1f)` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:58:42.198692 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a406faae-8b8b-4afa-b343-b884019db6e9 admin admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:58:42.217588 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 1251/2501] 10.222.0.13 () {60 vars in 1483 bytes} [Wed Aug 31 09:58:42 2022] PUT /identity/v3/projects/39a44ffc3d9e422c80bdbf19bd970be4/users/b069d8e2f0f5471484fa189202590464/roles/5d4ee63a3add47fc9a6b8b16d39e0a1f => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 09:58:42.222744 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a406faae-8b8b-4afa-b343-b884019db6e9 admin admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:58:42.288985 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:58:42.302254 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b963d9d6-98e7-4a8a-9f72-d152924fd1a9 None None] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:58:42.351170 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a406faae-8b8b-4afa-b343-b884019db6e9 admin admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:58:42.359988 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-592800df-81b7-47cd-b3e1-7f0eeb03b218 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:58:42.360720 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592800df-81b7-47cd-b3e1-7f0eeb03b218 admin admin] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:58:42.361186 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592800df-81b7-47cd-b3e1-7f0eeb03b218 admin admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:58:42.361681 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592800df-81b7-47cd-b3e1-7f0eeb03b218 admin admin] PATH_INFO: `/v3/roles` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:58:42.362824 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-592800df-81b7-47cd-b3e1-7f0eeb03b218 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:58:42.367422 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-592800df-81b7-47cd-b3e1-7f0eeb03b218 admin admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:58:42.388221 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 1252/2502] 10.222.0.13 () {58 vars in 1122 bytes} [Wed Aug 31 09:58:42 2022] GET /identity/v3/roles => generated 1420 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 09:58:42.391719 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-592800df-81b7-47cd-b3e1-7f0eeb03b218 admin admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:58:42.404720 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-592800df-81b7-47cd-b3e1-7f0eeb03b218 admin admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:58:42.411184 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e227be-a115-436e-84cc-41392b55b64a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:58:42.411972 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e227be-a115-436e-84cc-41392b55b64a admin admin] REQUEST_METHOD: `PUT` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:58:42.412306 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e227be-a115-436e-84cc-41392b55b64a admin admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:58:42.412679 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e227be-a115-436e-84cc-41392b55b64a admin admin] PATH_INFO: `/v3/projects/39a44ffc3d9e422c80bdbf19bd970be4/users/b069d8e2f0f5471484fa189202590464/roles/5d4ee63a3add47fc9a6b8b16d39e0a1f` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:58:42.413924 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e29de747-ed7e-42e3-a2ec-52b6bfc9bab9 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be216d156b54452cbbf978c7734343aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '49548eb44d62415582af820ab66decee', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:58:42.414999 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29de747-ed7e-42e3-a2ec-52b6bfc9bab9 service glance] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:58:42.415201 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02e227be-a115-436e-84cc-41392b55b64a admin admin] RBAC: Authorizing `identity:create_grant(project_id=39a44ffc3d9e422c80bdbf19bd970be4, user_id=b069d8e2f0f5471484fa189202590464, role_id=5d4ee63a3add47fc9a6b8b16d39e0a1f)` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:58:42.415487 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29de747-ed7e-42e3-a2ec-52b6bfc9bab9 service glance] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:58:42.415969 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29de747-ed7e-42e3-a2ec-52b6bfc9bab9 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:58:42.418173 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02e227be-a115-436e-84cc-41392b55b64a admin admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:58:42.434367 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 1253/2503] 10.222.0.13 () {60 vars in 1483 bytes} [Wed Aug 31 09:58:42 2022] PUT /identity/v3/projects/39a44ffc3d9e422c80bdbf19bd970be4/users/b069d8e2f0f5471484fa189202590464/roles/5d4ee63a3add47fc9a6b8b16d39e0a1f => generated 0 bytes in 45 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:58:42.446092 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6618bbd-c8b7-48b7-ae02-1aa1c0191ae4 None None] REQUEST_METHOD: `POST` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:58:42.446395 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6618bbd-c8b7-48b7-ae02-1aa1c0191ae4 None None] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:58:42.446649 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6618bbd-c8b7-48b7-ae02-1aa1c0191ae4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:58:42.496387 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.auth.core [None req-b6618bbd-c8b7-48b7-ae02-1aa1c0191ae4 None None] MFA Rules not processed for user `b069d8e2f0f5471484fa189202590464`. Rule list: `[]` (Enabled: `True`). {{(pid=88114) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:58:42.498652 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.fernet_utils [None req-b6618bbd-c8b7-48b7-ae02-1aa1c0191ae4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88114) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:58:42.571433 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e29de747-ed7e-42e3-a2ec-52b6bfc9bab9 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:58:42.574046 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e29de747-ed7e-42e3-a2ec-52b6bfc9bab9 service glance] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:58:42.588165 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:58:42.601511 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 1251/2504] 10.222.0.13 () {60 vars in 1365 bytes} [Wed Aug 31 09:58:42 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 454 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:58:42.655747 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 1254/2505] 10.222.0.13 () {58 vars in 958 bytes} [Wed Aug 31 09:58:42 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 09:58:42.670121 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e29de747-ed7e-42e3-a2ec-52b6bfc9bab9 service glance] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:58:42.673629 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.fernet_utils [None req-e29de747-ed7e-42e3-a2ec-52b6bfc9bab9 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:58:42.855837 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e29de747-ed7e-42e3-a2ec-52b6bfc9bab9 service glance] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:58:42.969190 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=100 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:58:43.010384 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d8f6af6-518a-4253-9ec5-799b4d1b4c5d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d52b69c8e94441dacfcff944326f169', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '49548eb44d62415582af820ab66decee', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:58:43.011405 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8f6af6-518a-4253-9ec5-799b4d1b4c5d service cinder] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:58:43.011847 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8f6af6-518a-4253-9ec5-799b4d1b4c5d service cinder] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:58:43.012477 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8f6af6-518a-4253-9ec5-799b4d1b4c5d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:58:43.017124 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6618bbd-c8b7-48b7-ae02-1aa1c0191ae4 None None] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:58:43.020617 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.fernet_utils [None req-b6618bbd-c8b7-48b7-ae02-1aa1c0191ae4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88114) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:58:43.037105 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d8f6af6-518a-4253-9ec5-799b4d1b4c5d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:58:43.038717 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d8f6af6-518a-4253-9ec5-799b4d1b4c5d service cinder] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:58:43.052480 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 1252/2506] 10.222.0.13 () {60 vars in 1365 bytes} [Wed Aug 31 09:58:42 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 384 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:58:43.125258 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d8f6af6-518a-4253-9ec5-799b4d1b4c5d service cinder] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:58:43.128582 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.fernet_utils [None req-0d8f6af6-518a-4253-9ec5-799b4d1b4c5d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:58:43.152839 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d8f6af6-518a-4253-9ec5-799b4d1b4c5d service cinder] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:58:43.160291 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6618bbd-c8b7-48b7-ae02-1aa1c0191ae4 None None] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:58:43.160873 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d48f5f2-add9-43bb-9093-42d63c9dbc64 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d52b69c8e94441dacfcff944326f169', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '49548eb44d62415582af820ab66decee', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:58:43.162063 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d48f5f2-add9-43bb-9093-42d63c9dbc64 service cinder] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:58:43.162654 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d48f5f2-add9-43bb-9093-42d63c9dbc64 service cinder] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:58:43.163147 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d48f5f2-add9-43bb-9093-42d63c9dbc64 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:58:43.170364 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-132c887d-93f2-433a-b2d0-9f4655092ecc service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ff3aff5ef444910be6dbb989d0e3194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '49548eb44d62415582af820ab66decee', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:58:43.171640 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132c887d-93f2-433a-b2d0-9f4655092ecc service placement] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:58:43.172183 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132c887d-93f2-433a-b2d0-9f4655092ecc service placement] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:58:43.172653 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132c887d-93f2-433a-b2d0-9f4655092ecc service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:58:43.177228 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.fernet_utils [None req-132c887d-93f2-433a-b2d0-9f4655092ecc service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88114) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:58:43.205577 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-132c887d-93f2-433a-b2d0-9f4655092ecc service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:58:43.209279 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-132c887d-93f2-433a-b2d0-9f4655092ecc service placement] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:58:43.227654 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 1255/2507] 10.222.0.13 () {60 vars in 1365 bytes} [Wed Aug 31 09:58:43 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 214 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:58:43.312325 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d48f5f2-add9-43bb-9093-42d63c9dbc64 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:58:43.315665 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d48f5f2-add9-43bb-9093-42d63c9dbc64 service cinder] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:58:43.352803 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 1253/2508] 10.222.0.13 () {60 vars in 1365 bytes} [Wed Aug 31 09:58:43 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 229 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 09:58:45.289304 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-132c887d-93f2-433a-b2d0-9f4655092ecc service placement] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:58:45.302222 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-132c887d-93f2-433a-b2d0-9f4655092ecc service placement] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:58:45.320107 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b064e137-8d4a-4d1d-9269-101daac80380 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ff3aff5ef444910be6dbb989d0e3194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '49548eb44d62415582af820ab66decee', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:58:45.321486 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b064e137-8d4a-4d1d-9269-101daac80380 service placement] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:58:45.322467 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b064e137-8d4a-4d1d-9269-101daac80380 service placement] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:58:45.323298 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b064e137-8d4a-4d1d-9269-101daac80380 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:58:45.327180 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.fernet_utils [None req-b064e137-8d4a-4d1d-9269-101daac80380 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88114) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:58:45.351368 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b064e137-8d4a-4d1d-9269-101daac80380 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:58:45.355346 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b064e137-8d4a-4d1d-9269-101daac80380 service placement] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:58:45.380743 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 1256/2509] 10.222.0.13 () {60 vars in 1365 bytes} [Wed Aug 31 09:58:45 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:58:47.054768 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384b2739-38d9-4e70-a92b-baa2dc288933 None None] REQUEST_METHOD: `POST` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:58:47.055469 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384b2739-38d9-4e70-a92b-baa2dc288933 None None] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:58:47.055469 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384b2739-38d9-4e70-a92b-baa2dc288933 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:58:47.080626 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.auth.core [None req-384b2739-38d9-4e70-a92b-baa2dc288933 None None] MFA Rules not processed for user `768cb67c1ef24251aaa113c12d81a250`. Rule list: `[]` (Enabled: `True`). {{(pid=88113) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:58:47.082039 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.fernet_utils [None req-384b2739-38d9-4e70-a92b-baa2dc288933 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:58:47.172895 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 1254/2510] 10.222.0.13 () {58 vars in 1009 bytes} [Wed Aug 31 09:58:47 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 09:58:47.194596 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b064e137-8d4a-4d1d-9269-101daac80380 service placement] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:58:47.197295 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.fernet_utils [None req-b064e137-8d4a-4d1d-9269-101daac80380 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88114) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:58:47.341420 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b064e137-8d4a-4d1d-9269-101daac80380 service placement] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:58:47.352007 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d756a5f1-d6aa-4028-ba73-e9cf1890405b service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '202bb9db7a8a4977ba788a290ed9a3fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '49548eb44d62415582af820ab66decee', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:58:47.353170 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d756a5f1-d6aa-4028-ba73-e9cf1890405b service swift] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:58:47.353774 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d756a5f1-d6aa-4028-ba73-e9cf1890405b service swift] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:58:47.354077 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d756a5f1-d6aa-4028-ba73-e9cf1890405b service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:58:47.376169 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d756a5f1-d6aa-4028-ba73-e9cf1890405b service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:58:47.377841 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d756a5f1-d6aa-4028-ba73-e9cf1890405b service swift] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:58:47.387345 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 1257/2511] 10.222.0.13 () {60 vars in 1384 bytes} [Wed Aug 31 09:58:47 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 194 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 31 09:58:48.021024 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-384b2739-38d9-4e70-a92b-baa2dc288933 None None] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:58:48.128446 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-384b2739-38d9-4e70-a92b-baa2dc288933 None None] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:58:48.135125 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f4eacb-4c9f-4a7e-9b9b-783874a2e6ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:58:48.136400 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f4eacb-4c9f-4a7e-9b9b-783874a2e6ad admin admin] REQUEST_METHOD: `DELETE` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:58:48.136889 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f4eacb-4c9f-4a7e-9b9b-783874a2e6ad admin admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:58:48.137454 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f4eacb-4c9f-4a7e-9b9b-783874a2e6ad admin admin] PATH_INFO: `/v3/users/47a02b245543433babd23d2bb41347f8` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:58:48.139360 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8f4eacb-4c9f-4a7e-9b9b-783874a2e6ad admin admin] RBAC: Authorizing `identity:delete_user(user_id=47a02b245543433babd23d2bb41347f8)` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:58:48.142143 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8f4eacb-4c9f-4a7e-9b9b-783874a2e6ad admin admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:58:48.190433 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:58:48.271696 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.notifications [None req-c8f4eacb-4c9f-4a7e-9b9b-783874a2e6ad admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '47a02b245543433babd23d2bb41347f8'} {{(pid=88113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:58:48.288909 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.notifications [None req-c8f4eacb-4c9f-4a7e-9b9b-783874a2e6ad admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '47a02b245543433babd23d2bb41347f8'} {{(pid=88113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:58:48.317794 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.notifications [None req-c8f4eacb-4c9f-4a7e-9b9b-783874a2e6ad admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '47a02b245543433babd23d2bb41347f8'} {{(pid=88113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:58:48.319308 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.notifications [None req-c8f4eacb-4c9f-4a7e-9b9b-783874a2e6ad admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '47a02b245543433babd23d2bb41347f8'} {{(pid=88113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:58:48.337217 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.notifications [None req-c8f4eacb-4c9f-4a7e-9b9b-783874a2e6ad admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '47a02b245543433babd23d2bb41347f8'} {{(pid=88113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:58:48.347314 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 1255/2512] 10.222.0.13 () {58 vars in 1224 bytes} [Wed Aug 31 09:58:48 2022] DELETE /identity/v3/users/47a02b245543433babd23d2bb41347f8 => generated 0 bytes in 327 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:58:48.352695 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e509f548-3d71-48a6-a505-105b7166144f None None] REQUEST_METHOD: `POST` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:58:48.353276 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e509f548-3d71-48a6-a505-105b7166144f None None] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:58:48.353619 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e509f548-3d71-48a6-a505-105b7166144f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:58:48.436925 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.auth.core [None req-e509f548-3d71-48a6-a505-105b7166144f None None] MFA Rules not processed for user `fd836cf590a145df9acf19cf3ae2ac56`. Rule list: `[]` (Enabled: `True`). {{(pid=88114) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:58:48.439536 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.fernet_utils [None req-e509f548-3d71-48a6-a505-105b7166144f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88114) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:58:48.612471 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=100 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:58:48.649855 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 1258/2513] 10.222.0.13 () {58 vars in 958 bytes} [Wed Aug 31 09:58:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 295 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 09:58:48.662869 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f4eacb-4c9f-4a7e-9b9b-783874a2e6ad admin admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:58:48.667027 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.fernet_utils [None req-c8f4eacb-4c9f-4a7e-9b9b-783874a2e6ad admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:58:48.863578 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f4eacb-4c9f-4a7e-9b9b-783874a2e6ad admin admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:58:48.876153 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0366ac29-0a3d-44c1-83c2-ec2f6bc312cb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9bdaacd597442c2bcde2c555f2cc140', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '49548eb44d62415582af820ab66decee', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:58:48.876913 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0366ac29-0a3d-44c1-83c2-ec2f6bc312cb service neutron] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:58:48.877195 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0366ac29-0a3d-44c1-83c2-ec2f6bc312cb service neutron] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:58:48.877498 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0366ac29-0a3d-44c1-83c2-ec2f6bc312cb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:58:48.897624 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0366ac29-0a3d-44c1-83c2-ec2f6bc312cb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:58:48.899616 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0366ac29-0a3d-44c1-83c2-ec2f6bc312cb service neutron] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:58:48.914867 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 1256/2514] 10.222.0.13 () {60 vars in 1365 bytes} [Wed Aug 31 09:58:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 255 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:58:49.848181 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e509f548-3d71-48a6-a505-105b7166144f None None] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:58:49.850330 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.fernet_utils [None req-e509f548-3d71-48a6-a505-105b7166144f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88114) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:58:49.878313 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e509f548-3d71-48a6-a505-105b7166144f None None] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:58:49.887527 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14dde76f-7494-4959-8670-591bdb295858 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:58:49.888693 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14dde76f-7494-4959-8670-591bdb295858 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:58:49.889788 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14dde76f-7494-4959-8670-591bdb295858 admin admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:58:49.890614 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14dde76f-7494-4959-8670-591bdb295858 admin admin] PATH_INFO: `/v3/projects/37bd863e37e049eb839a7b6286d37090` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:58:49.912675 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14dde76f-7494-4959-8670-591bdb295858 admin admin] RBAC: Authorizing `identity:delete_project(project_id=37bd863e37e049eb839a7b6286d37090)` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:58:49.923029 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14dde76f-7494-4959-8670-591bdb295858 admin admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:58:50.023220 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.notifications [None req-14dde76f-7494-4959-8670-591bdb295858 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '37bd863e37e049eb839a7b6286d37090', 'request_id': 'req-14dde76f-7494-4959-8670-591bdb295858'} {{(pid=88114) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:58:50.037374 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.notifications [None req-14dde76f-7494-4959-8670-591bdb295858 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '37bd863e37e049eb839a7b6286d37090', 'request_id': 'req-14dde76f-7494-4959-8670-591bdb295858'} {{(pid=88114) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:58:50.050069 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.notifications [None req-14dde76f-7494-4959-8670-591bdb295858 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '37bd863e37e049eb839a7b6286d37090', 'request_id': 'req-14dde76f-7494-4959-8670-591bdb295858'} {{(pid=88114) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:58:50.101976 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.notifications [None req-14dde76f-7494-4959-8670-591bdb295858 admin admin] The token cache is being invalidate because project 37bd863e37e049eb839a7b6286d37090 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88114) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 09:58:50.102461 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.notifications [None req-14dde76f-7494-4959-8670-591bdb295858 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88114) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:58:50.103791 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.notifications [None req-14dde76f-7494-4959-8670-591bdb295858 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88114) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:58:50.109485 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 1259/2515] 10.222.0.13 () {58 vars in 1233 bytes} [Wed Aug 31 09:58:49 2022] DELETE /identity/v3/projects/37bd863e37e049eb839a7b6286d37090 => generated 0 bytes in 262 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:58:50.113749 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0366ac29-0a3d-44c1-83c2-ec2f6bc312cb service neutron] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:58:50.138842 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0366ac29-0a3d-44c1-83c2-ec2f6bc312cb service neutron] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:58:50.299594 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67540363-ca67-4977-ae8d-51a716ccabea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:58:50.314130 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67540363-ca67-4977-ae8d-51a716ccabea admin admin] REQUEST_METHOD: `DELETE` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:58:50.315096 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67540363-ca67-4977-ae8d-51a716ccabea admin admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:58:50.315755 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67540363-ca67-4977-ae8d-51a716ccabea admin admin] PATH_INFO: `/v3/users/b069d8e2f0f5471484fa189202590464` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:58:50.318275 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67540363-ca67-4977-ae8d-51a716ccabea admin admin] RBAC: Authorizing `identity:delete_user(user_id=b069d8e2f0f5471484fa189202590464)` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:58:50.320540 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67540363-ca67-4977-ae8d-51a716ccabea admin admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:58:50.427072 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.notifications [None req-67540363-ca67-4977-ae8d-51a716ccabea admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b069d8e2f0f5471484fa189202590464'} {{(pid=88113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:58:50.432081 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:58:50.451055 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.notifications [None req-67540363-ca67-4977-ae8d-51a716ccabea admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b069d8e2f0f5471484fa189202590464'} {{(pid=88113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:58:50.504191 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.notifications [None req-67540363-ca67-4977-ae8d-51a716ccabea admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b069d8e2f0f5471484fa189202590464'} {{(pid=88113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:58:50.505996 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.notifications [None req-67540363-ca67-4977-ae8d-51a716ccabea admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b069d8e2f0f5471484fa189202590464'} {{(pid=88113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:58:50.523448 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.notifications [None req-67540363-ca67-4977-ae8d-51a716ccabea admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b069d8e2f0f5471484fa189202590464'} {{(pid=88113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:58:50.542126 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 1257/2516] 10.222.0.13 () {58 vars in 1224 bytes} [Wed Aug 31 09:58:50 2022] DELETE /identity/v3/users/b069d8e2f0f5471484fa189202590464 => generated 0 bytes in 425 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 09:58:51.480599 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14dde76f-7494-4959-8670-591bdb295858 admin admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:58:51.482765 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.fernet_utils [None req-14dde76f-7494-4959-8670-591bdb295858 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88114) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:58:51.559802 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14dde76f-7494-4959-8670-591bdb295858 admin admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:58:51.735693 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6171b4fb-3dc8-4da3-bcd9-5cbf6f3f274a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:58:51.736364 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6171b4fb-3dc8-4da3-bcd9-5cbf6f3f274a admin admin] REQUEST_METHOD: `DELETE` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:58:51.736755 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6171b4fb-3dc8-4da3-bcd9-5cbf6f3f274a admin admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:58:51.737182 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6171b4fb-3dc8-4da3-bcd9-5cbf6f3f274a admin admin] PATH_INFO: `/v3/projects/39a44ffc3d9e422c80bdbf19bd970be4` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:58:51.739366 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6171b4fb-3dc8-4da3-bcd9-5cbf6f3f274a admin admin] RBAC: Authorizing `identity:delete_project(project_id=39a44ffc3d9e422c80bdbf19bd970be4)` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:58:51.744941 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6171b4fb-3dc8-4da3-bcd9-5cbf6f3f274a admin admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:58:51.746881 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67540363-ca67-4977-ae8d-51a716ccabea admin admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:58:51.749082 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.fernet_utils [None req-67540363-ca67-4977-ae8d-51a716ccabea admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:58:51.766824 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67540363-ca67-4977-ae8d-51a716ccabea admin admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:58:51.777374 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e076459-6513-47ae-b372-578bafbba968 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:58:51.778229 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e076459-6513-47ae-b372-578bafbba968 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:58:51.778734 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e076459-6513-47ae-b372-578bafbba968 admin admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:58:51.779105 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e076459-6513-47ae-b372-578bafbba968 admin admin] PATH_INFO: `/v3/users/c89dbe311b24408c925f102eefd6d8c3` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:58:51.780848 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e076459-6513-47ae-b372-578bafbba968 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c89dbe311b24408c925f102eefd6d8c3)` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:58:51.783214 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e076459-6513-47ae-b372-578bafbba968 admin admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:58:51.836215 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.notifications [None req-6171b4fb-3dc8-4da3-bcd9-5cbf6f3f274a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '39a44ffc3d9e422c80bdbf19bd970be4', 'request_id': 'req-6171b4fb-3dc8-4da3-bcd9-5cbf6f3f274a'} {{(pid=88114) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:58:51.859801 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.notifications [None req-6171b4fb-3dc8-4da3-bcd9-5cbf6f3f274a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '39a44ffc3d9e422c80bdbf19bd970be4', 'request_id': 'req-6171b4fb-3dc8-4da3-bcd9-5cbf6f3f274a'} {{(pid=88114) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:58:51.870132 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.notifications [None req-6171b4fb-3dc8-4da3-bcd9-5cbf6f3f274a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '39a44ffc3d9e422c80bdbf19bd970be4', 'request_id': 'req-6171b4fb-3dc8-4da3-bcd9-5cbf6f3f274a'} {{(pid=88114) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:58:51.884611 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:58:51.891475 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.notifications [None req-6171b4fb-3dc8-4da3-bcd9-5cbf6f3f274a admin admin] The token cache is being invalidate because project 39a44ffc3d9e422c80bdbf19bd970be4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88114) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 09:58:51.892348 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.notifications [None req-6171b4fb-3dc8-4da3-bcd9-5cbf6f3f274a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88114) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:58:51.893691 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.notifications [None req-6171b4fb-3dc8-4da3-bcd9-5cbf6f3f274a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88114) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:58:51.898242 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 1260/2517] 10.222.0.13 () {58 vars in 1233 bytes} [Wed Aug 31 09:58:51 2022] DELETE /identity/v3/projects/39a44ffc3d9e422c80bdbf19bd970be4 => generated 0 bytes in 418 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:58:51.903472 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932b63e8-cdf6-49e6-96ad-e19471afbbde None None] REQUEST_METHOD: `POST` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:58:51.903762 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932b63e8-cdf6-49e6-96ad-e19471afbbde None None] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:58:51.904035 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932b63e8-cdf6-49e6-96ad-e19471afbbde None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:58:51.904648 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.notifications [None req-1e076459-6513-47ae-b372-578bafbba968 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c89dbe311b24408c925f102eefd6d8c3'} {{(pid=88113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:58:51.926533 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.notifications [None req-1e076459-6513-47ae-b372-578bafbba968 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c89dbe311b24408c925f102eefd6d8c3'} {{(pid=88113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:58:51.948912 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.auth.core [None req-932b63e8-cdf6-49e6-96ad-e19471afbbde None None] MFA Rules not processed for user `fd836cf590a145df9acf19cf3ae2ac56`. Rule list: `[]` (Enabled: `True`). {{(pid=88114) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:58:51.951430 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.fernet_utils [None req-932b63e8-cdf6-49e6-96ad-e19471afbbde None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88114) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:58:51.952821 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.notifications [None req-1e076459-6513-47ae-b372-578bafbba968 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c89dbe311b24408c925f102eefd6d8c3'} {{(pid=88113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:58:51.954171 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.notifications [None req-1e076459-6513-47ae-b372-578bafbba968 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c89dbe311b24408c925f102eefd6d8c3'} {{(pid=88113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:58:51.973162 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.notifications [None req-1e076459-6513-47ae-b372-578bafbba968 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c89dbe311b24408c925f102eefd6d8c3'} {{(pid=88113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:58:51.993899 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 1258/2518] 10.222.0.13 () {58 vars in 1224 bytes} [Wed Aug 31 09:58:51 2022] DELETE /identity/v3/users/c89dbe311b24408c925f102eefd6d8c3 => generated 0 bytes in 248 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:58:52.123859 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 1261/2519] 10.222.0.13 () {58 vars in 958 bytes} [Wed Aug 31 09:58:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 222 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:58:52.147486 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f2459a-9626-4995-90ae-a8e9b79792ad None None] REQUEST_METHOD: `POST` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:58:52.148013 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f2459a-9626-4995-90ae-a8e9b79792ad None None] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:58:52.148771 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f2459a-9626-4995-90ae-a8e9b79792ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:58:52.210188 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.auth.core [None req-24f2459a-9626-4995-90ae-a8e9b79792ad None None] MFA Rules not processed for user `fd836cf590a145df9acf19cf3ae2ac56`. Rule list: `[]` (Enabled: `True`). {{(pid=88113) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:58:52.211520 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.fernet_utils [None req-24f2459a-9626-4995-90ae-a8e9b79792ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:58:52.350908 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 1259/2520] 10.222.0.13 () {58 vars in 958 bytes} [Wed Aug 31 09:58:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 219 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 09:58:52.354862 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-932b63e8-cdf6-49e6-96ad-e19471afbbde None None] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:58:52.358352 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f2459a-9626-4995-90ae-a8e9b79792ad None None] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:58:52.360165 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.fernet_utils [None req-24f2459a-9626-4995-90ae-a8e9b79792ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:58:52.376974 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f2459a-9626-4995-90ae-a8e9b79792ad None None] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:58:52.382049 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-932b63e8-cdf6-49e6-96ad-e19471afbbde None None] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:58:52.383941 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-873edf88-e736-428d-a6d9-dd9336dae9ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:58:52.384607 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873edf88-e736-428d-a6d9-dd9336dae9ef admin admin] REQUEST_METHOD: `DELETE` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:58:52.384943 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873edf88-e736-428d-a6d9-dd9336dae9ef admin admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:58:52.385185 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873edf88-e736-428d-a6d9-dd9336dae9ef admin admin] PATH_INFO: `/v3/projects/b058d5f34eee47a58629407e166f3499` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:58:52.387528 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3116a66-1783-420d-a674-8071adee6b9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:58:52.388390 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3116a66-1783-420d-a674-8071adee6b9b admin admin] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:58:52.388866 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3116a66-1783-420d-a674-8071adee6b9b admin admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:58:52.389163 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3116a66-1783-420d-a674-8071adee6b9b admin admin] PATH_INFO: `/v3/domains` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:58:52.390137 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3116a66-1783-420d-a674-8071adee6b9b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88114) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:58:52.390452 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3116a66-1783-420d-a674-8071adee6b9b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:58:52.398868 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-873edf88-e736-428d-a6d9-dd9336dae9ef admin admin] RBAC: Authorizing `identity:delete_project(project_id=b058d5f34eee47a58629407e166f3499)` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:58:52.401752 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-873edf88-e736-428d-a6d9-dd9336dae9ef admin admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:58:52.405833 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3116a66-1783-420d-a674-8071adee6b9b admin admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:58:52.431930 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 1262/2521] 10.222.0.13 () {58 vars in 1153 bytes} [Wed Aug 31 09:58:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:58:52.447843 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3116a66-1783-420d-a674-8071adee6b9b admin admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:58:52.461115 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3116a66-1783-420d-a674-8071adee6b9b admin admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:58:52.467873 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec8df882-04cb-4a8e-90de-6e1720747f67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:58:52.469178 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8df882-04cb-4a8e-90de-6e1720747f67 admin admin] REQUEST_METHOD: `POST` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:58:52.469596 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8df882-04cb-4a8e-90de-6e1720747f67 admin admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:58:52.470167 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8df882-04cb-4a8e-90de-6e1720747f67 admin admin] PATH_INFO: `/v3/projects` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:58:52.471063 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec8df882-04cb-4a8e-90de-6e1720747f67 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:58:52.474247 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec8df882-04cb-4a8e-90de-6e1720747f67 admin admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:58:52.475724 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:58:52.496742 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.notifications [None req-873edf88-e736-428d-a6d9-dd9336dae9ef admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b058d5f34eee47a58629407e166f3499', 'request_id': 'req-873edf88-e736-428d-a6d9-dd9336dae9ef'} {{(pid=88113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:58:52.503649 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.notifications [None req-873edf88-e736-428d-a6d9-dd9336dae9ef admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b058d5f34eee47a58629407e166f3499', 'request_id': 'req-873edf88-e736-428d-a6d9-dd9336dae9ef'} {{(pid=88113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:58:52.526933 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.notifications [None req-873edf88-e736-428d-a6d9-dd9336dae9ef admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b058d5f34eee47a58629407e166f3499', 'request_id': 'req-873edf88-e736-428d-a6d9-dd9336dae9ef'} {{(pid=88113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:58:52.528475 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 1263/2522] 10.222.0.13 () {60 vars in 1153 bytes} [Wed Aug 31 09:58:52 2022] POST /identity/v3/projects => generated 407 bytes in 81 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 09:58:52.530776 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec8df882-04cb-4a8e-90de-6e1720747f67 admin admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:58:52.559827 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.notifications [None req-873edf88-e736-428d-a6d9-dd9336dae9ef admin admin] The token cache is being invalidate because project b058d5f34eee47a58629407e166f3499 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88113) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 09:58:52.560122 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.notifications [None req-873edf88-e736-428d-a6d9-dd9336dae9ef admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:58:52.561427 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.notifications [None req-873edf88-e736-428d-a6d9-dd9336dae9ef admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:58:52.564471 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 1260/2523] 10.222.0.13 () {58 vars in 1233 bytes} [Wed Aug 31 09:58:52 2022] DELETE /identity/v3/projects/b058d5f34eee47a58629407e166f3499 => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:58:52.575458 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-873edf88-e736-428d-a6d9-dd9336dae9ef admin admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:58:52.577989 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.fernet_utils [None req-873edf88-e736-428d-a6d9-dd9336dae9ef admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:58:52.647816 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec8df882-04cb-4a8e-90de-6e1720747f67 admin admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:58:52.660315 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a2af57-a0ad-4864-89db-1ccdda4e46a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:58:52.661156 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a2af57-a0ad-4864-89db-1ccdda4e46a5 admin admin] REQUEST_METHOD: `POST` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:58:52.661493 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a2af57-a0ad-4864-89db-1ccdda4e46a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:58:52.661772 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a2af57-a0ad-4864-89db-1ccdda4e46a5 admin admin] PATH_INFO: `/v3/users` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:58:52.662428 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55a2af57-a0ad-4864-89db-1ccdda4e46a5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:58:52.664732 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55a2af57-a0ad-4864-89db-1ccdda4e46a5 admin admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:58:52.726073 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-873edf88-e736-428d-a6d9-dd9336dae9ef admin admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:58:52.741971 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd2c56d1-b881-4ead-86f4-fd6a850b7db9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:58:52.744749 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2c56d1-b881-4ead-86f4-fd6a850b7db9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:58:52.744749 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2c56d1-b881-4ead-86f4-fd6a850b7db9 admin admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:58:52.744749 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2c56d1-b881-4ead-86f4-fd6a850b7db9 admin admin] PATH_INFO: `/v3/users/bc507f26f0fd43d0aa1e594eeb86265f` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:58:52.745985 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd2c56d1-b881-4ead-86f4-fd6a850b7db9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bc507f26f0fd43d0aa1e594eeb86265f)` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:58:52.749309 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd2c56d1-b881-4ead-86f4-fd6a850b7db9 admin admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:58:52.818855 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 1264/2524] 10.222.0.13 () {60 vars in 1144 bytes} [Wed Aug 31 09:58:52 2022] POST /identity/v3/users => generated 304 bytes in 289 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 09:58:52.829038 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a2af57-a0ad-4864-89db-1ccdda4e46a5 admin admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:58:52.831581 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.fernet_utils [None req-55a2af57-a0ad-4864-89db-1ccdda4e46a5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88114) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:58:52.850756 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a2af57-a0ad-4864-89db-1ccdda4e46a5 admin admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:58:52.855894 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7435fbc6-7d26-442e-b38a-d6a1d40619f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:58:52.856647 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7435fbc6-7d26-442e-b38a-d6a1d40619f1 admin admin] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:58:52.856924 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7435fbc6-7d26-442e-b38a-d6a1d40619f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:58:52.857281 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7435fbc6-7d26-442e-b38a-d6a1d40619f1 admin admin] PATH_INFO: `/v3/roles` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:58:52.858019 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7435fbc6-7d26-442e-b38a-d6a1d40619f1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:58:52.859882 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7435fbc6-7d26-442e-b38a-d6a1d40619f1 admin admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:58:52.884998 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.notifications [None req-dd2c56d1-b881-4ead-86f4-fd6a850b7db9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bc507f26f0fd43d0aa1e594eeb86265f'} {{(pid=88113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:58:52.891849 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 1265/2525] 10.222.0.13 () {58 vars in 1122 bytes} [Wed Aug 31 09:58:52 2022] GET /identity/v3/roles => generated 1420 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 09:58:52.898965 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7435fbc6-7d26-442e-b38a-d6a1d40619f1 admin admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:58:52.901232 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.notifications [None req-dd2c56d1-b881-4ead-86f4-fd6a850b7db9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bc507f26f0fd43d0aa1e594eeb86265f'} {{(pid=88113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:58:52.918735 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7435fbc6-7d26-442e-b38a-d6a1d40619f1 admin admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:58:52.925528 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e46f6254-8016-421c-bc88-f530773be472 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:58:52.926539 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46f6254-8016-421c-bc88-f530773be472 admin admin] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:58:52.927054 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46f6254-8016-421c-bc88-f530773be472 admin admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:58:52.927509 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46f6254-8016-421c-bc88-f530773be472 admin admin] PATH_INFO: `/v3/roles` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:58:52.928435 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e46f6254-8016-421c-bc88-f530773be472 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:58:52.931759 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e46f6254-8016-421c-bc88-f530773be472 admin admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:58:52.935446 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.notifications [None req-dd2c56d1-b881-4ead-86f4-fd6a850b7db9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bc507f26f0fd43d0aa1e594eeb86265f'} {{(pid=88113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:58:52.938206 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.notifications [None req-dd2c56d1-b881-4ead-86f4-fd6a850b7db9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bc507f26f0fd43d0aa1e594eeb86265f'} {{(pid=88113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:58:52.951984 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 1266/2526] 10.222.0.13 () {58 vars in 1122 bytes} [Wed Aug 31 09:58:52 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 09:58:52.957518 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.notifications [None req-dd2c56d1-b881-4ead-86f4-fd6a850b7db9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bc507f26f0fd43d0aa1e594eeb86265f'} {{(pid=88113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:58:52.958466 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e46f6254-8016-421c-bc88-f530773be472 admin admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:58:52.961762 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.fernet_utils [None req-e46f6254-8016-421c-bc88-f530773be472 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88114) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:58:52.981306 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 1261/2527] 10.222.0.13 () {58 vars in 1224 bytes} [Wed Aug 31 09:58:52 2022] DELETE /identity/v3/users/bc507f26f0fd43d0aa1e594eeb86265f => generated 0 bytes in 396 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:58:52.994078 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e46f6254-8016-421c-bc88-f530773be472 admin admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:58:53.050035 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:58:53.104555 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e81a268-96b0-4cdc-90e4-6017bebae1a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:58:53.105364 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e81a268-96b0-4cdc-90e4-6017bebae1a3 admin admin] REQUEST_METHOD: `PUT` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:58:53.105692 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e81a268-96b0-4cdc-90e4-6017bebae1a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:58:53.105972 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e81a268-96b0-4cdc-90e4-6017bebae1a3 admin admin] PATH_INFO: `/v3/projects/6cd60a0e70124fb4b0aaf707c4e1dd5f/users/6e7d73aef07b44db82b871bc60f6c298/roles/6c99f5d9f1f243eaa134d8540f21095d` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:58:53.129365 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e81a268-96b0-4cdc-90e4-6017bebae1a3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6cd60a0e70124fb4b0aaf707c4e1dd5f, user_id=6e7d73aef07b44db82b871bc60f6c298, role_id=6c99f5d9f1f243eaa134d8540f21095d)` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:58:53.132858 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e81a268-96b0-4cdc-90e4-6017bebae1a3 admin admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:58:53.155592 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 1267/2528] 10.222.0.13 () {60 vars in 1483 bytes} [Wed Aug 31 09:58:52 2022] PUT /identity/v3/projects/6cd60a0e70124fb4b0aaf707c4e1dd5f/users/6e7d73aef07b44db82b871bc60f6c298/roles/6c99f5d9f1f243eaa134d8540f21095d => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:58:53.168109 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c93b405-de2b-4c65-a393-d8c5e021e17a None None] REQUEST_METHOD: `POST` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:58:53.168428 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c93b405-de2b-4c65-a393-d8c5e021e17a None None] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:58:53.168659 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c93b405-de2b-4c65-a393-d8c5e021e17a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:58:53.213804 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.auth.core [None req-8c93b405-de2b-4c65-a393-d8c5e021e17a None None] MFA Rules not processed for user `6e7d73aef07b44db82b871bc60f6c298`. Rule list: `[]` (Enabled: `True`). {{(pid=88113) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:58:53.215262 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.fernet_utils [None req-8c93b405-de2b-4c65-a393-d8c5e021e17a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:58:53.278111 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:DELETE=15,keystone:SELECT=83,keystone:INSERT=2 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:58:53.330543 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e81a268-96b0-4cdc-90e4-6017bebae1a3 admin admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:58:53.332402 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.fernet_utils [None req-5e81a268-96b0-4cdc-90e4-6017bebae1a3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88114) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:58:53.354736 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 1262/2529] 10.222.0.13 () {58 vars in 958 bytes} [Wed Aug 31 09:58:53 2022] POST /identity/v3/auth/tokens => generated 3284 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:58:53.359043 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c93b405-de2b-4c65-a393-d8c5e021e17a None None] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:58:53.517845 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c93b405-de2b-4c65-a393-d8c5e021e17a None None] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:58:53.525710 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dfecdf0-655a-47eb-8036-db1eb0f8ab54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:58:53.526808 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dfecdf0-655a-47eb-8036-db1eb0f8ab54 admin admin] REQUEST_METHOD: `POST` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:58:53.527321 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dfecdf0-655a-47eb-8036-db1eb0f8ab54 admin admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:58:53.527818 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dfecdf0-655a-47eb-8036-db1eb0f8ab54 admin admin] PATH_INFO: `/v3/projects` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:58:53.528895 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dfecdf0-655a-47eb-8036-db1eb0f8ab54 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:58:53.531854 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dfecdf0-655a-47eb-8036-db1eb0f8ab54 admin admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:58:53.581562 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 1263/2530] 10.222.0.13 () {60 vars in 1153 bytes} [Wed Aug 31 09:58:53 2022] POST /identity/v3/projects => generated 407 bytes in 223 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 09:58:53.585970 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e81a268-96b0-4cdc-90e4-6017bebae1a3 admin admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:58:53.588485 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dfecdf0-655a-47eb-8036-db1eb0f8ab54 admin admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:58:53.695746 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0882581e-9fff-421b-9f8f-2128e55a56f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:58:53.696331 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0882581e-9fff-421b-9f8f-2128e55a56f9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:58:53.696946 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0882581e-9fff-421b-9f8f-2128e55a56f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:58:53.696946 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0882581e-9fff-421b-9f8f-2128e55a56f9 admin admin] PATH_INFO: `/v3/projects/061b65fc11484733bb6be383ff807570` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:58:53.699017 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0882581e-9fff-421b-9f8f-2128e55a56f9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=061b65fc11484733bb6be383ff807570)` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:58:53.701556 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0882581e-9fff-421b-9f8f-2128e55a56f9 admin admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:58:53.730961 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dfecdf0-655a-47eb-8036-db1eb0f8ab54 admin admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:58:53.738656 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a33354-8ea8-4919-8817-dec62f1a9ec3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:58:53.740200 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a33354-8ea8-4919-8817-dec62f1a9ec3 admin admin] REQUEST_METHOD: `POST` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:58:53.740804 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a33354-8ea8-4919-8817-dec62f1a9ec3 admin admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:58:53.741586 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a33354-8ea8-4919-8817-dec62f1a9ec3 admin admin] PATH_INFO: `/v3/users` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:58:53.742580 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86a33354-8ea8-4919-8817-dec62f1a9ec3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:58:53.745620 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86a33354-8ea8-4919-8817-dec62f1a9ec3 admin admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:58:53.784515 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.notifications [None req-0882581e-9fff-421b-9f8f-2128e55a56f9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '061b65fc11484733bb6be383ff807570', 'request_id': 'req-0882581e-9fff-421b-9f8f-2128e55a56f9'} {{(pid=88114) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:58:53.790378 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 1264/2531] 10.222.0.13 () {60 vars in 1144 bytes} [Wed Aug 31 09:58:53 2022] POST /identity/v3/users => generated 310 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 09:58:53.795172 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a33354-8ea8-4919-8817-dec62f1a9ec3 admin admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:58:53.801983 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.notifications [None req-0882581e-9fff-421b-9f8f-2128e55a56f9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '061b65fc11484733bb6be383ff807570', 'request_id': 'req-0882581e-9fff-421b-9f8f-2128e55a56f9'} {{(pid=88114) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:58:53.811139 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a33354-8ea8-4919-8817-dec62f1a9ec3 admin admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:58:53.814842 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.notifications [None req-0882581e-9fff-421b-9f8f-2128e55a56f9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '061b65fc11484733bb6be383ff807570', 'request_id': 'req-0882581e-9fff-421b-9f8f-2128e55a56f9'} {{(pid=88114) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:58:53.818363 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=93,keystone:INSERT=1,keystone:DELETE=6 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:58:53.850286 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.notifications [None req-0882581e-9fff-421b-9f8f-2128e55a56f9 admin admin] The token cache is being invalidate because project 061b65fc11484733bb6be383ff807570 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88114) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 09:58:53.850637 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.notifications [None req-0882581e-9fff-421b-9f8f-2128e55a56f9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88114) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:58:53.851465 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.notifications [None req-0882581e-9fff-421b-9f8f-2128e55a56f9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88114) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:58:53.853524 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 1268/2532] 10.222.0.13 () {58 vars in 1233 bytes} [Wed Aug 31 09:58:53 2022] DELETE /identity/v3/projects/061b65fc11484733bb6be383ff807570 => generated 0 bytes in 523 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:58:53.905887 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:58:53.938671 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-071ac847-290b-4557-8108-2f3f8ce9d86b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:58:53.939332 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071ac847-290b-4557-8108-2f3f8ce9d86b admin admin] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:58:53.939613 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071ac847-290b-4557-8108-2f3f8ce9d86b admin admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:58:53.939891 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071ac847-290b-4557-8108-2f3f8ce9d86b admin admin] PATH_INFO: `/v3/roles` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:58:53.940597 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-071ac847-290b-4557-8108-2f3f8ce9d86b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:58:53.943629 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-071ac847-290b-4557-8108-2f3f8ce9d86b admin admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:58:53.960150 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 1265/2533] 10.222.0.13 () {58 vars in 1122 bytes} [Wed Aug 31 09:58:53 2022] GET /identity/v3/roles => generated 1420 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 09:58:53.961937 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0882581e-9fff-421b-9f8f-2128e55a56f9 admin admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:58:53.963898 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.fernet_utils [None req-0882581e-9fff-421b-9f8f-2128e55a56f9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88114) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:58:53.978582 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0882581e-9fff-421b-9f8f-2128e55a56f9 admin admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:58:53.985174 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce14435-c32e-4387-8fa7-1aaa83940db3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:58:53.985915 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce14435-c32e-4387-8fa7-1aaa83940db3 admin admin] REQUEST_METHOD: `PUT` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:58:53.986228 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce14435-c32e-4387-8fa7-1aaa83940db3 admin admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:58:53.986634 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce14435-c32e-4387-8fa7-1aaa83940db3 admin admin] PATH_INFO: `/v3/projects/bca984647ee5413ea5fc84d397b01026/users/dfd3fce40f81403294833f2c1e8656f8/roles/5d4ee63a3add47fc9a6b8b16d39e0a1f` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:58:54.031263 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dce14435-c32e-4387-8fa7-1aaa83940db3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bca984647ee5413ea5fc84d397b01026, user_id=dfd3fce40f81403294833f2c1e8656f8, role_id=5d4ee63a3add47fc9a6b8b16d39e0a1f)` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:58:54.037031 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dce14435-c32e-4387-8fa7-1aaa83940db3 admin admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:58:54.077461 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-071ac847-290b-4557-8108-2f3f8ce9d86b admin admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:58:54.084745 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 1269/2534] 10.222.0.13 () {60 vars in 1483 bytes} [Wed Aug 31 09:58:53 2022] PUT /identity/v3/projects/bca984647ee5413ea5fc84d397b01026/users/dfd3fce40f81403294833f2c1e8656f8/roles/5d4ee63a3add47fc9a6b8b16d39e0a1f => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:58:54.098880 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-071ac847-290b-4557-8108-2f3f8ce9d86b admin admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:58:54.188349 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-224a1129-d38c-4d07-93d5-73f9350917db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:58:54.189195 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224a1129-d38c-4d07-93d5-73f9350917db admin admin] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:58:54.189512 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224a1129-d38c-4d07-93d5-73f9350917db admin admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:58:54.189852 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224a1129-d38c-4d07-93d5-73f9350917db admin admin] PATH_INFO: `/v3/roles` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:58:54.190572 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-224a1129-d38c-4d07-93d5-73f9350917db admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:58:54.192447 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-224a1129-d38c-4d07-93d5-73f9350917db admin admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:58:54.206209 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 1266/2535] 10.222.0.13 () {58 vars in 1122 bytes} [Wed Aug 31 09:58:54 2022] GET /identity/v3/roles => generated 1420 bytes in 131 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 09:58:54.211129 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce14435-c32e-4387-8fa7-1aaa83940db3 admin admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:58:54.222888 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce14435-c32e-4387-8fa7-1aaa83940db3 admin admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:58:54.232268 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e022da-eeeb-4cc3-b75b-f242cb7b2413 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:58:54.233328 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e022da-eeeb-4cc3-b75b-f242cb7b2413 admin admin] REQUEST_METHOD: `PUT` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:58:54.233848 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e022da-eeeb-4cc3-b75b-f242cb7b2413 admin admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:58:54.234274 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e022da-eeeb-4cc3-b75b-f242cb7b2413 admin admin] PATH_INFO: `/v3/projects/bca984647ee5413ea5fc84d397b01026/users/dfd3fce40f81403294833f2c1e8656f8/roles/5d4ee63a3add47fc9a6b8b16d39e0a1f` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:58:54.237224 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0e022da-eeeb-4cc3-b75b-f242cb7b2413 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bca984647ee5413ea5fc84d397b01026, user_id=dfd3fce40f81403294833f2c1e8656f8, role_id=5d4ee63a3add47fc9a6b8b16d39e0a1f)` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:58:54.240386 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0e022da-eeeb-4cc3-b75b-f242cb7b2413 admin admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:58:54.261587 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 1270/2536] 10.222.0.13 () {60 vars in 1483 bytes} [Wed Aug 31 09:58:54 2022] PUT /identity/v3/projects/bca984647ee5413ea5fc84d397b01026/users/dfd3fce40f81403294833f2c1e8656f8/roles/5d4ee63a3add47fc9a6b8b16d39e0a1f => generated 0 bytes in 42 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 09:58:54.263443 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b715aa7-45a2-4ef4-ac68-536db064a199 None None] REQUEST_METHOD: `POST` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:58:54.263735 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b715aa7-45a2-4ef4-ac68-536db064a199 None None] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:58:54.264013 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b715aa7-45a2-4ef4-ac68-536db064a199 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:58:54.303243 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.auth.core [None req-5b715aa7-45a2-4ef4-ac68-536db064a199 None None] MFA Rules not processed for user `dfd3fce40f81403294833f2c1e8656f8`. Rule list: `[]` (Enabled: `True`). {{(pid=88113) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:58:54.304822 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.fernet_utils [None req-5b715aa7-45a2-4ef4-ac68-536db064a199 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:58:54.419368 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 1267/2537] 10.222.0.13 () {58 vars in 958 bytes} [Wed Aug 31 09:58:54 2022] POST /identity/v3/auth/tokens => generated 3351 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:58:54.428984 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e022da-eeeb-4cc3-b75b-f242cb7b2413 admin admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:58:54.430971 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.fernet_utils [None req-e0e022da-eeeb-4cc3-b75b-f242cb7b2413 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88114) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:58:54.537451 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e022da-eeeb-4cc3-b75b-f242cb7b2413 admin admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:58:54.543773 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1208c87e-2277-4c73-8cba-e3c5ac3c4bcc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d52b69c8e94441dacfcff944326f169', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '49548eb44d62415582af820ab66decee', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:58:54.545009 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1208c87e-2277-4c73-8cba-e3c5ac3c4bcc service cinder] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:58:54.545834 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1208c87e-2277-4c73-8cba-e3c5ac3c4bcc service cinder] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:58:54.546575 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1208c87e-2277-4c73-8cba-e3c5ac3c4bcc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:58:54.567489 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1208c87e-2277-4c73-8cba-e3c5ac3c4bcc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:58:54.570117 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1208c87e-2277-4c73-8cba-e3c5ac3c4bcc service cinder] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:58:54.590249 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 1271/2538] 10.222.0.13 () {60 vars in 1365 bytes} [Wed Aug 31 09:58:54 2022] GET /identity/v3/auth/tokens => generated 3351 bytes in 162 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:58:54.838566 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b715aa7-45a2-4ef4-ac68-536db064a199 None None] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:58:54.870048 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b715aa7-45a2-4ef4-ac68-536db064a199 None None] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:58:54.886125 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e033ad-6b50-4f76-b83f-1f8f59c2bab9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d52b69c8e94441dacfcff944326f169', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '49548eb44d62415582af820ab66decee', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:58:54.887199 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e033ad-6b50-4f76-b83f-1f8f59c2bab9 service cinder] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:58:54.888628 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e033ad-6b50-4f76-b83f-1f8f59c2bab9 service cinder] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:58:54.889128 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e033ad-6b50-4f76-b83f-1f8f59c2bab9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:58:54.892377 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.fernet_utils [None req-d0e033ad-6b50-4f76-b83f-1f8f59c2bab9 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:58:54.979830 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0e033ad-6b50-4f76-b83f-1f8f59c2bab9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:58:54.981347 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0e033ad-6b50-4f76-b83f-1f8f59c2bab9 service cinder] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:58:54.991322 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 1268/2539] 10.222.0.13 () {60 vars in 1365 bytes} [Wed Aug 31 09:58:54 2022] GET /identity/v3/auth/tokens => generated 3284 bytes in 154 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:59:04.580757 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:INSERT=3,keystone:DELETE=1,keystone:SELECT=43 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:59:04.986550 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=99 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:59:06.778471 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1208c87e-2277-4c73-8cba-e3c5ac3c4bcc service cinder] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:59:06.850049 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1208c87e-2277-4c73-8cba-e3c5ac3c4bcc service cinder] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:59:06.854612 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-006ce574-e2a4-4f7c-a1a8-964dc293c928 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:59:06.855245 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006ce574-e2a4-4f7c-a1a8-964dc293c928 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:59:06.855523 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006ce574-e2a4-4f7c-a1a8-964dc293c928 admin admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:59:06.855798 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006ce574-e2a4-4f7c-a1a8-964dc293c928 admin admin] PATH_INFO: `/v3/users/6e7d73aef07b44db82b871bc60f6c298` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:59:06.856935 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-006ce574-e2a4-4f7c-a1a8-964dc293c928 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6e7d73aef07b44db82b871bc60f6c298)` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:59:06.858898 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-006ce574-e2a4-4f7c-a1a8-964dc293c928 admin admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:59:06.930537 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.notifications [None req-006ce574-e2a4-4f7c-a1a8-964dc293c928 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6e7d73aef07b44db82b871bc60f6c298'} {{(pid=88114) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:59:06.937515 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.notifications [None req-006ce574-e2a4-4f7c-a1a8-964dc293c928 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6e7d73aef07b44db82b871bc60f6c298'} {{(pid=88114) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:59:06.946112 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.notifications [None req-006ce574-e2a4-4f7c-a1a8-964dc293c928 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6e7d73aef07b44db82b871bc60f6c298'} {{(pid=88114) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:59:06.946742 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.notifications [None req-006ce574-e2a4-4f7c-a1a8-964dc293c928 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6e7d73aef07b44db82b871bc60f6c298'} {{(pid=88114) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:59:06.960701 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.notifications [None req-006ce574-e2a4-4f7c-a1a8-964dc293c928 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6e7d73aef07b44db82b871bc60f6c298'} {{(pid=88114) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:59:06.967840 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 1272/2540] 10.222.0.13 () {58 vars in 1224 bytes} [Wed Aug 31 09:59:06 2022] DELETE /identity/v3/users/6e7d73aef07b44db82b871bc60f6c298 => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:59:06.973348 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e662ae5-ef20-4efb-b802-21ef8bc6545f None None] REQUEST_METHOD: `POST` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:59:06.973871 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e662ae5-ef20-4efb-b802-21ef8bc6545f None None] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:59:06.974060 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e662ae5-ef20-4efb-b802-21ef8bc6545f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:59:06.996303 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.auth.core [None req-6e662ae5-ef20-4efb-b802-21ef8bc6545f None None] MFA Rules not processed for user `fd836cf590a145df9acf19cf3ae2ac56`. Rule list: `[]` (Enabled: `True`). {{(pid=88113) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:59:06.997655 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.fernet_utils [None req-6e662ae5-ef20-4efb-b802-21ef8bc6545f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:59:07.063468 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 1269/2541] 10.222.0.13 () {58 vars in 958 bytes} [Wed Aug 31 09:59:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:59:07.073912 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-006ce574-e2a4-4f7c-a1a8-964dc293c928 admin admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:59:07.075400 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.fernet_utils [None req-006ce574-e2a4-4f7c-a1a8-964dc293c928 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88114) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:59:07.137642 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-006ce574-e2a4-4f7c-a1a8-964dc293c928 admin admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:59:07.141227 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c127634-8b77-4bb4-997a-9715775fc0d3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9bdaacd597442c2bcde2c555f2cc140', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '49548eb44d62415582af820ab66decee', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:59:07.141860 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c127634-8b77-4bb4-997a-9715775fc0d3 service neutron] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:59:07.142139 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c127634-8b77-4bb4-997a-9715775fc0d3 service neutron] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:59:07.142409 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c127634-8b77-4bb4-997a-9715775fc0d3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:59:07.152193 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c127634-8b77-4bb4-997a-9715775fc0d3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:59:07.153645 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c127634-8b77-4bb4-997a-9715775fc0d3 service neutron] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:59:07.164390 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 1273/2542] 10.222.0.13 () {60 vars in 1365 bytes} [Wed Aug 31 09:59:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:59:07.454256 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e662ae5-ef20-4efb-b802-21ef8bc6545f None None] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:59:07.455700 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.fernet_utils [None req-6e662ae5-ef20-4efb-b802-21ef8bc6545f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:59:07.470351 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e662ae5-ef20-4efb-b802-21ef8bc6545f None None] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:59:07.476129 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1074c968-65ad-40e7-94ea-1f8d530c7d17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:59:07.477460 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1074c968-65ad-40e7-94ea-1f8d530c7d17 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:59:07.477748 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1074c968-65ad-40e7-94ea-1f8d530c7d17 admin admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:59:07.478234 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1074c968-65ad-40e7-94ea-1f8d530c7d17 admin admin] PATH_INFO: `/v3/projects/6cd60a0e70124fb4b0aaf707c4e1dd5f` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:59:07.479511 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1074c968-65ad-40e7-94ea-1f8d530c7d17 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6cd60a0e70124fb4b0aaf707c4e1dd5f)` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:59:07.481300 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1074c968-65ad-40e7-94ea-1f8d530c7d17 admin admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:59:07.529442 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.notifications [None req-1074c968-65ad-40e7-94ea-1f8d530c7d17 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6cd60a0e70124fb4b0aaf707c4e1dd5f', 'request_id': 'req-1074c968-65ad-40e7-94ea-1f8d530c7d17'} {{(pid=88113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:59:07.535992 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.notifications [None req-1074c968-65ad-40e7-94ea-1f8d530c7d17 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6cd60a0e70124fb4b0aaf707c4e1dd5f', 'request_id': 'req-1074c968-65ad-40e7-94ea-1f8d530c7d17'} {{(pid=88113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:59:07.543272 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.notifications [None req-1074c968-65ad-40e7-94ea-1f8d530c7d17 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6cd60a0e70124fb4b0aaf707c4e1dd5f', 'request_id': 'req-1074c968-65ad-40e7-94ea-1f8d530c7d17'} {{(pid=88113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:59:07.555134 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.notifications [None req-1074c968-65ad-40e7-94ea-1f8d530c7d17 admin admin] The token cache is being invalidate because project 6cd60a0e70124fb4b0aaf707c4e1dd5f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88113) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 09:59:07.555420 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.notifications [None req-1074c968-65ad-40e7-94ea-1f8d530c7d17 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:59:07.556051 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.notifications [None req-1074c968-65ad-40e7-94ea-1f8d530c7d17 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:59:07.557571 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 1270/2543] 10.222.0.13 () {58 vars in 1233 bytes} [Wed Aug 31 09:59:07 2022] DELETE /identity/v3/projects/6cd60a0e70124fb4b0aaf707c4e1dd5f => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:59:07.561699 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c127634-8b77-4bb4-997a-9715775fc0d3 service neutron] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:59:07.571190 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c127634-8b77-4bb4-997a-9715775fc0d3 service neutron] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:59:07.588815 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:59:07.631069 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed79c48-5b5a-429a-9744-ff1029d55da8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:59:07.631721 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed79c48-5b5a-429a-9744-ff1029d55da8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:59:07.631986 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed79c48-5b5a-429a-9744-ff1029d55da8 admin admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:59:07.632262 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed79c48-5b5a-429a-9744-ff1029d55da8 admin admin] PATH_INFO: `/v3/users/dfd3fce40f81403294833f2c1e8656f8` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:59:07.633395 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bed79c48-5b5a-429a-9744-ff1029d55da8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=dfd3fce40f81403294833f2c1e8656f8)` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:59:07.635333 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bed79c48-5b5a-429a-9744-ff1029d55da8 admin admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:59:07.680756 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.notifications [None req-bed79c48-5b5a-429a-9744-ff1029d55da8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dfd3fce40f81403294833f2c1e8656f8'} {{(pid=88114) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:59:07.687713 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.notifications [None req-bed79c48-5b5a-429a-9744-ff1029d55da8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dfd3fce40f81403294833f2c1e8656f8'} {{(pid=88114) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:59:07.695810 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.notifications [None req-bed79c48-5b5a-429a-9744-ff1029d55da8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dfd3fce40f81403294833f2c1e8656f8'} {{(pid=88114) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:59:07.696435 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.notifications [None req-bed79c48-5b5a-429a-9744-ff1029d55da8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dfd3fce40f81403294833f2c1e8656f8'} {{(pid=88114) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:59:07.708934 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.notifications [None req-bed79c48-5b5a-429a-9744-ff1029d55da8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dfd3fce40f81403294833f2c1e8656f8'} {{(pid=88114) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:59:07.714760 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 1274/2544] 10.222.0.13 () {58 vars in 1224 bytes} [Wed Aug 31 09:59:07 2022] DELETE /identity/v3/users/dfd3fce40f81403294833f2c1e8656f8 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:59:08.312610 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1074c968-65ad-40e7-94ea-1f8d530c7d17 admin admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:59:08.316969 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.fernet_utils [None req-1074c968-65ad-40e7-94ea-1f8d530c7d17 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:59:08.327726 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1074c968-65ad-40e7-94ea-1f8d530c7d17 admin admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:59:08.399180 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c88ae9-0cdf-4186-bffc-1f794fc3d696 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:59:08.399822 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c88ae9-0cdf-4186-bffc-1f794fc3d696 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:59:08.400096 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c88ae9-0cdf-4186-bffc-1f794fc3d696 admin admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:59:08.400368 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c88ae9-0cdf-4186-bffc-1f794fc3d696 admin admin] PATH_INFO: `/v3/projects/bca984647ee5413ea5fc84d397b01026` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:59:08.401538 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0c88ae9-0cdf-4186-bffc-1f794fc3d696 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bca984647ee5413ea5fc84d397b01026)` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:59:08.403464 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0c88ae9-0cdf-4186-bffc-1f794fc3d696 admin admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:59:08.438254 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:59:08.457244 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.notifications [None req-f0c88ae9-0cdf-4186-bffc-1f794fc3d696 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bca984647ee5413ea5fc84d397b01026', 'request_id': 'req-f0c88ae9-0cdf-4186-bffc-1f794fc3d696'} {{(pid=88113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:59:08.463010 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.notifications [None req-f0c88ae9-0cdf-4186-bffc-1f794fc3d696 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bca984647ee5413ea5fc84d397b01026', 'request_id': 'req-f0c88ae9-0cdf-4186-bffc-1f794fc3d696'} {{(pid=88113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:59:08.471735 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.notifications [None req-f0c88ae9-0cdf-4186-bffc-1f794fc3d696 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bca984647ee5413ea5fc84d397b01026', 'request_id': 'req-f0c88ae9-0cdf-4186-bffc-1f794fc3d696'} {{(pid=88113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:59:08.485152 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.notifications [None req-f0c88ae9-0cdf-4186-bffc-1f794fc3d696 admin admin] The token cache is being invalidate because project bca984647ee5413ea5fc84d397b01026 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88113) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 09:59:08.485462 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.notifications [None req-f0c88ae9-0cdf-4186-bffc-1f794fc3d696 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:59:08.486167 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.notifications [None req-f0c88ae9-0cdf-4186-bffc-1f794fc3d696 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:59:08.487711 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 1271/2545] 10.222.0.13 () {58 vars in 1233 bytes} [Wed Aug 31 09:59:08 2022] DELETE /identity/v3/projects/bca984647ee5413ea5fc84d397b01026 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 09:59:08.492948 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8fe702-bc6a-4763-ba93-d83246316472 None None] REQUEST_METHOD: `POST` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:59:08.493453 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8fe702-bc6a-4763-ba93-d83246316472 None None] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:59:08.493572 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8fe702-bc6a-4763-ba93-d83246316472 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:59:08.514382 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.auth.core [None req-1d8fe702-bc6a-4763-ba93-d83246316472 None None] MFA Rules not processed for user `fd836cf590a145df9acf19cf3ae2ac56`. Rule list: `[]` (Enabled: `True`). {{(pid=88114) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:59:08.515599 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.fernet_utils [None req-1d8fe702-bc6a-4763-ba93-d83246316472 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88114) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:59:08.580076 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 1275/2546] 10.222.0.13 () {58 vars in 958 bytes} [Wed Aug 31 09:59:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:59:08.590396 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce7f5cd-703e-4ab7-9c0e-0ddcf80b28eb None None] REQUEST_METHOD: `POST` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:59:08.590684 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce7f5cd-703e-4ab7-9c0e-0ddcf80b28eb None None] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:59:08.590960 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce7f5cd-703e-4ab7-9c0e-0ddcf80b28eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:59:08.611756 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.auth.core [None req-cce7f5cd-703e-4ab7-9c0e-0ddcf80b28eb None None] MFA Rules not processed for user `fd836cf590a145df9acf19cf3ae2ac56`. Rule list: `[]` (Enabled: `True`). {{(pid=88113) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:59:08.613079 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.fernet_utils [None req-cce7f5cd-703e-4ab7-9c0e-0ddcf80b28eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:59:08.626618 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 1272/2547] 10.222.0.13 () {58 vars in 958 bytes} [Wed Aug 31 09:59:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:59:08.631942 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d8fe702-bc6a-4763-ba93-d83246316472 None None] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:59:08.641846 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d8fe702-bc6a-4763-ba93-d83246316472 None None] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:59:08.647455 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f29f030-c159-492d-9c3f-bee37e0d86d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:59:08.648061 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f29f030-c159-492d-9c3f-bee37e0d86d3 admin admin] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:59:08.648340 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f29f030-c159-492d-9c3f-bee37e0d86d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:59:08.648610 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f29f030-c159-492d-9c3f-bee37e0d86d3 admin admin] PATH_INFO: `/v3/domains` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:59:08.649262 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f29f030-c159-492d-9c3f-bee37e0d86d3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88114) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:59:08.649594 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f29f030-c159-492d-9c3f-bee37e0d86d3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:59:08.651188 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f29f030-c159-492d-9c3f-bee37e0d86d3 admin admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:59:08.664655 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 1276/2548] 10.222.0.13 () {58 vars in 1153 bytes} [Wed Aug 31 09:59:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:59:08.669310 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce7f5cd-703e-4ab7-9c0e-0ddcf80b28eb None None] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:59:08.678932 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce7f5cd-703e-4ab7-9c0e-0ddcf80b28eb None None] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:59:08.683871 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad44f895-abcd-4315-a37f-8cab18ba5c38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:59:08.684474 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad44f895-abcd-4315-a37f-8cab18ba5c38 admin admin] REQUEST_METHOD: `POST` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:59:08.684749 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad44f895-abcd-4315-a37f-8cab18ba5c38 admin admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:59:08.685019 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad44f895-abcd-4315-a37f-8cab18ba5c38 admin admin] PATH_INFO: `/v3/projects` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:59:08.685709 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad44f895-abcd-4315-a37f-8cab18ba5c38 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:59:08.687614 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad44f895-abcd-4315-a37f-8cab18ba5c38 admin admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:59:08.718499 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 1273/2549] 10.222.0.13 () {60 vars in 1153 bytes} [Wed Aug 31 09:59:08 2022] POST /identity/v3/projects => generated 387 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 09:59:08.723874 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f29f030-c159-492d-9c3f-bee37e0d86d3 admin admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:59:08.734624 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f29f030-c159-492d-9c3f-bee37e0d86d3 admin admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:59:08.762663 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:59:08.797937 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a06bd6a-c1f5-4fb5-a254-db2618e1c968 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:59:08.798584 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a06bd6a-c1f5-4fb5-a254-db2618e1c968 admin admin] REQUEST_METHOD: `POST` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:59:08.798866 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a06bd6a-c1f5-4fb5-a254-db2618e1c968 admin admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:59:08.799145 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a06bd6a-c1f5-4fb5-a254-db2618e1c968 admin admin] PATH_INFO: `/v3/users` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:59:08.799800 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a06bd6a-c1f5-4fb5-a254-db2618e1c968 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:59:08.801930 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a06bd6a-c1f5-4fb5-a254-db2618e1c968 admin admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:59:08.826440 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 1277/2550] 10.222.0.13 () {60 vars in 1144 bytes} [Wed Aug 31 09:59:08 2022] POST /identity/v3/users => generated 294 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 09:59:08.830746 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad44f895-abcd-4315-a37f-8cab18ba5c38 admin admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:59:08.843237 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad44f895-abcd-4315-a37f-8cab18ba5c38 admin admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:59:08.848657 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c7d9da-b7ad-43aa-912d-b263cb89d604 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:59:08.849317 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c7d9da-b7ad-43aa-912d-b263cb89d604 admin admin] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:59:08.849631 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c7d9da-b7ad-43aa-912d-b263cb89d604 admin admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:59:08.849904 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c7d9da-b7ad-43aa-912d-b263cb89d604 admin admin] PATH_INFO: `/v3/roles` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:59:08.850585 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1c7d9da-b7ad-43aa-912d-b263cb89d604 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:59:08.852200 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1c7d9da-b7ad-43aa-912d-b263cb89d604 admin admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:59:08.863398 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 1274/2551] 10.222.0.13 () {58 vars in 1122 bytes} [Wed Aug 31 09:59:08 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 09:59:08.867832 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a06bd6a-c1f5-4fb5-a254-db2618e1c968 admin admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:59:08.879151 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a06bd6a-c1f5-4fb5-a254-db2618e1c968 admin admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:59:08.884368 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e932db23-7ecf-4a11-978b-27e577fe7a32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:59:08.885011 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e932db23-7ecf-4a11-978b-27e577fe7a32 admin admin] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:59:08.885237 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e932db23-7ecf-4a11-978b-27e577fe7a32 admin admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:59:08.885539 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e932db23-7ecf-4a11-978b-27e577fe7a32 admin admin] PATH_INFO: `/v3/roles` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:59:08.886181 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e932db23-7ecf-4a11-978b-27e577fe7a32 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:59:08.887773 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e932db23-7ecf-4a11-978b-27e577fe7a32 admin admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:59:08.899742 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 1278/2552] 10.222.0.13 () {58 vars in 1122 bytes} [Wed Aug 31 09:59:08 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 09:59:08.903952 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c7d9da-b7ad-43aa-912d-b263cb89d604 admin admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:59:08.915005 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c7d9da-b7ad-43aa-912d-b263cb89d604 admin admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:59:08.919871 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-856fc324-1928-4412-85f2-4fdb65dcda94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:59:08.920559 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856fc324-1928-4412-85f2-4fdb65dcda94 admin admin] REQUEST_METHOD: `PUT` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:59:08.920836 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856fc324-1928-4412-85f2-4fdb65dcda94 admin admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:59:08.921108 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856fc324-1928-4412-85f2-4fdb65dcda94 admin admin] PATH_INFO: `/v3/projects/da380bde83f44d34bc172e9c8eb16438/users/b2e1f8f802a446ddbe8803047b257d8e/roles/6c99f5d9f1f243eaa134d8540f21095d` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:59:08.937870 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-856fc324-1928-4412-85f2-4fdb65dcda94 admin admin] RBAC: Authorizing `identity:create_grant(project_id=da380bde83f44d34bc172e9c8eb16438, user_id=b2e1f8f802a446ddbe8803047b257d8e, role_id=6c99f5d9f1f243eaa134d8540f21095d)` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:59:08.941111 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-856fc324-1928-4412-85f2-4fdb65dcda94 admin admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:59:08.954607 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 1275/2553] 10.222.0.13 () {60 vars in 1483 bytes} [Wed Aug 31 09:59:08 2022] PUT /identity/v3/projects/da380bde83f44d34bc172e9c8eb16438/users/b2e1f8f802a446ddbe8803047b257d8e/roles/6c99f5d9f1f243eaa134d8540f21095d => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:59:08.966147 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa6dc21-f879-4967-99a0-520653b2c8df None None] REQUEST_METHOD: `POST` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:59:08.966428 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa6dc21-f879-4967-99a0-520653b2c8df None None] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:59:08.966704 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa6dc21-f879-4967-99a0-520653b2c8df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:59:08.997580 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.auth.core [None req-ffa6dc21-f879-4967-99a0-520653b2c8df None None] MFA Rules not processed for user `b2e1f8f802a446ddbe8803047b257d8e`. Rule list: `[]` (Enabled: `True`). {{(pid=88114) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:59:08.998816 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.fernet_utils [None req-ffa6dc21-f879-4967-99a0-520653b2c8df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88114) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:59:09.077740 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 1279/2554] 10.222.0.13 () {58 vars in 958 bytes} [Wed Aug 31 09:59:08 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:59:09.083219 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-856fc324-1928-4412-85f2-4fdb65dcda94 admin admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:59:09.150588 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-856fc324-1928-4412-85f2-4fdb65dcda94 admin admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:59:09.155759 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a12643-71f4-4a8b-9f39-a9231f82bb0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:59:09.156398 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a12643-71f4-4a8b-9f39-a9231f82bb0c admin admin] REQUEST_METHOD: `POST` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:59:09.156680 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a12643-71f4-4a8b-9f39-a9231f82bb0c admin admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:59:09.156955 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a12643-71f4-4a8b-9f39-a9231f82bb0c admin admin] PATH_INFO: `/v3/projects` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:59:09.157662 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6a12643-71f4-4a8b-9f39-a9231f82bb0c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:59:09.160240 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6a12643-71f4-4a8b-9f39-a9231f82bb0c admin admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:59:09.193580 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 1276/2555] 10.222.0.13 () {60 vars in 1153 bytes} [Wed Aug 31 09:59:09 2022] POST /identity/v3/projects => generated 385 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 09:59:09.198218 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa6dc21-f879-4967-99a0-520653b2c8df None None] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:59:09.210851 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa6dc21-f879-4967-99a0-520653b2c8df None None] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:59:09.281496 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da9ec78-a897-41bd-a154-06d709f9fde6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:59:09.282118 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da9ec78-a897-41bd-a154-06d709f9fde6 admin admin] REQUEST_METHOD: `POST` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:59:09.282394 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da9ec78-a897-41bd-a154-06d709f9fde6 admin admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:59:09.282663 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da9ec78-a897-41bd-a154-06d709f9fde6 admin admin] PATH_INFO: `/v3/users` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:59:09.283308 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7da9ec78-a897-41bd-a154-06d709f9fde6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:59:09.286309 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7da9ec78-a897-41bd-a154-06d709f9fde6 admin admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:59:09.309304 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 1280/2556] 10.222.0.13 () {60 vars in 1144 bytes} [Wed Aug 31 09:59:09 2022] POST /identity/v3/users => generated 299 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 09:59:09.313455 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a12643-71f4-4a8b-9f39-a9231f82bb0c admin admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:59:09.324510 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a12643-71f4-4a8b-9f39-a9231f82bb0c admin admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:59:09.329581 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cfd73d3-cd26-413f-93b5-e615f74292e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:59:09.330256 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cfd73d3-cd26-413f-93b5-e615f74292e3 admin admin] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:59:09.330535 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cfd73d3-cd26-413f-93b5-e615f74292e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:59:09.330809 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cfd73d3-cd26-413f-93b5-e615f74292e3 admin admin] PATH_INFO: `/v3/roles` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:59:09.332405 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cfd73d3-cd26-413f-93b5-e615f74292e3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:59:09.333634 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cfd73d3-cd26-413f-93b5-e615f74292e3 admin admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:59:09.345374 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 1277/2557] 10.222.0.13 () {58 vars in 1122 bytes} [Wed Aug 31 09:59:09 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 09:59:09.350899 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da9ec78-a897-41bd-a154-06d709f9fde6 admin admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:59:09.361206 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da9ec78-a897-41bd-a154-06d709f9fde6 admin admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:59:09.366375 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8917cf81-8b9b-4a66-ab07-c259c11283ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:59:09.367036 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8917cf81-8b9b-4a66-ab07-c259c11283ce admin admin] REQUEST_METHOD: `PUT` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:59:09.367313 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8917cf81-8b9b-4a66-ab07-c259c11283ce admin admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:59:09.367584 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8917cf81-8b9b-4a66-ab07-c259c11283ce admin admin] PATH_INFO: `/v3/projects/b3ae2a8841574395ae627eea43e4d63d/users/ae21c35cda494adaa195c9ab0b38ac9b/roles/5d4ee63a3add47fc9a6b8b16d39e0a1f` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:59:09.387130 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8917cf81-8b9b-4a66-ab07-c259c11283ce admin admin] RBAC: Authorizing `identity:create_grant(project_id=b3ae2a8841574395ae627eea43e4d63d, user_id=ae21c35cda494adaa195c9ab0b38ac9b, role_id=5d4ee63a3add47fc9a6b8b16d39e0a1f)` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:59:09.389911 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8917cf81-8b9b-4a66-ab07-c259c11283ce admin admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:59:09.406125 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 1281/2558] 10.222.0.13 () {60 vars in 1483 bytes} [Wed Aug 31 09:59:09 2022] PUT /identity/v3/projects/b3ae2a8841574395ae627eea43e4d63d/users/ae21c35cda494adaa195c9ab0b38ac9b/roles/5d4ee63a3add47fc9a6b8b16d39e0a1f => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:59:09.411936 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cfd73d3-cd26-413f-93b5-e615f74292e3 admin admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:59:09.425895 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cfd73d3-cd26-413f-93b5-e615f74292e3 admin admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:59:09.450259 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:59:09.501056 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dfeacac-865f-46ec-9ece-4ee8ac364d6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:59:09.501611 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dfeacac-865f-46ec-9ece-4ee8ac364d6d admin admin] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:59:09.501887 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dfeacac-865f-46ec-9ece-4ee8ac364d6d admin admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:59:09.502159 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dfeacac-865f-46ec-9ece-4ee8ac364d6d admin admin] PATH_INFO: `/v3/roles` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:59:09.502810 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dfeacac-865f-46ec-9ece-4ee8ac364d6d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:59:09.504523 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dfeacac-865f-46ec-9ece-4ee8ac364d6d admin admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:59:09.519001 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 1278/2559] 10.222.0.13 () {58 vars in 1122 bytes} [Wed Aug 31 09:59:09 2022] GET /identity/v3/roles => generated 1420 bytes in 104 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 09:59:09.521168 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8917cf81-8b9b-4a66-ab07-c259c11283ce admin admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:59:09.528202 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:59:09.533343 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8917cf81-8b9b-4a66-ab07-c259c11283ce admin admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:59:09.541074 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f47c33ad-2a4b-4dd9-b05c-521a3311cb57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:59:09.541792 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47c33ad-2a4b-4dd9-b05c-521a3311cb57 admin admin] REQUEST_METHOD: `PUT` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:59:09.542067 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47c33ad-2a4b-4dd9-b05c-521a3311cb57 admin admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:59:09.542335 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47c33ad-2a4b-4dd9-b05c-521a3311cb57 admin admin] PATH_INFO: `/v3/projects/b3ae2a8841574395ae627eea43e4d63d/users/ae21c35cda494adaa195c9ab0b38ac9b/roles/5d4ee63a3add47fc9a6b8b16d39e0a1f` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:59:09.544633 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f47c33ad-2a4b-4dd9-b05c-521a3311cb57 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b3ae2a8841574395ae627eea43e4d63d, user_id=ae21c35cda494adaa195c9ab0b38ac9b, role_id=5d4ee63a3add47fc9a6b8b16d39e0a1f)` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:59:09.547149 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f47c33ad-2a4b-4dd9-b05c-521a3311cb57 admin admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:59:09.559417 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 1282/2560] 10.222.0.13 () {60 vars in 1483 bytes} [Wed Aug 31 09:59:09 2022] PUT /identity/v3/projects/b3ae2a8841574395ae627eea43e4d63d/users/ae21c35cda494adaa195c9ab0b38ac9b/roles/5d4ee63a3add47fc9a6b8b16d39e0a1f => generated 0 bytes in 40 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:59:09.571076 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03fbf75c-6252-45a1-80aa-edcc5df6b07d None None] REQUEST_METHOD: `POST` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:59:09.571598 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03fbf75c-6252-45a1-80aa-edcc5df6b07d None None] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:59:09.572113 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03fbf75c-6252-45a1-80aa-edcc5df6b07d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:59:09.605278 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.auth.core [None req-03fbf75c-6252-45a1-80aa-edcc5df6b07d None None] MFA Rules not processed for user `ae21c35cda494adaa195c9ab0b38ac9b`. Rule list: `[]` (Enabled: `True`). {{(pid=88113) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:59:09.606728 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.fernet_utils [None req-03fbf75c-6252-45a1-80aa-edcc5df6b07d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:59:09.696363 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 1279/2561] 10.222.0.13 () {58 vars in 958 bytes} [Wed Aug 31 09:59:09 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:59:09.707981 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f47c33ad-2a4b-4dd9-b05c-521a3311cb57 admin admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:59:09.709940 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.fernet_utils [None req-f47c33ad-2a4b-4dd9-b05c-521a3311cb57 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88114) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:59:09.780233 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f47c33ad-2a4b-4dd9-b05c-521a3311cb57 admin admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:59:09.784106 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51eee57e-aac4-465e-8908-2a5c45cc73ba service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d52b69c8e94441dacfcff944326f169', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '49548eb44d62415582af820ab66decee', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:59:09.785414 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51eee57e-aac4-465e-8908-2a5c45cc73ba service cinder] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:59:09.785727 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51eee57e-aac4-465e-8908-2a5c45cc73ba service cinder] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:59:09.785940 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51eee57e-aac4-465e-8908-2a5c45cc73ba service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:59:09.795990 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51eee57e-aac4-465e-8908-2a5c45cc73ba service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:59:09.797484 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51eee57e-aac4-465e-8908-2a5c45cc73ba service cinder] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:59:09.808764 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 1283/2562] 10.222.0.13 () {60 vars in 1365 bytes} [Wed Aug 31 09:59:09 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:59:10.512927 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03fbf75c-6252-45a1-80aa-edcc5df6b07d None None] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:59:10.583295 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03fbf75c-6252-45a1-80aa-edcc5df6b07d None None] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:59:10.590156 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c263a8-ea12-4318-92c7-9c322c109467 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:59:10.590813 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c263a8-ea12-4318-92c7-9c322c109467 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:59:10.591101 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c263a8-ea12-4318-92c7-9c322c109467 admin admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:59:10.593853 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c263a8-ea12-4318-92c7-9c322c109467 admin admin] PATH_INFO: `/v3/users/b2e1f8f802a446ddbe8803047b257d8e` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:59:10.595428 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87c263a8-ea12-4318-92c7-9c322c109467 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b2e1f8f802a446ddbe8803047b257d8e)` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:59:10.598796 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87c263a8-ea12-4318-92c7-9c322c109467 admin admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:59:10.674342 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.notifications [None req-87c263a8-ea12-4318-92c7-9c322c109467 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b2e1f8f802a446ddbe8803047b257d8e'} {{(pid=88113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:59:10.679878 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:59:10.693356 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.notifications [None req-87c263a8-ea12-4318-92c7-9c322c109467 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b2e1f8f802a446ddbe8803047b257d8e'} {{(pid=88113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:59:10.713532 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.notifications [None req-87c263a8-ea12-4318-92c7-9c322c109467 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b2e1f8f802a446ddbe8803047b257d8e'} {{(pid=88113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:59:10.714315 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.notifications [None req-87c263a8-ea12-4318-92c7-9c322c109467 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b2e1f8f802a446ddbe8803047b257d8e'} {{(pid=88113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:59:10.724715 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.notifications [None req-87c263a8-ea12-4318-92c7-9c322c109467 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b2e1f8f802a446ddbe8803047b257d8e'} {{(pid=88113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:59:10.738258 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 1280/2563] 10.222.0.13 () {58 vars in 1224 bytes} [Wed Aug 31 09:59:10 2022] DELETE /identity/v3/users/b2e1f8f802a446ddbe8803047b257d8e => generated 0 bytes in 226 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:59:10.743928 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138f08b6-3363-46dd-b98c-ab08e9c298f1 None None] REQUEST_METHOD: `POST` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:59:10.744421 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138f08b6-3363-46dd-b98c-ab08e9c298f1 None None] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:59:10.744538 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138f08b6-3363-46dd-b98c-ab08e9c298f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:59:10.765437 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.auth.core [None req-138f08b6-3363-46dd-b98c-ab08e9c298f1 None None] MFA Rules not processed for user `fd836cf590a145df9acf19cf3ae2ac56`. Rule list: `[]` (Enabled: `True`). {{(pid=88114) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:59:10.767147 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.fernet_utils [None req-138f08b6-3363-46dd-b98c-ab08e9c298f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88114) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:59:10.828095 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 1284/2564] 10.222.0.13 () {58 vars in 958 bytes} [Wed Aug 31 09:59:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:59:10.839721 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c263a8-ea12-4318-92c7-9c322c109467 admin admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:59:10.841169 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.fernet_utils [None req-87c263a8-ea12-4318-92c7-9c322c109467 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:59:10.900566 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c263a8-ea12-4318-92c7-9c322c109467 admin admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:59:10.904344 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b7cf3f-c4ad-447c-b972-617dcf5a4b23 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9bdaacd597442c2bcde2c555f2cc140', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '49548eb44d62415582af820ab66decee', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:59:10.904944 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b7cf3f-c4ad-447c-b972-617dcf5a4b23 service neutron] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:59:10.905222 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b7cf3f-c4ad-447c-b972-617dcf5a4b23 service neutron] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:59:10.905525 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b7cf3f-c4ad-447c-b972-617dcf5a4b23 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:59:10.915379 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4b7cf3f-c4ad-447c-b972-617dcf5a4b23 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:59:10.916945 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4b7cf3f-c4ad-447c-b972-617dcf5a4b23 service neutron] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:59:10.926396 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 1281/2565] 10.222.0.13 () {60 vars in 1365 bytes} [Wed Aug 31 09:59:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:59:11.205274 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-138f08b6-3363-46dd-b98c-ab08e9c298f1 None None] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:59:11.207032 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.fernet_utils [None req-138f08b6-3363-46dd-b98c-ab08e9c298f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88114) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:59:11.220427 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-138f08b6-3363-46dd-b98c-ab08e9c298f1 None None] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:59:11.225661 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66cfba83-eb31-4bfe-869b-38b535864e94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:59:11.226299 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66cfba83-eb31-4bfe-869b-38b535864e94 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:59:11.226571 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66cfba83-eb31-4bfe-869b-38b535864e94 admin admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:59:11.226838 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66cfba83-eb31-4bfe-869b-38b535864e94 admin admin] PATH_INFO: `/v3/projects/da380bde83f44d34bc172e9c8eb16438` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:59:11.227984 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66cfba83-eb31-4bfe-869b-38b535864e94 admin admin] RBAC: Authorizing `identity:delete_project(project_id=da380bde83f44d34bc172e9c8eb16438)` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:59:11.229982 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66cfba83-eb31-4bfe-869b-38b535864e94 admin admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:59:11.277725 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.notifications [None req-66cfba83-eb31-4bfe-869b-38b535864e94 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'da380bde83f44d34bc172e9c8eb16438', 'request_id': 'req-66cfba83-eb31-4bfe-869b-38b535864e94'} {{(pid=88114) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:59:11.283907 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.notifications [None req-66cfba83-eb31-4bfe-869b-38b535864e94 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'da380bde83f44d34bc172e9c8eb16438', 'request_id': 'req-66cfba83-eb31-4bfe-869b-38b535864e94'} {{(pid=88114) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:59:11.290444 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.notifications [None req-66cfba83-eb31-4bfe-869b-38b535864e94 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'da380bde83f44d34bc172e9c8eb16438', 'request_id': 'req-66cfba83-eb31-4bfe-869b-38b535864e94'} {{(pid=88114) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:59:11.303702 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.notifications [None req-66cfba83-eb31-4bfe-869b-38b535864e94 admin admin] The token cache is being invalidate because project da380bde83f44d34bc172e9c8eb16438 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88114) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 09:59:11.303995 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.notifications [None req-66cfba83-eb31-4bfe-869b-38b535864e94 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88114) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:59:11.304649 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.notifications [None req-66cfba83-eb31-4bfe-869b-38b535864e94 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88114) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:59:11.306189 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 1285/2566] 10.222.0.13 () {58 vars in 1233 bytes} [Wed Aug 31 09:59:11 2022] DELETE /identity/v3/projects/da380bde83f44d34bc172e9c8eb16438 => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:59:11.310533 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b7cf3f-c4ad-447c-b972-617dcf5a4b23 service neutron] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:59:11.320711 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b7cf3f-c4ad-447c-b972-617dcf5a4b23 service neutron] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:59:11.377722 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a448d9ac-93d8-4ed0-8a0b-2cee8e456653 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:59:11.378279 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a448d9ac-93d8-4ed0-8a0b-2cee8e456653 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:59:11.378618 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a448d9ac-93d8-4ed0-8a0b-2cee8e456653 admin admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:59:11.379099 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a448d9ac-93d8-4ed0-8a0b-2cee8e456653 admin admin] PATH_INFO: `/v3/users/ae21c35cda494adaa195c9ab0b38ac9b` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:59:11.380387 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a448d9ac-93d8-4ed0-8a0b-2cee8e456653 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ae21c35cda494adaa195c9ab0b38ac9b)` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:59:11.382354 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a448d9ac-93d8-4ed0-8a0b-2cee8e456653 admin admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:59:11.443199 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.notifications [None req-a448d9ac-93d8-4ed0-8a0b-2cee8e456653 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ae21c35cda494adaa195c9ab0b38ac9b'} {{(pid=88113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:59:11.450394 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.notifications [None req-a448d9ac-93d8-4ed0-8a0b-2cee8e456653 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ae21c35cda494adaa195c9ab0b38ac9b'} {{(pid=88113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:59:11.461454 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:59:11.464309 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.notifications [None req-a448d9ac-93d8-4ed0-8a0b-2cee8e456653 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ae21c35cda494adaa195c9ab0b38ac9b'} {{(pid=88113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:59:11.465345 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.notifications [None req-a448d9ac-93d8-4ed0-8a0b-2cee8e456653 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ae21c35cda494adaa195c9ab0b38ac9b'} {{(pid=88113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:59:11.474699 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.notifications [None req-a448d9ac-93d8-4ed0-8a0b-2cee8e456653 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ae21c35cda494adaa195c9ab0b38ac9b'} {{(pid=88113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:59:11.481291 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 1282/2567] 10.222.0.13 () {58 vars in 1224 bytes} [Wed Aug 31 09:59:11 2022] DELETE /identity/v3/users/ae21c35cda494adaa195c9ab0b38ac9b => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:59:11.779925 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66cfba83-eb31-4bfe-869b-38b535864e94 admin admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:59:11.782623 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.fernet_utils [None req-66cfba83-eb31-4bfe-869b-38b535864e94 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88114) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:59:11.813340 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66cfba83-eb31-4bfe-869b-38b535864e94 admin admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:59:11.825826 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:59:11.883999 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46dc3af0-ace5-45c4-b06f-78bc9e1404cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:59:11.884503 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46dc3af0-ace5-45c4-b06f-78bc9e1404cf admin admin] REQUEST_METHOD: `DELETE` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:59:11.884926 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46dc3af0-ace5-45c4-b06f-78bc9e1404cf admin admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:59:11.885063 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46dc3af0-ace5-45c4-b06f-78bc9e1404cf admin admin] PATH_INFO: `/v3/projects/b3ae2a8841574395ae627eea43e4d63d` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:59:11.886276 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46dc3af0-ace5-45c4-b06f-78bc9e1404cf admin admin] RBAC: Authorizing `identity:delete_project(project_id=b3ae2a8841574395ae627eea43e4d63d)` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:59:11.888235 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46dc3af0-ace5-45c4-b06f-78bc9e1404cf admin admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:59:11.948560 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.notifications [None req-46dc3af0-ace5-45c4-b06f-78bc9e1404cf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b3ae2a8841574395ae627eea43e4d63d', 'request_id': 'req-46dc3af0-ace5-45c4-b06f-78bc9e1404cf'} {{(pid=88114) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:59:11.959558 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.notifications [None req-46dc3af0-ace5-45c4-b06f-78bc9e1404cf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b3ae2a8841574395ae627eea43e4d63d', 'request_id': 'req-46dc3af0-ace5-45c4-b06f-78bc9e1404cf'} {{(pid=88114) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:59:11.968009 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.notifications [None req-46dc3af0-ace5-45c4-b06f-78bc9e1404cf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b3ae2a8841574395ae627eea43e4d63d', 'request_id': 'req-46dc3af0-ace5-45c4-b06f-78bc9e1404cf'} {{(pid=88114) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:59:11.987779 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.notifications [None req-46dc3af0-ace5-45c4-b06f-78bc9e1404cf admin admin] The token cache is being invalidate because project b3ae2a8841574395ae627eea43e4d63d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88114) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 09:59:11.988182 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.notifications [None req-46dc3af0-ace5-45c4-b06f-78bc9e1404cf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88114) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:59:11.989249 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.notifications [None req-46dc3af0-ace5-45c4-b06f-78bc9e1404cf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88114) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:59:11.991213 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 1286/2568] 10.222.0.13 () {58 vars in 1233 bytes} [Wed Aug 31 09:59:11 2022] DELETE /identity/v3/projects/b3ae2a8841574395ae627eea43e4d63d => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:59:11.997647 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf1570c-adbd-4df3-882e-6385e1958072 None None] REQUEST_METHOD: `POST` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:59:11.998199 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf1570c-adbd-4df3-882e-6385e1958072 None None] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:59:11.998471 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf1570c-adbd-4df3-882e-6385e1958072 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:59:12.023646 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.auth.core [None req-cdf1570c-adbd-4df3-882e-6385e1958072 None None] MFA Rules not processed for user `fd836cf590a145df9acf19cf3ae2ac56`. Rule list: `[]` (Enabled: `True`). {{(pid=88113) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:59:12.024900 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.fernet_utils [None req-cdf1570c-adbd-4df3-882e-6385e1958072 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:59:12.111564 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 1283/2569] 10.222.0.13 () {58 vars in 958 bytes} [Wed Aug 31 09:59:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:59:12.121109 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7661f441-6634-441b-8fa1-8e297d4fd439 None None] REQUEST_METHOD: `POST` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:59:12.121457 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7661f441-6634-441b-8fa1-8e297d4fd439 None None] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:59:12.121735 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7661f441-6634-441b-8fa1-8e297d4fd439 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:59:12.141754 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.auth.core [None req-7661f441-6634-441b-8fa1-8e297d4fd439 None None] MFA Rules not processed for user `fd836cf590a145df9acf19cf3ae2ac56`. Rule list: `[]` (Enabled: `True`). {{(pid=88114) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:59:12.142959 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.fernet_utils [None req-7661f441-6634-441b-8fa1-8e297d4fd439 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88114) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:59:12.152029 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 1287/2570] 10.222.0.13 () {58 vars in 958 bytes} [Wed Aug 31 09:59:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 09:59:12.157781 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf1570c-adbd-4df3-882e-6385e1958072 None None] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:59:12.169056 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf1570c-adbd-4df3-882e-6385e1958072 None None] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:59:12.174025 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-655be75c-d6e4-4e69-b030-298148e83c17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:59:12.174588 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655be75c-d6e4-4e69-b030-298148e83c17 admin admin] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:59:12.174864 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655be75c-d6e4-4e69-b030-298148e83c17 admin admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:59:12.175137 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655be75c-d6e4-4e69-b030-298148e83c17 admin admin] PATH_INFO: `/v3/domains` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:59:12.175833 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-655be75c-d6e4-4e69-b030-298148e83c17 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88113) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:59:12.176137 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-655be75c-d6e4-4e69-b030-298148e83c17 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:59:12.177803 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-655be75c-d6e4-4e69-b030-298148e83c17 admin admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:59:12.192478 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 1284/2571] 10.222.0.13 () {58 vars in 1153 bytes} [Wed Aug 31 09:59:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:59:12.197102 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7661f441-6634-441b-8fa1-8e297d4fd439 None None] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:59:12.207347 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7661f441-6634-441b-8fa1-8e297d4fd439 None None] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:59:12.212364 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3052432-ba07-4dba-a866-5ec702a02a06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:59:12.212971 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3052432-ba07-4dba-a866-5ec702a02a06 admin admin] REQUEST_METHOD: `POST` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:59:12.213241 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3052432-ba07-4dba-a866-5ec702a02a06 admin admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:59:12.213547 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3052432-ba07-4dba-a866-5ec702a02a06 admin admin] PATH_INFO: `/v3/projects` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:59:12.214191 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3052432-ba07-4dba-a866-5ec702a02a06 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:59:12.216100 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3052432-ba07-4dba-a866-5ec702a02a06 admin admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:59:12.245531 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 1288/2572] 10.222.0.13 () {60 vars in 1153 bytes} [Wed Aug 31 09:59:12 2022] POST /identity/v3/projects => generated 383 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 09:59:12.249746 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-655be75c-d6e4-4e69-b030-298148e83c17 admin admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:59:12.260055 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-655be75c-d6e4-4e69-b030-298148e83c17 admin admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:59:12.323945 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc30301-3bdf-4b84-a5ff-7b0516137c2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:59:12.324560 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc30301-3bdf-4b84-a5ff-7b0516137c2b admin admin] REQUEST_METHOD: `POST` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:59:12.324836 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc30301-3bdf-4b84-a5ff-7b0516137c2b admin admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:59:12.325105 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc30301-3bdf-4b84-a5ff-7b0516137c2b admin admin] PATH_INFO: `/v3/users` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:59:12.325767 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fc30301-3bdf-4b84-a5ff-7b0516137c2b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:59:12.327789 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fc30301-3bdf-4b84-a5ff-7b0516137c2b admin admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:59:12.351854 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 1285/2573] 10.222.0.13 () {60 vars in 1144 bytes} [Wed Aug 31 09:59:12 2022] POST /identity/v3/users => generated 292 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 09:59:12.355909 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3052432-ba07-4dba-a866-5ec702a02a06 admin admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:59:12.371729 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3052432-ba07-4dba-a866-5ec702a02a06 admin admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:59:12.376506 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed69a8b7-234a-455c-90db-b3d728831edc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:59:12.377108 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed69a8b7-234a-455c-90db-b3d728831edc admin admin] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:59:12.377470 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed69a8b7-234a-455c-90db-b3d728831edc admin admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:59:12.377723 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed69a8b7-234a-455c-90db-b3d728831edc admin admin] PATH_INFO: `/v3/roles` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:59:12.378420 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed69a8b7-234a-455c-90db-b3d728831edc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:59:12.380131 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed69a8b7-234a-455c-90db-b3d728831edc admin admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:59:12.390293 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 1289/2574] 10.222.0.13 () {58 vars in 1122 bytes} [Wed Aug 31 09:59:12 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 09:59:12.395243 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc30301-3bdf-4b84-a5ff-7b0516137c2b admin admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:59:12.405328 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc30301-3bdf-4b84-a5ff-7b0516137c2b admin admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:59:12.411683 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f1efb5e-2451-4e4f-9e4f-70de95ff0433 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:59:12.412222 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1efb5e-2451-4e4f-9e4f-70de95ff0433 admin admin] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:59:12.412500 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1efb5e-2451-4e4f-9e4f-70de95ff0433 admin admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:59:12.412770 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1efb5e-2451-4e4f-9e4f-70de95ff0433 admin admin] PATH_INFO: `/v3/roles` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:59:12.413566 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f1efb5e-2451-4e4f-9e4f-70de95ff0433 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:59:12.415093 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f1efb5e-2451-4e4f-9e4f-70de95ff0433 admin admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:59:12.426691 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 1286/2575] 10.222.0.13 () {58 vars in 1122 bytes} [Wed Aug 31 09:59:12 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 09:59:12.431639 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed69a8b7-234a-455c-90db-b3d728831edc admin admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:59:12.442335 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed69a8b7-234a-455c-90db-b3d728831edc admin admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:59:12.446908 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f3d7fc0-98b0-45ca-a905-b793be4126e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:59:12.447553 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3d7fc0-98b0-45ca-a905-b793be4126e9 admin admin] REQUEST_METHOD: `PUT` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:59:12.447828 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3d7fc0-98b0-45ca-a905-b793be4126e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:59:12.448099 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3d7fc0-98b0-45ca-a905-b793be4126e9 admin admin] PATH_INFO: `/v3/projects/75f6da6d2c7a4db39293e4adb38a5990/users/608bc3d8f0a2482fada92205852f3b4c/roles/6c99f5d9f1f243eaa134d8540f21095d` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:59:12.463781 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f3d7fc0-98b0-45ca-a905-b793be4126e9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=75f6da6d2c7a4db39293e4adb38a5990, user_id=608bc3d8f0a2482fada92205852f3b4c, role_id=6c99f5d9f1f243eaa134d8540f21095d)` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:59:12.466548 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f3d7fc0-98b0-45ca-a905-b793be4126e9 admin admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:59:12.480245 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 1290/2576] 10.222.0.13 () {60 vars in 1483 bytes} [Wed Aug 31 09:59:12 2022] PUT /identity/v3/projects/75f6da6d2c7a4db39293e4adb38a5990/users/608bc3d8f0a2482fada92205852f3b4c/roles/6c99f5d9f1f243eaa134d8540f21095d => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:59:12.491158 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81187c9-6ce4-4481-aa09-1ed64f31614e None None] REQUEST_METHOD: `POST` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:59:12.491441 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81187c9-6ce4-4481-aa09-1ed64f31614e None None] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:59:12.491719 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81187c9-6ce4-4481-aa09-1ed64f31614e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:59:12.521363 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.auth.core [None req-c81187c9-6ce4-4481-aa09-1ed64f31614e None None] MFA Rules not processed for user `608bc3d8f0a2482fada92205852f3b4c`. Rule list: `[]` (Enabled: `True`). {{(pid=88113) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:59:12.522812 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.fernet_utils [None req-c81187c9-6ce4-4481-aa09-1ed64f31614e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:59:12.566806 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:DELETE=2,keystone:SELECT=94,keystone:INSERT=4 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:59:12.590968 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 1287/2577] 10.222.0.13 () {58 vars in 958 bytes} [Wed Aug 31 09:59:12 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:59:12.602640 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f3d7fc0-98b0-45ca-a905-b793be4126e9 admin admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:59:12.604359 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.fernet_utils [None req-9f3d7fc0-98b0-45ca-a905-b793be4126e9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88114) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:59:12.651684 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:59:12.666012 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f3d7fc0-98b0-45ca-a905-b793be4126e9 admin admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:59:12.670115 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c08435-e01d-4959-aaae-a0afc8ec5e94 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d52b69c8e94441dacfcff944326f169', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '49548eb44d62415582af820ab66decee', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:59:12.670950 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c08435-e01d-4959-aaae-a0afc8ec5e94 service cinder] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:59:12.671316 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c08435-e01d-4959-aaae-a0afc8ec5e94 service cinder] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:59:12.671681 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c08435-e01d-4959-aaae-a0afc8ec5e94 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:59:12.681666 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9c08435-e01d-4959-aaae-a0afc8ec5e94 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:59:12.683102 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9c08435-e01d-4959-aaae-a0afc8ec5e94 service cinder] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:59:12.692124 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 1291/2578] 10.222.0.13 () {60 vars in 1365 bytes} [Wed Aug 31 09:59:12 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 09:59:16.536064 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c81187c9-6ce4-4481-aa09-1ed64f31614e None None] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:59:16.538175 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.fernet_utils [None req-c81187c9-6ce4-4481-aa09-1ed64f31614e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:59:16.626552 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c81187c9-6ce4-4481-aa09-1ed64f31614e None None] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:59:16.631969 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b530a2-5d36-466f-b2d4-a46924157f30 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc076493239d4ccdbdc37c0369c49529', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '49548eb44d62415582af820ab66decee', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:59:16.632596 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b530a2-5d36-466f-b2d4-a46924157f30 service nova] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:59:16.632874 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b530a2-5d36-466f-b2d4-a46924157f30 service nova] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:59:16.633209 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b530a2-5d36-466f-b2d4-a46924157f30 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:59:16.635082 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.fernet_utils [None req-75b530a2-5d36-466f-b2d4-a46924157f30 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:59:16.650055 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75b530a2-5d36-466f-b2d4-a46924157f30 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:59:16.651707 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75b530a2-5d36-466f-b2d4-a46924157f30 service nova] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:59:16.665826 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 1288/2579] 10.222.0.13 () {60 vars in 1365 bytes} [Wed Aug 31 09:59:16 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:59:17.157431 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b4b8e0-e3c3-4636-8da9-07429f71db5c None None] REQUEST_METHOD: `POST` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:59:17.158006 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b4b8e0-e3c3-4636-8da9-07429f71db5c None None] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:59:17.158006 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b4b8e0-e3c3-4636-8da9-07429f71db5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:59:17.182911 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.auth.core [None req-e3b4b8e0-e3c3-4636-8da9-07429f71db5c None None] MFA Rules not processed for user `1090c027746845239055973b44207556`. Rule list: `[]` (Enabled: `True`). {{(pid=88114) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:59:17.184241 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.fernet_utils [None req-e3b4b8e0-e3c3-4636-8da9-07429f71db5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88114) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:59:17.260124 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 1292/2580] 10.222.0.13 () {58 vars in 958 bytes} [Wed Aug 31 09:59:17 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:59:18.449788 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b530a2-5d36-466f-b2d4-a46924157f30 service nova] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:59:18.515442 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b530a2-5d36-466f-b2d4-a46924157f30 service nova] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:59:18.520924 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c5dcf9-ef25-4c12-8b58-731fff410b8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:59:18.521665 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c5dcf9-ef25-4c12-8b58-731fff410b8c admin admin] REQUEST_METHOD: `DELETE` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:59:18.521946 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c5dcf9-ef25-4c12-8b58-731fff410b8c admin admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:59:18.522223 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c5dcf9-ef25-4c12-8b58-731fff410b8c admin admin] PATH_INFO: `/v3/users/608bc3d8f0a2482fada92205852f3b4c` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:59:18.523393 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85c5dcf9-ef25-4c12-8b58-731fff410b8c admin admin] RBAC: Authorizing `identity:delete_user(user_id=608bc3d8f0a2482fada92205852f3b4c)` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:59:18.525331 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85c5dcf9-ef25-4c12-8b58-731fff410b8c admin admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:59:18.574569 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.notifications [None req-85c5dcf9-ef25-4c12-8b58-731fff410b8c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '608bc3d8f0a2482fada92205852f3b4c'} {{(pid=88113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:59:18.581257 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.notifications [None req-85c5dcf9-ef25-4c12-8b58-731fff410b8c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '608bc3d8f0a2482fada92205852f3b4c'} {{(pid=88113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:59:18.591996 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=92,keystone:DELETE=8 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:59:18.594237 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.notifications [None req-85c5dcf9-ef25-4c12-8b58-731fff410b8c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '608bc3d8f0a2482fada92205852f3b4c'} {{(pid=88113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:59:18.594959 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.notifications [None req-85c5dcf9-ef25-4c12-8b58-731fff410b8c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '608bc3d8f0a2482fada92205852f3b4c'} {{(pid=88113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:59:18.603780 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.notifications [None req-85c5dcf9-ef25-4c12-8b58-731fff410b8c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '608bc3d8f0a2482fada92205852f3b4c'} {{(pid=88113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:59:18.609882 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 1289/2581] 10.222.0.13 () {58 vars in 1224 bytes} [Wed Aug 31 09:59:18 2022] DELETE /identity/v3/users/608bc3d8f0a2482fada92205852f3b4c => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:59:18.616306 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961e3693-1613-4246-b8bf-5528c9b64583 None None] REQUEST_METHOD: `POST` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:59:18.617097 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961e3693-1613-4246-b8bf-5528c9b64583 None None] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:59:18.617259 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961e3693-1613-4246-b8bf-5528c9b64583 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:59:18.638742 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.auth.core [None req-961e3693-1613-4246-b8bf-5528c9b64583 None None] MFA Rules not processed for user `fd836cf590a145df9acf19cf3ae2ac56`. Rule list: `[]` (Enabled: `True`). {{(pid=88114) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:59:18.639934 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.fernet_utils [None req-961e3693-1613-4246-b8bf-5528c9b64583 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88114) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:59:18.705459 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 1293/2582] 10.222.0.13 () {58 vars in 958 bytes} [Wed Aug 31 09:59:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:59:18.715832 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c5dcf9-ef25-4c12-8b58-731fff410b8c admin admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:59:18.717411 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.fernet_utils [None req-85c5dcf9-ef25-4c12-8b58-731fff410b8c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:59:18.788231 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c5dcf9-ef25-4c12-8b58-731fff410b8c admin admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:59:18.792043 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e64545-db93-4fbd-ad21-179a74510173 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9bdaacd597442c2bcde2c555f2cc140', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '49548eb44d62415582af820ab66decee', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:59:18.792669 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e64545-db93-4fbd-ad21-179a74510173 service neutron] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:59:18.792941 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e64545-db93-4fbd-ad21-179a74510173 service neutron] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:59:18.793233 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e64545-db93-4fbd-ad21-179a74510173 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:59:18.804277 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78e64545-db93-4fbd-ad21-179a74510173 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:59:18.805770 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78e64545-db93-4fbd-ad21-179a74510173 service neutron] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:59:18.815933 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 1290/2583] 10.222.0.13 () {60 vars in 1365 bytes} [Wed Aug 31 09:59:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:59:19.130578 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-961e3693-1613-4246-b8bf-5528c9b64583 None None] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:59:19.132022 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.fernet_utils [None req-961e3693-1613-4246-b8bf-5528c9b64583 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88114) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:59:19.145210 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-961e3693-1613-4246-b8bf-5528c9b64583 None None] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:59:19.151226 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c1f691-35c6-4e98-869e-0c21b553f4d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:59:19.151869 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c1f691-35c6-4e98-869e-0c21b553f4d0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:59:19.153435 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c1f691-35c6-4e98-869e-0c21b553f4d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:59:19.153435 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c1f691-35c6-4e98-869e-0c21b553f4d0 admin admin] PATH_INFO: `/v3/projects/75f6da6d2c7a4db39293e4adb38a5990` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:59:19.153679 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4c1f691-35c6-4e98-869e-0c21b553f4d0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=75f6da6d2c7a4db39293e4adb38a5990)` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:59:19.155538 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4c1f691-35c6-4e98-869e-0c21b553f4d0 admin admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:59:19.215638 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.notifications [None req-d4c1f691-35c6-4e98-869e-0c21b553f4d0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '75f6da6d2c7a4db39293e4adb38a5990', 'request_id': 'req-d4c1f691-35c6-4e98-869e-0c21b553f4d0'} {{(pid=88114) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:59:19.224144 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.notifications [None req-d4c1f691-35c6-4e98-869e-0c21b553f4d0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '75f6da6d2c7a4db39293e4adb38a5990', 'request_id': 'req-d4c1f691-35c6-4e98-869e-0c21b553f4d0'} {{(pid=88114) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:59:19.232521 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.notifications [None req-d4c1f691-35c6-4e98-869e-0c21b553f4d0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '75f6da6d2c7a4db39293e4adb38a5990', 'request_id': 'req-d4c1f691-35c6-4e98-869e-0c21b553f4d0'} {{(pid=88114) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:59:19.248460 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.notifications [None req-d4c1f691-35c6-4e98-869e-0c21b553f4d0 admin admin] The token cache is being invalidate because project 75f6da6d2c7a4db39293e4adb38a5990 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88114) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 09:59:19.248750 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.notifications [None req-d4c1f691-35c6-4e98-869e-0c21b553f4d0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88114) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:59:19.249411 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.notifications [None req-d4c1f691-35c6-4e98-869e-0c21b553f4d0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88114) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:59:19.253430 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 1294/2584] 10.222.0.13 () {58 vars in 1233 bytes} [Wed Aug 31 09:59:19 2022] DELETE /identity/v3/projects/75f6da6d2c7a4db39293e4adb38a5990 => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:59:19.258606 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b60cad-43a8-4a7c-9ec1-e3f670e2c668 None None] REQUEST_METHOD: `POST` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:59:19.258983 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b60cad-43a8-4a7c-9ec1-e3f670e2c668 None None] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:59:19.259354 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b60cad-43a8-4a7c-9ec1-e3f670e2c668 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:59:19.290229 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.auth.core [None req-93b60cad-43a8-4a7c-9ec1-e3f670e2c668 None None] MFA Rules not processed for user `fd836cf590a145df9acf19cf3ae2ac56`. Rule list: `[]` (Enabled: `True`). {{(pid=88113) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:59:19.291552 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.fernet_utils [None req-93b60cad-43a8-4a7c-9ec1-e3f670e2c668 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:59:19.375233 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 1291/2585] 10.222.0.13 () {58 vars in 958 bytes} [Wed Aug 31 09:59:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 09:59:19.384663 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697a531b-1ad5-441e-90ec-b3b0eaf78882 None None] REQUEST_METHOD: `POST` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:59:19.384942 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697a531b-1ad5-441e-90ec-b3b0eaf78882 None None] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:59:19.385209 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697a531b-1ad5-441e-90ec-b3b0eaf78882 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:59:19.408177 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.auth.core [None req-697a531b-1ad5-441e-90ec-b3b0eaf78882 None None] MFA Rules not processed for user `fd836cf590a145df9acf19cf3ae2ac56`. Rule list: `[]` (Enabled: `True`). {{(pid=88114) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:59:19.410923 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.fernet_utils [None req-697a531b-1ad5-441e-90ec-b3b0eaf78882 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88114) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:59:19.425264 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 1295/2586] 10.222.0.13 () {58 vars in 958 bytes} [Wed Aug 31 09:59:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:59:19.430601 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b60cad-43a8-4a7c-9ec1-e3f670e2c668 None None] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:59:19.441431 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b60cad-43a8-4a7c-9ec1-e3f670e2c668 None None] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:59:19.446515 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc538cb-b512-45c2-953e-80b96c64fffe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:59:19.447161 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc538cb-b512-45c2-953e-80b96c64fffe admin admin] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:59:19.447442 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc538cb-b512-45c2-953e-80b96c64fffe admin admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:59:19.447712 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc538cb-b512-45c2-953e-80b96c64fffe admin admin] PATH_INFO: `/v3/domains` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:59:19.448381 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dc538cb-b512-45c2-953e-80b96c64fffe admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88113) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 09:59:19.448679 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dc538cb-b512-45c2-953e-80b96c64fffe admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:59:19.450364 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dc538cb-b512-45c2-953e-80b96c64fffe admin admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:59:19.463784 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 1292/2587] 10.222.0.13 () {58 vars in 1153 bytes} [Wed Aug 31 09:59:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 09:59:19.468480 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-697a531b-1ad5-441e-90ec-b3b0eaf78882 None None] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:59:19.479042 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-697a531b-1ad5-441e-90ec-b3b0eaf78882 None None] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:59:19.483887 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-298a2437-93a3-4112-8103-348b89b5e23a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:59:19.484496 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298a2437-93a3-4112-8103-348b89b5e23a admin admin] REQUEST_METHOD: `POST` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:59:19.484770 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298a2437-93a3-4112-8103-348b89b5e23a admin admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:59:19.485035 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298a2437-93a3-4112-8103-348b89b5e23a admin admin] PATH_INFO: `/v3/projects` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:59:19.485708 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-298a2437-93a3-4112-8103-348b89b5e23a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:59:19.487621 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-298a2437-93a3-4112-8103-348b89b5e23a admin admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:59:19.493443 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:59:19.522717 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 1296/2588] 10.222.0.13 () {60 vars in 1153 bytes} [Wed Aug 31 09:59:19 2022] POST /identity/v3/projects => generated 397 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 09:59:19.524289 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc538cb-b512-45c2-953e-80b96c64fffe admin admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:59:19.534515 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc538cb-b512-45c2-953e-80b96c64fffe admin admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:59:19.595650 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ed4c6d-eed2-42ed-a42c-f94844c14b4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:59:19.596265 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ed4c6d-eed2-42ed-a42c-f94844c14b4a admin admin] REQUEST_METHOD: `POST` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:59:19.596539 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ed4c6d-eed2-42ed-a42c-f94844c14b4a admin admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:59:19.596807 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ed4c6d-eed2-42ed-a42c-f94844c14b4a admin admin] PATH_INFO: `/v3/users` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:59:19.597468 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79ed4c6d-eed2-42ed-a42c-f94844c14b4a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:59:19.600062 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79ed4c6d-eed2-42ed-a42c-f94844c14b4a admin admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:59:19.624730 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 1293/2589] 10.222.0.13 () {60 vars in 1144 bytes} [Wed Aug 31 09:59:19 2022] POST /identity/v3/users => generated 299 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 09:59:19.629027 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-298a2437-93a3-4112-8103-348b89b5e23a admin admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:59:19.640121 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-298a2437-93a3-4112-8103-348b89b5e23a admin admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:59:19.645924 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b8ee82f-5074-451f-a9dd-08232c00f7e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:59:19.646564 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8ee82f-5074-451f-a9dd-08232c00f7e8 admin admin] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:59:19.646834 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8ee82f-5074-451f-a9dd-08232c00f7e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:59:19.647103 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8ee82f-5074-451f-a9dd-08232c00f7e8 admin admin] PATH_INFO: `/v3/roles` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:59:19.647752 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b8ee82f-5074-451f-a9dd-08232c00f7e8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:59:19.649816 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b8ee82f-5074-451f-a9dd-08232c00f7e8 admin admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:59:19.662710 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 1297/2590] 10.222.0.13 () {58 vars in 1122 bytes} [Wed Aug 31 09:59:19 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 09:59:19.667279 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ed4c6d-eed2-42ed-a42c-f94844c14b4a admin admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:59:19.678176 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ed4c6d-eed2-42ed-a42c-f94844c14b4a admin admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:59:19.683166 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e723864d-59ad-43f1-8af7-83cedcc478b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:59:19.683786 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e723864d-59ad-43f1-8af7-83cedcc478b1 admin admin] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:59:19.684072 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e723864d-59ad-43f1-8af7-83cedcc478b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:59:19.684345 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e723864d-59ad-43f1-8af7-83cedcc478b1 admin admin] PATH_INFO: `/v3/roles` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:59:19.685172 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e723864d-59ad-43f1-8af7-83cedcc478b1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:59:19.686663 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e723864d-59ad-43f1-8af7-83cedcc478b1 admin admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:59:19.688213 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:DELETE=2,keystone:SELECT=94,keystone:INSERT=4 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:59:19.700215 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 1294/2591] 10.222.0.13 () {58 vars in 1122 bytes} [Wed Aug 31 09:59:19 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 09:59:19.704426 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b8ee82f-5074-451f-a9dd-08232c00f7e8 admin admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:59:19.715284 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b8ee82f-5074-451f-a9dd-08232c00f7e8 admin admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:59:19.720529 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-741e666f-55ff-43b6-833c-65c084461ea1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:59:19.722587 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741e666f-55ff-43b6-833c-65c084461ea1 admin admin] REQUEST_METHOD: `PUT` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:59:19.722587 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741e666f-55ff-43b6-833c-65c084461ea1 admin admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:59:19.722587 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741e666f-55ff-43b6-833c-65c084461ea1 admin admin] PATH_INFO: `/v3/projects/d489cbde458b4bbca7d35469bd9c822e/users/d11c9d332a254394a9dca6b821531f41/roles/6c99f5d9f1f243eaa134d8540f21095d` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:59:19.742023 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-741e666f-55ff-43b6-833c-65c084461ea1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d489cbde458b4bbca7d35469bd9c822e, user_id=d11c9d332a254394a9dca6b821531f41, role_id=6c99f5d9f1f243eaa134d8540f21095d)` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:59:19.744705 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-741e666f-55ff-43b6-833c-65c084461ea1 admin admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:59:19.760380 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 1298/2592] 10.222.0.13 () {60 vars in 1483 bytes} [Wed Aug 31 09:59:19 2022] PUT /identity/v3/projects/d489cbde458b4bbca7d35469bd9c822e/users/d11c9d332a254394a9dca6b821531f41/roles/6c99f5d9f1f243eaa134d8540f21095d => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:59:19.771582 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d2ccdc-8687-4d4c-a248-0aa0819d8c4f None None] REQUEST_METHOD: `POST` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:59:19.772076 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d2ccdc-8687-4d4c-a248-0aa0819d8c4f None None] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:59:19.772458 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d2ccdc-8687-4d4c-a248-0aa0819d8c4f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:59:19.806671 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.auth.core [None req-47d2ccdc-8687-4d4c-a248-0aa0819d8c4f None None] MFA Rules not processed for user `d11c9d332a254394a9dca6b821531f41`. Rule list: `[]` (Enabled: `True`). {{(pid=88113) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:59:19.808088 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.fernet_utils [None req-47d2ccdc-8687-4d4c-a248-0aa0819d8c4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:59:19.896027 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 1295/2593] 10.222.0.13 () {58 vars in 958 bytes} [Wed Aug 31 09:59:19 2022] POST /identity/v3/auth/tokens => generated 3274 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:59:19.908879 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-741e666f-55ff-43b6-833c-65c084461ea1 admin admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:59:19.911628 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.fernet_utils [None req-741e666f-55ff-43b6-833c-65c084461ea1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88114) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:59:20.001769 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c15f7ce-d98b-4a44-adbb-3f2b1c8ae211 None None] REQUEST_METHOD: `POST` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:59:20.002083 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c15f7ce-d98b-4a44-adbb-3f2b1c8ae211 None None] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:59:20.002467 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c15f7ce-d98b-4a44-adbb-3f2b1c8ae211 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:59:20.027987 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.auth.core [None req-0c15f7ce-d98b-4a44-adbb-3f2b1c8ae211 None None] MFA Rules not processed for user `768cb67c1ef24251aaa113c12d81a250`. Rule list: `[]` (Enabled: `True`). {{(pid=88113) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:59:20.029403 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.fernet_utils [None req-0c15f7ce-d98b-4a44-adbb-3f2b1c8ae211 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:59:20.044922 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-741e666f-55ff-43b6-833c-65c084461ea1 admin admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:59:20.052868 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8eb87e1-ab7f-490a-b582-b2991f0da494 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d52b69c8e94441dacfcff944326f169', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '49548eb44d62415582af820ab66decee', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:59:20.054319 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8eb87e1-ab7f-490a-b582-b2991f0da494 service cinder] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:59:20.054769 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8eb87e1-ab7f-490a-b582-b2991f0da494 service cinder] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:59:20.055215 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8eb87e1-ab7f-490a-b582-b2991f0da494 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:59:20.073829 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8eb87e1-ab7f-490a-b582-b2991f0da494 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:59:20.075896 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8eb87e1-ab7f-490a-b582-b2991f0da494 service cinder] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:59:20.087972 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 1299/2594] 10.222.0.13 () {60 vars in 1365 bytes} [Wed Aug 31 09:59:19 2022] GET /identity/v3/auth/tokens => generated 3274 bytes in 180 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:59:20.136819 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 1296/2595] 10.222.0.13 () {58 vars in 967 bytes} [Wed Aug 31 09:59:20 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:59:20.154344 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8eb87e1-ab7f-490a-b582-b2991f0da494 service cinder] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:59:20.156088 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.fernet_utils [None req-e8eb87e1-ab7f-490a-b582-b2991f0da494 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88114) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:59:20.288383 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8eb87e1-ab7f-490a-b582-b2991f0da494 service cinder] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:59:20.296594 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4918974f-de6b-4db1-8566-4b2048facd61 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '202bb9db7a8a4977ba788a290ed9a3fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '49548eb44d62415582af820ab66decee', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:59:20.297909 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4918974f-de6b-4db1-8566-4b2048facd61 service swift] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:59:20.298490 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4918974f-de6b-4db1-8566-4b2048facd61 service swift] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:59:20.298968 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4918974f-de6b-4db1-8566-4b2048facd61 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:59:20.313246 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4918974f-de6b-4db1-8566-4b2048facd61 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:59:20.315844 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4918974f-de6b-4db1-8566-4b2048facd61 service swift] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:59:20.336262 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 1300/2596] 10.222.0.13 () {60 vars in 1384 bytes} [Wed Aug 31 09:59:20 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 182 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 31 09:59:29.581716 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c15f7ce-d98b-4a44-adbb-3f2b1c8ae211 None None] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:59:29.681836 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c15f7ce-d98b-4a44-adbb-3f2b1c8ae211 None None] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:59:29.686517 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad98ccb-3897-467a-9344-1e06a78a2564 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:59:29.687170 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad98ccb-3897-467a-9344-1e06a78a2564 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:59:29.687441 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad98ccb-3897-467a-9344-1e06a78a2564 admin admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:59:29.687713 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad98ccb-3897-467a-9344-1e06a78a2564 admin admin] PATH_INFO: `/v3/users/d11c9d332a254394a9dca6b821531f41` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:59:29.688897 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ad98ccb-3897-467a-9344-1e06a78a2564 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d11c9d332a254394a9dca6b821531f41)` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:59:29.690915 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ad98ccb-3897-467a-9344-1e06a78a2564 admin admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:59:29.734045 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:59:29.753281 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.notifications [None req-7ad98ccb-3897-467a-9344-1e06a78a2564 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd11c9d332a254394a9dca6b821531f41'} {{(pid=88113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:59:29.762583 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.notifications [None req-7ad98ccb-3897-467a-9344-1e06a78a2564 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd11c9d332a254394a9dca6b821531f41'} {{(pid=88113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:59:29.776234 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.notifications [None req-7ad98ccb-3897-467a-9344-1e06a78a2564 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd11c9d332a254394a9dca6b821531f41'} {{(pid=88113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:59:29.776987 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.notifications [None req-7ad98ccb-3897-467a-9344-1e06a78a2564 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd11c9d332a254394a9dca6b821531f41'} {{(pid=88113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:59:29.788431 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.notifications [None req-7ad98ccb-3897-467a-9344-1e06a78a2564 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd11c9d332a254394a9dca6b821531f41'} {{(pid=88113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:59:29.795748 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 1297/2597] 10.222.0.13 () {58 vars in 1224 bytes} [Wed Aug 31 09:59:29 2022] DELETE /identity/v3/users/d11c9d332a254394a9dca6b821531f41 => generated 0 bytes in 215 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:59:29.801552 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bda3b8f-69ab-42aa-810f-97cee9f46034 None None] REQUEST_METHOD: `POST` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:59:29.802196 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bda3b8f-69ab-42aa-810f-97cee9f46034 None None] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:59:29.802196 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bda3b8f-69ab-42aa-810f-97cee9f46034 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:59:29.822411 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.auth.core [None req-2bda3b8f-69ab-42aa-810f-97cee9f46034 None None] MFA Rules not processed for user `fd836cf590a145df9acf19cf3ae2ac56`. Rule list: `[]` (Enabled: `True`). {{(pid=88114) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 09:59:29.823763 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.fernet_utils [None req-2bda3b8f-69ab-42aa-810f-97cee9f46034 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88114) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:59:29.874119 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:59:29.898360 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 1301/2598] 10.222.0.13 () {58 vars in 958 bytes} [Wed Aug 31 09:59:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 09:59:29.909367 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad98ccb-3897-467a-9344-1e06a78a2564 admin admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:59:29.911097 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.fernet_utils [None req-7ad98ccb-3897-467a-9344-1e06a78a2564 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:59:29.977360 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad98ccb-3897-467a-9344-1e06a78a2564 admin admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:59:29.981300 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b57bcbb1-515a-4e25-a505-ef48bb53f19d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9bdaacd597442c2bcde2c555f2cc140', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '49548eb44d62415582af820ab66decee', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:59:29.981947 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57bcbb1-515a-4e25-a505-ef48bb53f19d service neutron] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:59:29.982292 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57bcbb1-515a-4e25-a505-ef48bb53f19d service neutron] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:59:29.982484 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57bcbb1-515a-4e25-a505-ef48bb53f19d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:59:29.993490 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b57bcbb1-515a-4e25-a505-ef48bb53f19d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:59:29.994866 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b57bcbb1-515a-4e25-a505-ef48bb53f19d service neutron] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:59:30.006771 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 1298/2599] 10.222.0.13 () {60 vars in 1365 bytes} [Wed Aug 31 09:59:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:59:30.380628 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bda3b8f-69ab-42aa-810f-97cee9f46034 None None] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:59:30.383941 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.fernet_utils [None req-2bda3b8f-69ab-42aa-810f-97cee9f46034 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88114) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:59:30.397109 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bda3b8f-69ab-42aa-810f-97cee9f46034 None None] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:59:30.407689 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e90fcc4-15d2-4b21-a319-89cc8ccb5abb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:59:30.408346 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e90fcc4-15d2-4b21-a319-89cc8ccb5abb admin admin] REQUEST_METHOD: `DELETE` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:59:30.408612 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e90fcc4-15d2-4b21-a319-89cc8ccb5abb admin admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:59:30.408882 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e90fcc4-15d2-4b21-a319-89cc8ccb5abb admin admin] PATH_INFO: `/v3/projects/d489cbde458b4bbca7d35469bd9c822e` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:59:30.410155 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e90fcc4-15d2-4b21-a319-89cc8ccb5abb admin admin] RBAC: Authorizing `identity:delete_project(project_id=d489cbde458b4bbca7d35469bd9c822e)` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:59:30.412158 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e90fcc4-15d2-4b21-a319-89cc8ccb5abb admin admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:59:30.463262 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.notifications [None req-9e90fcc4-15d2-4b21-a319-89cc8ccb5abb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd489cbde458b4bbca7d35469bd9c822e', 'request_id': 'req-9e90fcc4-15d2-4b21-a319-89cc8ccb5abb'} {{(pid=88114) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:59:30.469895 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.notifications [None req-9e90fcc4-15d2-4b21-a319-89cc8ccb5abb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd489cbde458b4bbca7d35469bd9c822e', 'request_id': 'req-9e90fcc4-15d2-4b21-a319-89cc8ccb5abb'} {{(pid=88114) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:59:30.476743 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.notifications [None req-9e90fcc4-15d2-4b21-a319-89cc8ccb5abb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd489cbde458b4bbca7d35469bd9c822e', 'request_id': 'req-9e90fcc4-15d2-4b21-a319-89cc8ccb5abb'} {{(pid=88114) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:59:30.491137 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.notifications [None req-9e90fcc4-15d2-4b21-a319-89cc8ccb5abb admin admin] The token cache is being invalidate because project d489cbde458b4bbca7d35469bd9c822e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88114) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 09:59:30.491438 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.notifications [None req-9e90fcc4-15d2-4b21-a319-89cc8ccb5abb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88114) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:59:30.492070 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.notifications [None req-9e90fcc4-15d2-4b21-a319-89cc8ccb5abb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88114) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 09:59:30.493632 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 1302/2600] 10.222.0.13 () {58 vars in 1233 bytes} [Wed Aug 31 09:59:30 2022] DELETE /identity/v3/projects/d489cbde458b4bbca7d35469bd9c822e => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 09:59:40.000737 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=43,keystone:DELETE=5,keystone:INSERT=1 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:59:40.486844 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=35,keystone:DELETE=7,keystone:INSERT=1 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:59:41.182418 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b57bcbb1-515a-4e25-a505-ef48bb53f19d service neutron] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:59:41.183863 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.fernet_utils [None req-b57bcbb1-515a-4e25-a505-ef48bb53f19d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:59:41.195631 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b57bcbb1-515a-4e25-a505-ef48bb53f19d service neutron] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:59:41.248911 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a45162-8580-49f7-bd7b-573213b080b0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9bdaacd597442c2bcde2c555f2cc140', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '49548eb44d62415582af820ab66decee', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:59:41.249564 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a45162-8580-49f7-bd7b-573213b080b0 service neutron] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:59:41.249847 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a45162-8580-49f7-bd7b-573213b080b0 service neutron] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:59:41.250125 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a45162-8580-49f7-bd7b-573213b080b0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:59:41.251885 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.fernet_utils [None req-80a45162-8580-49f7-bd7b-573213b080b0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:59:41.315778 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80a45162-8580-49f7-bd7b-573213b080b0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:59:41.317439 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80a45162-8580-49f7-bd7b-573213b080b0 service neutron] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:59:41.327768 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 1299/2601] 10.222.0.13 () {60 vars in 1365 bytes} [Wed Aug 31 09:59:41 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:59:43.164016 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e90fcc4-15d2-4b21-a319-89cc8ccb5abb admin admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:59:43.165498 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.fernet_utils [None req-9e90fcc4-15d2-4b21-a319-89cc8ccb5abb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88114) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:59:43.243519 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e90fcc4-15d2-4b21-a319-89cc8ccb5abb admin admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:59:43.249092 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-890f306d-0241-4da0-b593-aac803776a93 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc076493239d4ccdbdc37c0369c49529', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '49548eb44d62415582af820ab66decee', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:59:43.249734 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890f306d-0241-4da0-b593-aac803776a93 service nova] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:59:43.250014 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890f306d-0241-4da0-b593-aac803776a93 service nova] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:59:43.250289 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890f306d-0241-4da0-b593-aac803776a93 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:59:43.252043 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.fernet_utils [None req-890f306d-0241-4da0-b593-aac803776a93 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88114) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:59:43.265878 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-890f306d-0241-4da0-b593-aac803776a93 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:59:43.267474 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-890f306d-0241-4da0-b593-aac803776a93 service nova] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:59:43.279227 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 1303/2602] 10.222.0.13 () {60 vars in 1365 bytes} [Wed Aug 31 09:59:43 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 09:59:51.322986 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=50 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:59:51.418240 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a45162-8580-49f7-bd7b-573213b080b0 service neutron] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:59:51.430720 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a45162-8580-49f7-bd7b-573213b080b0 service neutron] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:59:51.436781 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f0dad2-3309-4786-bb20-c6860a05fe78 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc076493239d4ccdbdc37c0369c49529', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '49548eb44d62415582af820ab66decee', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:59:51.437549 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f0dad2-3309-4786-bb20-c6860a05fe78 service nova] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:59:51.437888 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f0dad2-3309-4786-bb20-c6860a05fe78 service nova] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:59:51.438162 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f0dad2-3309-4786-bb20-c6860a05fe78 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:59:51.439924 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.fernet_utils [None req-40f0dad2-3309-4786-bb20-c6860a05fe78 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:59:51.456317 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40f0dad2-3309-4786-bb20-c6860a05fe78 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:59:51.458805 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40f0dad2-3309-4786-bb20-c6860a05fe78 service nova] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:59:51.484956 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 1300/2603] 10.222.0.13 () {60 vars in 1365 bytes} [Wed Aug 31 09:59:51 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 67 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 09:59:53.274667 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=34 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 09:59:57.498636 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-890f306d-0241-4da0-b593-aac803776a93 service nova] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 09:59:57.500112 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.fernet_utils [None req-890f306d-0241-4da0-b593-aac803776a93 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88114) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:59:57.512228 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-890f306d-0241-4da0-b593-aac803776a93 service nova] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 09:59:57.516523 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4379a71b-2fdf-4936-88d4-49de8b6acff8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9bdaacd597442c2bcde2c555f2cc140', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '49548eb44d62415582af820ab66decee', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 09:59:57.517121 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4379a71b-2fdf-4936-88d4-49de8b6acff8 service neutron] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 09:59:57.517814 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4379a71b-2fdf-4936-88d4-49de8b6acff8 service neutron] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 09:59:57.517814 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4379a71b-2fdf-4936-88d4-49de8b6acff8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 09:59:57.520148 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.fernet_utils [None req-4379a71b-2fdf-4936-88d4-49de8b6acff8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88114) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 09:59:57.587066 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4379a71b-2fdf-4936-88d4-49de8b6acff8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 09:59:57.588509 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4379a71b-2fdf-4936-88d4-49de8b6acff8 service neutron] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 09:59:57.598976 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 1304/2604] 10.222.0.13 () {60 vars in 1365 bytes} [Wed Aug 31 09:59:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 10:00:00.551666 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f0dad2-3309-4786-bb20-c6860a05fe78 service nova] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:00:00.553210 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.fernet_utils [None req-40f0dad2-3309-4786-bb20-c6860a05fe78 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:00:00.569720 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f0dad2-3309-4786-bb20-c6860a05fe78 service nova] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:00:00.577710 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd750f2b-1600-452f-ac1d-976c0cb99e37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:00:00.578321 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd750f2b-1600-452f-ac1d-976c0cb99e37 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:00:00.578605 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd750f2b-1600-452f-ac1d-976c0cb99e37 admin admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:00:00.578884 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd750f2b-1600-452f-ac1d-976c0cb99e37 admin admin] PATH_INFO: `/v3/users/25b4233e5f0e41bf99b3fb3f5f6229c9` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:00:00.594758 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd750f2b-1600-452f-ac1d-976c0cb99e37 admin admin] RBAC: Authorizing `identity:delete_user(user_id=25b4233e5f0e41bf99b3fb3f5f6229c9)` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:00:00.597407 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd750f2b-1600-452f-ac1d-976c0cb99e37 admin admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:00:00.649781 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.notifications [None req-dd750f2b-1600-452f-ac1d-976c0cb99e37 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '25b4233e5f0e41bf99b3fb3f5f6229c9'} {{(pid=88113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 10:00:00.656959 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.notifications [None req-dd750f2b-1600-452f-ac1d-976c0cb99e37 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '25b4233e5f0e41bf99b3fb3f5f6229c9'} {{(pid=88113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 10:00:00.669933 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.notifications [None req-dd750f2b-1600-452f-ac1d-976c0cb99e37 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '25b4233e5f0e41bf99b3fb3f5f6229c9'} {{(pid=88113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 10:00:00.670639 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.notifications [None req-dd750f2b-1600-452f-ac1d-976c0cb99e37 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '25b4233e5f0e41bf99b3fb3f5f6229c9'} {{(pid=88113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 10:00:00.678706 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.notifications [None req-dd750f2b-1600-452f-ac1d-976c0cb99e37 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '25b4233e5f0e41bf99b3fb3f5f6229c9'} {{(pid=88113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 10:00:00.684728 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 1301/2605] 10.222.0.13 () {58 vars in 1224 bytes} [Wed Aug 31 10:00:00 2022] DELETE /identity/v3/users/25b4233e5f0e41bf99b3fb3f5f6229c9 => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 10:00:00.866838 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4379a71b-2fdf-4936-88d4-49de8b6acff8 service neutron] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:00:00.879553 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4379a71b-2fdf-4936-88d4-49de8b6acff8 service neutron] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:00:00.942135 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d1204e4-ae22-4738-8c67-bb6498936d53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:00:00.942784 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1204e4-ae22-4738-8c67-bb6498936d53 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:00:00.943062 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1204e4-ae22-4738-8c67-bb6498936d53 admin admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:00:00.943335 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1204e4-ae22-4738-8c67-bb6498936d53 admin admin] PATH_INFO: `/v3/projects/5ed9143757584694a5139ada8dbd5570` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:00:00.958754 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d1204e4-ae22-4738-8c67-bb6498936d53 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5ed9143757584694a5139ada8dbd5570)` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:00:00.961076 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d1204e4-ae22-4738-8c67-bb6498936d53 admin admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:00:01.013914 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.notifications [None req-2d1204e4-ae22-4738-8c67-bb6498936d53 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5ed9143757584694a5139ada8dbd5570', 'request_id': 'req-2d1204e4-ae22-4738-8c67-bb6498936d53'} {{(pid=88114) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 10:00:01.023850 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.notifications [None req-2d1204e4-ae22-4738-8c67-bb6498936d53 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5ed9143757584694a5139ada8dbd5570', 'request_id': 'req-2d1204e4-ae22-4738-8c67-bb6498936d53'} {{(pid=88114) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 10:00:01.029947 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.notifications [None req-2d1204e4-ae22-4738-8c67-bb6498936d53 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5ed9143757584694a5139ada8dbd5570', 'request_id': 'req-2d1204e4-ae22-4738-8c67-bb6498936d53'} {{(pid=88114) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 10:00:01.044272 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.notifications [None req-2d1204e4-ae22-4738-8c67-bb6498936d53 admin admin] The token cache is being invalidate because project 5ed9143757584694a5139ada8dbd5570 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88114) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 10:00:01.044571 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.notifications [None req-2d1204e4-ae22-4738-8c67-bb6498936d53 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88114) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 10:00:01.045230 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.notifications [None req-2d1204e4-ae22-4738-8c67-bb6498936d53 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88114) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 10:00:01.047080 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 1305/2606] 10.222.0.13 () {58 vars in 1233 bytes} [Wed Aug 31 10:00:00 2022] DELETE /identity/v3/projects/5ed9143757584694a5139ada8dbd5570 => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 10:00:01.054219 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65b216a-5577-45f3-a185-cfc748b411e6 None None] REQUEST_METHOD: `POST` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:00:01.054648 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65b216a-5577-45f3-a185-cfc748b411e6 None None] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:00:01.055060 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65b216a-5577-45f3-a185-cfc748b411e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:00:01.080117 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.auth.core [None req-b65b216a-5577-45f3-a185-cfc748b411e6 None None] MFA Rules not processed for user `fd836cf590a145df9acf19cf3ae2ac56`. Rule list: `[]` (Enabled: `True`). {{(pid=88113) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 10:00:01.081920 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.fernet_utils [None req-b65b216a-5577-45f3-a185-cfc748b411e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:00:01.144618 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 1302/2607] 10.222.0.13 () {58 vars in 958 bytes} [Wed Aug 31 10:00:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 10:00:01.152475 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73eb88a-22b5-4ca1-b6c5-d1be1bca1b7f None None] REQUEST_METHOD: `POST` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:00:01.152761 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73eb88a-22b5-4ca1-b6c5-d1be1bca1b7f None None] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:00:01.153036 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73eb88a-22b5-4ca1-b6c5-d1be1bca1b7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:00:01.173928 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.auth.core [None req-d73eb88a-22b5-4ca1-b6c5-d1be1bca1b7f None None] MFA Rules not processed for user `fd836cf590a145df9acf19cf3ae2ac56`. Rule list: `[]` (Enabled: `True`). {{(pid=88114) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 10:00:01.176128 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.fernet_utils [None req-d73eb88a-22b5-4ca1-b6c5-d1be1bca1b7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88114) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:00:01.183256 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 1306/2608] 10.222.0.13 () {58 vars in 958 bytes} [Wed Aug 31 10:00:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 10:00:01.187504 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b65b216a-5577-45f3-a185-cfc748b411e6 None None] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:00:01.197501 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b65b216a-5577-45f3-a185-cfc748b411e6 None None] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:00:01.202100 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5334584-fa42-4ab2-b6f6-b33a769f4724 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:00:01.202692 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5334584-fa42-4ab2-b6f6-b33a769f4724 admin admin] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:00:01.202968 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5334584-fa42-4ab2-b6f6-b33a769f4724 admin admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:00:01.203238 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5334584-fa42-4ab2-b6f6-b33a769f4724 admin admin] PATH_INFO: `/v3/domains` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:00:01.203881 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5334584-fa42-4ab2-b6f6-b33a769f4724 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88113) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:00:01.204208 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5334584-fa42-4ab2-b6f6-b33a769f4724 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:00:01.205863 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5334584-fa42-4ab2-b6f6-b33a769f4724 admin admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:00:01.218419 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 1303/2609] 10.222.0.13 () {58 vars in 1153 bytes} [Wed Aug 31 10:00:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:00:01.223835 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d73eb88a-22b5-4ca1-b6c5-d1be1bca1b7f None None] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:00:01.233025 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d73eb88a-22b5-4ca1-b6c5-d1be1bca1b7f None None] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:00:01.238224 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb4704f1-926e-41b5-870e-e3dd67a5b4e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:00:01.239082 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4704f1-926e-41b5-870e-e3dd67a5b4e2 admin admin] REQUEST_METHOD: `POST` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:00:01.239357 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4704f1-926e-41b5-870e-e3dd67a5b4e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:00:01.239625 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4704f1-926e-41b5-870e-e3dd67a5b4e2 admin admin] PATH_INFO: `/v3/projects` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:00:01.240250 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb4704f1-926e-41b5-870e-e3dd67a5b4e2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:00:01.242174 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb4704f1-926e-41b5-870e-e3dd67a5b4e2 admin admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:00:01.247529 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:00:01.271484 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 1307/2610] 10.222.0.13 () {60 vars in 1153 bytes} [Wed Aug 31 10:00:01 2022] POST /identity/v3/projects => generated 379 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 10:00:01.275698 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5334584-fa42-4ab2-b6f6-b33a769f4724 admin admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:00:01.285953 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5334584-fa42-4ab2-b6f6-b33a769f4724 admin admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:00:01.318195 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:00:01.344607 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e65438-9e4a-47e2-86ad-5275f7c96a57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:00:01.345202 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e65438-9e4a-47e2-86ad-5275f7c96a57 admin admin] REQUEST_METHOD: `POST` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:00:01.345507 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e65438-9e4a-47e2-86ad-5275f7c96a57 admin admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:00:01.345785 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e65438-9e4a-47e2-86ad-5275f7c96a57 admin admin] PATH_INFO: `/v3/users` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:00:01.346414 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99e65438-9e4a-47e2-86ad-5275f7c96a57 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:00:01.348454 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99e65438-9e4a-47e2-86ad-5275f7c96a57 admin admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:00:01.369369 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 1304/2611] 10.222.0.13 () {60 vars in 1144 bytes} [Wed Aug 31 10:00:01 2022] POST /identity/v3/users => generated 290 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 10:00:01.373492 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb4704f1-926e-41b5-870e-e3dd67a5b4e2 admin admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:00:01.384303 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb4704f1-926e-41b5-870e-e3dd67a5b4e2 admin admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:00:01.389086 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b843be4-3aee-425b-bd8e-58a87451e09d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:00:01.389709 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b843be4-3aee-425b-bd8e-58a87451e09d admin admin] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:00:01.389986 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b843be4-3aee-425b-bd8e-58a87451e09d admin admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:00:01.390255 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b843be4-3aee-425b-bd8e-58a87451e09d admin admin] PATH_INFO: `/v3/roles` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:00:01.390900 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b843be4-3aee-425b-bd8e-58a87451e09d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:00:01.392511 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b843be4-3aee-425b-bd8e-58a87451e09d admin admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:00:01.403468 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 1308/2612] 10.222.0.13 () {58 vars in 1122 bytes} [Wed Aug 31 10:00:01 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 10:00:01.407422 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e65438-9e4a-47e2-86ad-5275f7c96a57 admin admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:00:01.417995 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e65438-9e4a-47e2-86ad-5275f7c96a57 admin admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:00:01.422585 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a46135-43f0-4b2b-ae18-ebcdbdbcfdcc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:00:01.423178 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a46135-43f0-4b2b-ae18-ebcdbdbcfdcc admin admin] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:00:01.423451 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a46135-43f0-4b2b-ae18-ebcdbdbcfdcc admin admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:00:01.423717 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a46135-43f0-4b2b-ae18-ebcdbdbcfdcc admin admin] PATH_INFO: `/v3/roles` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:00:01.424402 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3a46135-43f0-4b2b-ae18-ebcdbdbcfdcc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:00:01.426680 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3a46135-43f0-4b2b-ae18-ebcdbdbcfdcc admin admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:00:01.437548 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 1305/2613] 10.222.0.13 () {58 vars in 1122 bytes} [Wed Aug 31 10:00:01 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 10:00:01.441744 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b843be4-3aee-425b-bd8e-58a87451e09d admin admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:00:01.451623 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b843be4-3aee-425b-bd8e-58a87451e09d admin admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:00:01.456498 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61464bbe-2252-4d34-b553-a29de885115e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:00:01.457131 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61464bbe-2252-4d34-b553-a29de885115e admin admin] REQUEST_METHOD: `PUT` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:00:01.457442 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61464bbe-2252-4d34-b553-a29de885115e admin admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:00:01.457721 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61464bbe-2252-4d34-b553-a29de885115e admin admin] PATH_INFO: `/v3/projects/fcb0af9875ed40fc83260ff500d0bc4b/users/fe70a84d6043403a88557bec74566463/roles/6c99f5d9f1f243eaa134d8540f21095d` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:00:01.472984 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61464bbe-2252-4d34-b553-a29de885115e admin admin] RBAC: Authorizing `identity:create_grant(project_id=fcb0af9875ed40fc83260ff500d0bc4b, user_id=fe70a84d6043403a88557bec74566463, role_id=6c99f5d9f1f243eaa134d8540f21095d)` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:00:01.475814 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61464bbe-2252-4d34-b553-a29de885115e admin admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:00:01.490272 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 1309/2614] 10.222.0.13 () {60 vars in 1483 bytes} [Wed Aug 31 10:00:01 2022] PUT /identity/v3/projects/fcb0af9875ed40fc83260ff500d0bc4b/users/fe70a84d6043403a88557bec74566463/roles/6c99f5d9f1f243eaa134d8540f21095d => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 10:00:01.501494 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54e9f48-382d-4ba7-a13a-06c1c63a896f None None] REQUEST_METHOD: `POST` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:00:01.501784 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54e9f48-382d-4ba7-a13a-06c1c63a896f None None] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:00:01.502056 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54e9f48-382d-4ba7-a13a-06c1c63a896f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:00:01.530722 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.auth.core [None req-a54e9f48-382d-4ba7-a13a-06c1c63a896f None None] MFA Rules not processed for user `fe70a84d6043403a88557bec74566463`. Rule list: `[]` (Enabled: `True`). {{(pid=88113) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 10:00:01.531925 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.fernet_utils [None req-a54e9f48-382d-4ba7-a13a-06c1c63a896f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:00:01.598764 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 1306/2615] 10.222.0.13 () {58 vars in 958 bytes} [Wed Aug 31 10:00:01 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 10:00:01.603269 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61464bbe-2252-4d34-b553-a29de885115e admin admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:00:01.668005 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61464bbe-2252-4d34-b553-a29de885115e admin admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:00:01.672628 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10569e5f-dd40-4058-854e-d47ea9119e3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:00:01.673248 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10569e5f-dd40-4058-854e-d47ea9119e3d admin admin] REQUEST_METHOD: `POST` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:00:01.673553 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10569e5f-dd40-4058-854e-d47ea9119e3d admin admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:00:01.673822 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10569e5f-dd40-4058-854e-d47ea9119e3d admin admin] PATH_INFO: `/v3/projects` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:00:01.674510 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10569e5f-dd40-4058-854e-d47ea9119e3d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:00:01.676566 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10569e5f-dd40-4058-854e-d47ea9119e3d admin admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:00:01.703417 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 1310/2616] 10.222.0.13 () {60 vars in 1153 bytes} [Wed Aug 31 10:00:01 2022] POST /identity/v3/projects => generated 377 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 10:00:01.708119 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a54e9f48-382d-4ba7-a13a-06c1c63a896f None None] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:00:01.719248 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a54e9f48-382d-4ba7-a13a-06c1c63a896f None None] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:00:01.781798 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e46f78-e3dd-4dc7-a09a-ce9e1d9e348c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:00:01.782411 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e46f78-e3dd-4dc7-a09a-ce9e1d9e348c admin admin] REQUEST_METHOD: `POST` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:00:01.782690 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e46f78-e3dd-4dc7-a09a-ce9e1d9e348c admin admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:00:01.782960 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e46f78-e3dd-4dc7-a09a-ce9e1d9e348c admin admin] PATH_INFO: `/v3/users` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:00:01.783603 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14e46f78-e3dd-4dc7-a09a-ce9e1d9e348c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:00:01.786903 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14e46f78-e3dd-4dc7-a09a-ce9e1d9e348c admin admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:00:01.810399 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 1307/2617] 10.222.0.13 () {60 vars in 1144 bytes} [Wed Aug 31 10:00:01 2022] POST /identity/v3/users => generated 289 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 10:00:01.811517 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10569e5f-dd40-4058-854e-d47ea9119e3d admin admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:00:01.822417 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10569e5f-dd40-4058-854e-d47ea9119e3d admin admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:00:01.827043 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc65905-e321-4a41-9101-cd5bc3f99b06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:00:01.827632 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc65905-e321-4a41-9101-cd5bc3f99b06 admin admin] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:00:01.827908 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc65905-e321-4a41-9101-cd5bc3f99b06 admin admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:00:01.828178 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc65905-e321-4a41-9101-cd5bc3f99b06 admin admin] PATH_INFO: `/v3/roles` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:00:01.828817 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bc65905-e321-4a41-9101-cd5bc3f99b06 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:00:01.830430 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bc65905-e321-4a41-9101-cd5bc3f99b06 admin admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:00:01.841924 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 1311/2618] 10.222.0.13 () {58 vars in 1122 bytes} [Wed Aug 31 10:00:01 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 10:00:01.845981 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e46f78-e3dd-4dc7-a09a-ce9e1d9e348c admin admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:00:01.857486 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e46f78-e3dd-4dc7-a09a-ce9e1d9e348c admin admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:00:01.862179 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6344a919-b55c-4f54-942b-ea545898ae33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:00:01.863320 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6344a919-b55c-4f54-942b-ea545898ae33 admin admin] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:00:01.863602 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6344a919-b55c-4f54-942b-ea545898ae33 admin admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:00:01.864168 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6344a919-b55c-4f54-942b-ea545898ae33 admin admin] PATH_INFO: `/v3/roles` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:00:01.864814 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6344a919-b55c-4f54-942b-ea545898ae33 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:00:01.866442 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6344a919-b55c-4f54-942b-ea545898ae33 admin admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:00:01.876400 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 1308/2619] 10.222.0.13 () {58 vars in 1122 bytes} [Wed Aug 31 10:00:01 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 10:00:01.880659 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc65905-e321-4a41-9101-cd5bc3f99b06 admin admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:00:01.891051 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc65905-e321-4a41-9101-cd5bc3f99b06 admin admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:00:01.896189 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63de3f9e-2721-4230-aebe-15fb674d2126 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:00:01.896833 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63de3f9e-2721-4230-aebe-15fb674d2126 admin admin] REQUEST_METHOD: `PUT` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:00:01.897107 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63de3f9e-2721-4230-aebe-15fb674d2126 admin admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:00:01.897375 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63de3f9e-2721-4230-aebe-15fb674d2126 admin admin] PATH_INFO: `/v3/projects/63308b6f3152442ba8e104b945b16c03/users/f4fa5cad11eb40208d50b82492270394/roles/6c99f5d9f1f243eaa134d8540f21095d` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:00:01.912229 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63de3f9e-2721-4230-aebe-15fb674d2126 admin admin] RBAC: Authorizing `identity:create_grant(project_id=63308b6f3152442ba8e104b945b16c03, user_id=f4fa5cad11eb40208d50b82492270394, role_id=6c99f5d9f1f243eaa134d8540f21095d)` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:00:01.914871 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63de3f9e-2721-4230-aebe-15fb674d2126 admin admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:00:01.928178 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 1312/2620] 10.222.0.13 () {60 vars in 1483 bytes} [Wed Aug 31 10:00:01 2022] PUT /identity/v3/projects/63308b6f3152442ba8e104b945b16c03/users/f4fa5cad11eb40208d50b82492270394/roles/6c99f5d9f1f243eaa134d8540f21095d => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 10:00:01.938427 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade69a53-f102-425f-9e06-8ee6f12c6a0d None None] REQUEST_METHOD: `POST` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:00:01.938850 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade69a53-f102-425f-9e06-8ee6f12c6a0d None None] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:00:01.939043 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade69a53-f102-425f-9e06-8ee6f12c6a0d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:00:01.968157 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.auth.core [None req-ade69a53-f102-425f-9e06-8ee6f12c6a0d None None] MFA Rules not processed for user `f4fa5cad11eb40208d50b82492270394`. Rule list: `[]` (Enabled: `True`). {{(pid=88113) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 10:00:01.969491 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.fernet_utils [None req-ade69a53-f102-425f-9e06-8ee6f12c6a0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:00:01.980048 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:00:02.038731 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 1309/2621] 10.222.0.13 () {58 vars in 958 bytes} [Wed Aug 31 10:00:01 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 10:00:02.043194 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63de3f9e-2721-4230-aebe-15fb674d2126 admin admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:00:02.107878 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63de3f9e-2721-4230-aebe-15fb674d2126 admin admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:00:02.112535 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c813961-2902-45f2-b7a9-cd6456fb2f33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:00:02.115145 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c813961-2902-45f2-b7a9-cd6456fb2f33 admin admin] REQUEST_METHOD: `POST` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:00:02.115145 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c813961-2902-45f2-b7a9-cd6456fb2f33 admin admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:00:02.115145 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c813961-2902-45f2-b7a9-cd6456fb2f33 admin admin] PATH_INFO: `/v3/projects` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:00:02.115145 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c813961-2902-45f2-b7a9-cd6456fb2f33 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:00:02.116829 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c813961-2902-45f2-b7a9-cd6456fb2f33 admin admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:00:02.122532 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:00:02.150317 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 1313/2622] 10.222.0.13 () {60 vars in 1153 bytes} [Wed Aug 31 10:00:02 2022] POST /identity/v3/projects => generated 379 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 10:00:02.152439 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade69a53-f102-425f-9e06-8ee6f12c6a0d None None] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:00:02.164164 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade69a53-f102-425f-9e06-8ee6f12c6a0d None None] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:00:02.224110 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d774893-4347-45ad-bb68-bb5d5f7580af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:00:02.224715 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d774893-4347-45ad-bb68-bb5d5f7580af admin admin] REQUEST_METHOD: `POST` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:00:02.224989 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d774893-4347-45ad-bb68-bb5d5f7580af admin admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:00:02.225253 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d774893-4347-45ad-bb68-bb5d5f7580af admin admin] PATH_INFO: `/v3/users` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:00:02.225955 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d774893-4347-45ad-bb68-bb5d5f7580af admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:00:02.227888 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d774893-4347-45ad-bb68-bb5d5f7580af admin admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:00:02.249644 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 1310/2623] 10.222.0.13 () {60 vars in 1144 bytes} [Wed Aug 31 10:00:02 2022] POST /identity/v3/users => generated 296 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 10:00:02.254889 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c813961-2902-45f2-b7a9-cd6456fb2f33 admin admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:00:02.274620 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c813961-2902-45f2-b7a9-cd6456fb2f33 admin admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:00:02.283047 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ead478b-46ab-4c77-8726-ce9614ede8d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:00:02.284105 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ead478b-46ab-4c77-8726-ce9614ede8d6 admin admin] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:00:02.284615 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ead478b-46ab-4c77-8726-ce9614ede8d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:00:02.285112 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ead478b-46ab-4c77-8726-ce9614ede8d6 admin admin] PATH_INFO: `/v3/roles` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:00:02.286360 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ead478b-46ab-4c77-8726-ce9614ede8d6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:00:02.289296 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ead478b-46ab-4c77-8726-ce9614ede8d6 admin admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:00:02.308222 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 1314/2624] 10.222.0.13 () {58 vars in 1122 bytes} [Wed Aug 31 10:00:02 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 10:00:02.312437 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d774893-4347-45ad-bb68-bb5d5f7580af admin admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:00:02.323702 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d774893-4347-45ad-bb68-bb5d5f7580af admin admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:00:02.328377 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a5d4df-a7b9-48c8-9439-6c5c678cd747 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:00:02.329009 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a5d4df-a7b9-48c8-9439-6c5c678cd747 admin admin] REQUEST_METHOD: `PUT` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:00:02.329285 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a5d4df-a7b9-48c8-9439-6c5c678cd747 admin admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:00:02.329588 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a5d4df-a7b9-48c8-9439-6c5c678cd747 admin admin] PATH_INFO: `/v3/projects/117b90bbfe314c6ab5cdf63b66d45914/users/d5e6cee747ad4dfe9a4df1850fb44285/roles/5d4ee63a3add47fc9a6b8b16d39e0a1f` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:00:02.346405 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2a5d4df-a7b9-48c8-9439-6c5c678cd747 admin admin] RBAC: Authorizing `identity:create_grant(project_id=117b90bbfe314c6ab5cdf63b66d45914, user_id=d5e6cee747ad4dfe9a4df1850fb44285, role_id=5d4ee63a3add47fc9a6b8b16d39e0a1f)` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:00:02.349865 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2a5d4df-a7b9-48c8-9439-6c5c678cd747 admin admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:00:02.364800 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 1311/2625] 10.222.0.13 () {60 vars in 1483 bytes} [Wed Aug 31 10:00:02 2022] PUT /identity/v3/projects/117b90bbfe314c6ab5cdf63b66d45914/users/d5e6cee747ad4dfe9a4df1850fb44285/roles/5d4ee63a3add47fc9a6b8b16d39e0a1f => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 10:00:02.369175 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ead478b-46ab-4c77-8726-ce9614ede8d6 admin admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:00:02.380213 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ead478b-46ab-4c77-8726-ce9614ede8d6 admin admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:00:02.442723 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d1d0f04-cf1f-4535-84e0-ec2cd5b016a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:00:02.443329 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1d0f04-cf1f-4535-84e0-ec2cd5b016a2 admin admin] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:00:02.443608 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1d0f04-cf1f-4535-84e0-ec2cd5b016a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:00:02.443930 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1d0f04-cf1f-4535-84e0-ec2cd5b016a2 admin admin] PATH_INFO: `/v3/roles` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:00:02.444522 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d1d0f04-cf1f-4535-84e0-ec2cd5b016a2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:00:02.446145 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d1d0f04-cf1f-4535-84e0-ec2cd5b016a2 admin admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:00:02.456072 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 1315/2626] 10.222.0.13 () {58 vars in 1122 bytes} [Wed Aug 31 10:00:02 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 10:00:02.460520 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a5d4df-a7b9-48c8-9439-6c5c678cd747 admin admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:00:02.470935 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a5d4df-a7b9-48c8-9439-6c5c678cd747 admin admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:00:02.475976 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c1f30e2-bcb3-4977-8d79-e9185c2e437a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:00:02.476625 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1f30e2-bcb3-4977-8d79-e9185c2e437a admin admin] REQUEST_METHOD: `PUT` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:00:02.476904 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1f30e2-bcb3-4977-8d79-e9185c2e437a admin admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:00:02.477171 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1f30e2-bcb3-4977-8d79-e9185c2e437a admin admin] PATH_INFO: `/v3/projects/117b90bbfe314c6ab5cdf63b66d45914/users/d5e6cee747ad4dfe9a4df1850fb44285/roles/5d4ee63a3add47fc9a6b8b16d39e0a1f` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:00:02.479134 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c1f30e2-bcb3-4977-8d79-e9185c2e437a admin admin] RBAC: Authorizing `identity:create_grant(project_id=117b90bbfe314c6ab5cdf63b66d45914, user_id=d5e6cee747ad4dfe9a4df1850fb44285, role_id=5d4ee63a3add47fc9a6b8b16d39e0a1f)` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:00:02.481894 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c1f30e2-bcb3-4977-8d79-e9185c2e437a admin admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:00:02.493455 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 1312/2627] 10.222.0.13 () {60 vars in 1483 bytes} [Wed Aug 31 10:00:02 2022] PUT /identity/v3/projects/117b90bbfe314c6ab5cdf63b66d45914/users/d5e6cee747ad4dfe9a4df1850fb44285/roles/5d4ee63a3add47fc9a6b8b16d39e0a1f => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 10:00:02.504422 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cebee67b-ffc7-4b30-b238-3059634e6f7f None None] REQUEST_METHOD: `POST` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:00:02.504708 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cebee67b-ffc7-4b30-b238-3059634e6f7f None None] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:00:02.504985 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cebee67b-ffc7-4b30-b238-3059634e6f7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:00:02.658095 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.auth.core [None req-cebee67b-ffc7-4b30-b238-3059634e6f7f None None] MFA Rules not processed for user `d5e6cee747ad4dfe9a4df1850fb44285`. Rule list: `[]` (Enabled: `True`). {{(pid=88114) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 10:00:02.659334 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.fernet_utils [None req-cebee67b-ffc7-4b30-b238-3059634e6f7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88114) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:00:02.725763 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 1316/2628] 10.222.0.13 () {58 vars in 958 bytes} [Wed Aug 31 10:00:02 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 223 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 10:00:02.736762 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c1f30e2-bcb3-4977-8d79-e9185c2e437a admin admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:00:02.739434 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.fernet_utils [None req-2c1f30e2-bcb3-4977-8d79-e9185c2e437a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:00:02.797567 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c1f30e2-bcb3-4977-8d79-e9185c2e437a admin admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:00:02.801098 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73723539-344d-482c-9d4f-0d812d0a122d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d52b69c8e94441dacfcff944326f169', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '49548eb44d62415582af820ab66decee', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:00:02.801733 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73723539-344d-482c-9d4f-0d812d0a122d service cinder] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:00:02.802007 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73723539-344d-482c-9d4f-0d812d0a122d service cinder] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:00:02.802273 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73723539-344d-482c-9d4f-0d812d0a122d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:00:02.811234 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73723539-344d-482c-9d4f-0d812d0a122d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:00:02.812657 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73723539-344d-482c-9d4f-0d812d0a122d service cinder] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:00:02.822803 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 1313/2629] 10.222.0.13 () {60 vars in 1365 bytes} [Wed Aug 31 10:00:02 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 10:00:02.866282 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cebee67b-ffc7-4b30-b238-3059634e6f7f None None] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:00:02.867936 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.fernet_utils [None req-cebee67b-ffc7-4b30-b238-3059634e6f7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88114) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:00:02.880852 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cebee67b-ffc7-4b30-b238-3059634e6f7f None None] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:00:02.884415 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34331ce8-a744-4c45-9769-96db045b9a34 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d52b69c8e94441dacfcff944326f169', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '49548eb44d62415582af820ab66decee', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:00:02.885006 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34331ce8-a744-4c45-9769-96db045b9a34 service cinder] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:00:02.885277 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34331ce8-a744-4c45-9769-96db045b9a34 service cinder] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:00:02.885571 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34331ce8-a744-4c45-9769-96db045b9a34 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:00:02.939967 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:00:02.943105 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34331ce8-a744-4c45-9769-96db045b9a34 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:00:02.944909 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34331ce8-a744-4c45-9769-96db045b9a34 service cinder] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:00:02.954810 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 1317/2630] 10.222.0.13 () {60 vars in 1365 bytes} [Wed Aug 31 10:00:02 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 10:00:05.851287 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73723539-344d-482c-9d4f-0d812d0a122d service cinder] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:00:05.860221 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:00:05.916979 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73723539-344d-482c-9d4f-0d812d0a122d service cinder] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:00:05.921585 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d5a03ba-c105-42bf-ac7a-4259b0ee6ccf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:00:05.922813 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5a03ba-c105-42bf-ac7a-4259b0ee6ccf admin admin] REQUEST_METHOD: `DELETE` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:00:05.922813 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5a03ba-c105-42bf-ac7a-4259b0ee6ccf admin admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:00:05.922813 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5a03ba-c105-42bf-ac7a-4259b0ee6ccf admin admin] PATH_INFO: `/v3/users/fe70a84d6043403a88557bec74566463` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:00:05.923907 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d5a03ba-c105-42bf-ac7a-4259b0ee6ccf admin admin] RBAC: Authorizing `identity:delete_user(user_id=fe70a84d6043403a88557bec74566463)` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:00:05.926464 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d5a03ba-c105-42bf-ac7a-4259b0ee6ccf admin admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:00:05.977616 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.notifications [None req-9d5a03ba-c105-42bf-ac7a-4259b0ee6ccf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fe70a84d6043403a88557bec74566463'} {{(pid=88113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 10:00:05.990238 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.notifications [None req-9d5a03ba-c105-42bf-ac7a-4259b0ee6ccf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fe70a84d6043403a88557bec74566463'} {{(pid=88113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 10:00:06.006152 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.notifications [None req-9d5a03ba-c105-42bf-ac7a-4259b0ee6ccf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fe70a84d6043403a88557bec74566463'} {{(pid=88113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 10:00:06.006894 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.notifications [None req-9d5a03ba-c105-42bf-ac7a-4259b0ee6ccf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fe70a84d6043403a88557bec74566463'} {{(pid=88113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 10:00:06.014545 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.notifications [None req-9d5a03ba-c105-42bf-ac7a-4259b0ee6ccf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fe70a84d6043403a88557bec74566463'} {{(pid=88113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 10:00:06.022502 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 1314/2631] 10.222.0.13 () {58 vars in 1224 bytes} [Wed Aug 31 10:00:05 2022] DELETE /identity/v3/users/fe70a84d6043403a88557bec74566463 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 10:00:06.026973 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05075f5f-3dd2-4cbb-b947-e750a5792bb1 None None] REQUEST_METHOD: `POST` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:00:06.027440 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05075f5f-3dd2-4cbb-b947-e750a5792bb1 None None] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:00:06.027571 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05075f5f-3dd2-4cbb-b947-e750a5792bb1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:00:06.046775 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.auth.core [None req-05075f5f-3dd2-4cbb-b947-e750a5792bb1 None None] MFA Rules not processed for user `fd836cf590a145df9acf19cf3ae2ac56`. Rule list: `[]` (Enabled: `True`). {{(pid=88114) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 10:00:06.047969 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.fernet_utils [None req-05075f5f-3dd2-4cbb-b947-e750a5792bb1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88114) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:00:06.109021 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 1318/2632] 10.222.0.13 () {58 vars in 958 bytes} [Wed Aug 31 10:00:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 10:00:06.120239 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d5a03ba-c105-42bf-ac7a-4259b0ee6ccf admin admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:00:06.121734 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.fernet_utils [None req-9d5a03ba-c105-42bf-ac7a-4259b0ee6ccf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:00:06.181590 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d5a03ba-c105-42bf-ac7a-4259b0ee6ccf admin admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:00:06.185241 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fbbfc6f-c530-4f78-ba01-43145ab7afad service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9bdaacd597442c2bcde2c555f2cc140', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '49548eb44d62415582af820ab66decee', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:00:06.185876 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fbbfc6f-c530-4f78-ba01-43145ab7afad service neutron] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:00:06.186155 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fbbfc6f-c530-4f78-ba01-43145ab7afad service neutron] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:00:06.186426 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fbbfc6f-c530-4f78-ba01-43145ab7afad service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:00:06.196150 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fbbfc6f-c530-4f78-ba01-43145ab7afad service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:00:06.197624 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fbbfc6f-c530-4f78-ba01-43145ab7afad service neutron] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:00:06.208123 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 1315/2633] 10.222.0.13 () {60 vars in 1365 bytes} [Wed Aug 31 10:00:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 10:00:06.482601 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05075f5f-3dd2-4cbb-b947-e750a5792bb1 None None] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:00:06.484101 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.fernet_utils [None req-05075f5f-3dd2-4cbb-b947-e750a5792bb1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88114) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:00:06.499768 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05075f5f-3dd2-4cbb-b947-e750a5792bb1 None None] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:00:06.505739 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d9b6073-3304-4f47-87b0-053f317b83bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:00:06.506457 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9b6073-3304-4f47-87b0-053f317b83bd admin admin] REQUEST_METHOD: `DELETE` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:00:06.506842 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9b6073-3304-4f47-87b0-053f317b83bd admin admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:00:06.507215 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9b6073-3304-4f47-87b0-053f317b83bd admin admin] PATH_INFO: `/v3/projects/fcb0af9875ed40fc83260ff500d0bc4b` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:00:06.508623 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d9b6073-3304-4f47-87b0-053f317b83bd admin admin] RBAC: Authorizing `identity:delete_project(project_id=fcb0af9875ed40fc83260ff500d0bc4b)` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:00:06.510799 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d9b6073-3304-4f47-87b0-053f317b83bd admin admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:00:06.556502 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.notifications [None req-1d9b6073-3304-4f47-87b0-053f317b83bd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fcb0af9875ed40fc83260ff500d0bc4b', 'request_id': 'req-1d9b6073-3304-4f47-87b0-053f317b83bd'} {{(pid=88114) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 10:00:06.562461 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.notifications [None req-1d9b6073-3304-4f47-87b0-053f317b83bd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fcb0af9875ed40fc83260ff500d0bc4b', 'request_id': 'req-1d9b6073-3304-4f47-87b0-053f317b83bd'} {{(pid=88114) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 10:00:06.569517 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.notifications [None req-1d9b6073-3304-4f47-87b0-053f317b83bd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fcb0af9875ed40fc83260ff500d0bc4b', 'request_id': 'req-1d9b6073-3304-4f47-87b0-053f317b83bd'} {{(pid=88114) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 10:00:06.583176 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.notifications [None req-1d9b6073-3304-4f47-87b0-053f317b83bd admin admin] The token cache is being invalidate because project fcb0af9875ed40fc83260ff500d0bc4b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88114) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 10:00:06.583468 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.notifications [None req-1d9b6073-3304-4f47-87b0-053f317b83bd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88114) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 10:00:06.584097 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.notifications [None req-1d9b6073-3304-4f47-87b0-053f317b83bd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88114) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 10:00:06.585629 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 1319/2634] 10.222.0.13 () {58 vars in 1233 bytes} [Wed Aug 31 10:00:06 2022] DELETE /identity/v3/projects/fcb0af9875ed40fc83260ff500d0bc4b => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 10:00:06.591428 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fbbfc6f-c530-4f78-ba01-43145ab7afad service neutron] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:00:06.601585 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fbbfc6f-c530-4f78-ba01-43145ab7afad service neutron] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:00:06.629133 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:00:06.663860 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4320e19-83f7-42b7-a667-a6152a231b21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:00:06.664509 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4320e19-83f7-42b7-a667-a6152a231b21 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:00:06.664790 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4320e19-83f7-42b7-a667-a6152a231b21 admin admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:00:06.665065 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4320e19-83f7-42b7-a667-a6152a231b21 admin admin] PATH_INFO: `/v3/users/f4fa5cad11eb40208d50b82492270394` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:00:06.666234 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4320e19-83f7-42b7-a667-a6152a231b21 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f4fa5cad11eb40208d50b82492270394)` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:00:06.668173 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4320e19-83f7-42b7-a667-a6152a231b21 admin admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:00:06.716421 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.notifications [None req-f4320e19-83f7-42b7-a667-a6152a231b21 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f4fa5cad11eb40208d50b82492270394'} {{(pid=88113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 10:00:06.723345 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.notifications [None req-f4320e19-83f7-42b7-a667-a6152a231b21 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f4fa5cad11eb40208d50b82492270394'} {{(pid=88113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 10:00:06.735977 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.notifications [None req-f4320e19-83f7-42b7-a667-a6152a231b21 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f4fa5cad11eb40208d50b82492270394'} {{(pid=88113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 10:00:06.736656 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.notifications [None req-f4320e19-83f7-42b7-a667-a6152a231b21 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f4fa5cad11eb40208d50b82492270394'} {{(pid=88113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 10:00:06.744397 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.notifications [None req-f4320e19-83f7-42b7-a667-a6152a231b21 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f4fa5cad11eb40208d50b82492270394'} {{(pid=88113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 10:00:06.750247 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 1316/2635] 10.222.0.13 () {58 vars in 1224 bytes} [Wed Aug 31 10:00:06 2022] DELETE /identity/v3/users/f4fa5cad11eb40208d50b82492270394 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 10:00:07.031625 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d9b6073-3304-4f47-87b0-053f317b83bd admin admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:00:07.033190 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.fernet_utils [None req-1d9b6073-3304-4f47-87b0-053f317b83bd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88114) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:00:07.049316 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d9b6073-3304-4f47-87b0-053f317b83bd admin admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:00:07.114100 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b9d41d6-dc3c-4f72-afce-86a41255b422 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:00:07.114635 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9d41d6-dc3c-4f72-afce-86a41255b422 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:00:07.114916 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9d41d6-dc3c-4f72-afce-86a41255b422 admin admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:00:07.115191 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9d41d6-dc3c-4f72-afce-86a41255b422 admin admin] PATH_INFO: `/v3/projects/63308b6f3152442ba8e104b945b16c03` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:00:07.116377 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b9d41d6-dc3c-4f72-afce-86a41255b422 admin admin] RBAC: Authorizing `identity:delete_project(project_id=63308b6f3152442ba8e104b945b16c03)` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:00:07.118315 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b9d41d6-dc3c-4f72-afce-86a41255b422 admin admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:00:07.142295 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:00:07.168117 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.notifications [None req-4b9d41d6-dc3c-4f72-afce-86a41255b422 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '63308b6f3152442ba8e104b945b16c03', 'request_id': 'req-4b9d41d6-dc3c-4f72-afce-86a41255b422'} {{(pid=88114) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 10:00:07.174561 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.notifications [None req-4b9d41d6-dc3c-4f72-afce-86a41255b422 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '63308b6f3152442ba8e104b945b16c03', 'request_id': 'req-4b9d41d6-dc3c-4f72-afce-86a41255b422'} {{(pid=88114) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 10:00:07.181138 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.notifications [None req-4b9d41d6-dc3c-4f72-afce-86a41255b422 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '63308b6f3152442ba8e104b945b16c03', 'request_id': 'req-4b9d41d6-dc3c-4f72-afce-86a41255b422'} {{(pid=88114) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 10:00:07.194085 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.notifications [None req-4b9d41d6-dc3c-4f72-afce-86a41255b422 admin admin] The token cache is being invalidate because project 63308b6f3152442ba8e104b945b16c03 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88114) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 10:00:07.194377 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.notifications [None req-4b9d41d6-dc3c-4f72-afce-86a41255b422 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88114) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 10:00:07.195000 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.notifications [None req-4b9d41d6-dc3c-4f72-afce-86a41255b422 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88114) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 10:00:07.196510 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 1320/2636] 10.222.0.13 () {58 vars in 1233 bytes} [Wed Aug 31 10:00:07 2022] DELETE /identity/v3/projects/63308b6f3152442ba8e104b945b16c03 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 10:00:07.200514 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4320e19-83f7-42b7-a667-a6152a231b21 admin admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:00:07.202051 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.fernet_utils [None req-f4320e19-83f7-42b7-a667-a6152a231b21 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:00:07.215867 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4320e19-83f7-42b7-a667-a6152a231b21 admin admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:00:07.277559 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8272da1-1f94-4661-92dd-d82d4b405632 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:00:07.278012 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8272da1-1f94-4661-92dd-d82d4b405632 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:00:07.278344 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8272da1-1f94-4661-92dd-d82d4b405632 admin admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:00:07.278549 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8272da1-1f94-4661-92dd-d82d4b405632 admin admin] PATH_INFO: `/v3/users/d5e6cee747ad4dfe9a4df1850fb44285` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:00:07.279667 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8272da1-1f94-4661-92dd-d82d4b405632 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d5e6cee747ad4dfe9a4df1850fb44285)` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:00:07.281645 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8272da1-1f94-4661-92dd-d82d4b405632 admin admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:00:07.330238 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.notifications [None req-b8272da1-1f94-4661-92dd-d82d4b405632 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd5e6cee747ad4dfe9a4df1850fb44285'} {{(pid=88113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 10:00:07.332525 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=82,keystone:DELETE=17,keystone:INSERT=1 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:00:07.338744 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.notifications [None req-b8272da1-1f94-4661-92dd-d82d4b405632 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd5e6cee747ad4dfe9a4df1850fb44285'} {{(pid=88113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 10:00:07.352087 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.notifications [None req-b8272da1-1f94-4661-92dd-d82d4b405632 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd5e6cee747ad4dfe9a4df1850fb44285'} {{(pid=88113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 10:00:07.352780 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.notifications [None req-b8272da1-1f94-4661-92dd-d82d4b405632 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd5e6cee747ad4dfe9a4df1850fb44285'} {{(pid=88113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 10:00:07.360694 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.notifications [None req-b8272da1-1f94-4661-92dd-d82d4b405632 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd5e6cee747ad4dfe9a4df1850fb44285'} {{(pid=88113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 10:00:07.368784 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 1317/2637] 10.222.0.13 () {58 vars in 1224 bytes} [Wed Aug 31 10:00:07 2022] DELETE /identity/v3/users/d5e6cee747ad4dfe9a4df1850fb44285 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 10:00:07.643016 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b9d41d6-dc3c-4f72-afce-86a41255b422 admin admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:00:07.645572 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.fernet_utils [None req-4b9d41d6-dc3c-4f72-afce-86a41255b422 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88114) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:00:07.658214 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b9d41d6-dc3c-4f72-afce-86a41255b422 admin admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:00:07.724826 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d73ae8eb-47bb-43a6-a6ab-12f8a71cc33d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:00:07.725497 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73ae8eb-47bb-43a6-a6ab-12f8a71cc33d admin admin] REQUEST_METHOD: `DELETE` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:00:07.725786 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73ae8eb-47bb-43a6-a6ab-12f8a71cc33d admin admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:00:07.726045 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73ae8eb-47bb-43a6-a6ab-12f8a71cc33d admin admin] PATH_INFO: `/v3/projects/117b90bbfe314c6ab5cdf63b66d45914` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:00:07.727165 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d73ae8eb-47bb-43a6-a6ab-12f8a71cc33d admin admin] RBAC: Authorizing `identity:delete_project(project_id=117b90bbfe314c6ab5cdf63b66d45914)` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:00:07.729084 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d73ae8eb-47bb-43a6-a6ab-12f8a71cc33d admin admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:00:07.775039 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.notifications [None req-d73ae8eb-47bb-43a6-a6ab-12f8a71cc33d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '117b90bbfe314c6ab5cdf63b66d45914', 'request_id': 'req-d73ae8eb-47bb-43a6-a6ab-12f8a71cc33d'} {{(pid=88114) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 10:00:07.781150 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.notifications [None req-d73ae8eb-47bb-43a6-a6ab-12f8a71cc33d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '117b90bbfe314c6ab5cdf63b66d45914', 'request_id': 'req-d73ae8eb-47bb-43a6-a6ab-12f8a71cc33d'} {{(pid=88114) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 10:00:07.787798 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.notifications [None req-d73ae8eb-47bb-43a6-a6ab-12f8a71cc33d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '117b90bbfe314c6ab5cdf63b66d45914', 'request_id': 'req-d73ae8eb-47bb-43a6-a6ab-12f8a71cc33d'} {{(pid=88114) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 10:00:07.799984 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.notifications [None req-d73ae8eb-47bb-43a6-a6ab-12f8a71cc33d admin admin] The token cache is being invalidate because project 117b90bbfe314c6ab5cdf63b66d45914 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88114) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 10:00:07.800268 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.notifications [None req-d73ae8eb-47bb-43a6-a6ab-12f8a71cc33d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88114) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 10:00:07.801080 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.notifications [None req-d73ae8eb-47bb-43a6-a6ab-12f8a71cc33d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88114) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 10:00:07.802568 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 1321/2638] 10.222.0.13 () {58 vars in 1233 bytes} [Wed Aug 31 10:00:07 2022] DELETE /identity/v3/projects/117b90bbfe314c6ab5cdf63b66d45914 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 10:00:07.808566 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b086e37-4a62-4d67-ad38-30d82e484027 None None] REQUEST_METHOD: `POST` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:00:07.808850 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b086e37-4a62-4d67-ad38-30d82e484027 None None] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:00:07.809124 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b086e37-4a62-4d67-ad38-30d82e484027 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:00:07.828852 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.auth.core [None req-8b086e37-4a62-4d67-ad38-30d82e484027 None None] MFA Rules not processed for user `fd836cf590a145df9acf19cf3ae2ac56`. Rule list: `[]` (Enabled: `True`). {{(pid=88113) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 10:00:07.830075 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.fernet_utils [None req-8b086e37-4a62-4d67-ad38-30d82e484027 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:00:07.890245 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 1318/2639] 10.222.0.13 () {58 vars in 958 bytes} [Wed Aug 31 10:00:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 10:00:07.900568 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61173b5-4794-43b0-adda-f25fc6ac5c19 None None] REQUEST_METHOD: `POST` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:00:07.900855 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61173b5-4794-43b0-adda-f25fc6ac5c19 None None] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:00:07.901120 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61173b5-4794-43b0-adda-f25fc6ac5c19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:00:07.922349 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.auth.core [None req-b61173b5-4794-43b0-adda-f25fc6ac5c19 None None] MFA Rules not processed for user `fd836cf590a145df9acf19cf3ae2ac56`. Rule list: `[]` (Enabled: `True`). {{(pid=88114) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 10:00:07.923560 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.fernet_utils [None req-b61173b5-4794-43b0-adda-f25fc6ac5c19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88114) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:00:07.931910 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 1322/2640] 10.222.0.13 () {58 vars in 958 bytes} [Wed Aug 31 10:00:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 10:00:07.936318 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b086e37-4a62-4d67-ad38-30d82e484027 None None] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:00:07.947335 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b086e37-4a62-4d67-ad38-30d82e484027 None None] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:00:07.952075 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2793eaf-7105-4ef6-8732-cac783f7675a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:00:07.952676 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2793eaf-7105-4ef6-8732-cac783f7675a admin admin] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:00:07.952955 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2793eaf-7105-4ef6-8732-cac783f7675a admin admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:00:07.953223 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2793eaf-7105-4ef6-8732-cac783f7675a admin admin] PATH_INFO: `/v3/domains` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:00:07.953909 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2793eaf-7105-4ef6-8732-cac783f7675a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88113) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:00:07.954211 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2793eaf-7105-4ef6-8732-cac783f7675a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:00:07.955859 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2793eaf-7105-4ef6-8732-cac783f7675a admin admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:00:07.968611 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 1319/2641] 10.222.0.13 () {58 vars in 1153 bytes} [Wed Aug 31 10:00:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:00:07.973732 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b61173b5-4794-43b0-adda-f25fc6ac5c19 None None] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:00:07.982875 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b61173b5-4794-43b0-adda-f25fc6ac5c19 None None] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:00:07.987535 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef749e5a-16a0-4c13-b1cb-9c43384a284c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:00:07.988122 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef749e5a-16a0-4c13-b1cb-9c43384a284c admin admin] REQUEST_METHOD: `POST` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:00:07.988388 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef749e5a-16a0-4c13-b1cb-9c43384a284c admin admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:00:07.988650 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef749e5a-16a0-4c13-b1cb-9c43384a284c admin admin] PATH_INFO: `/v3/projects` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:00:07.989572 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef749e5a-16a0-4c13-b1cb-9c43384a284c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:00:07.991514 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef749e5a-16a0-4c13-b1cb-9c43384a284c admin admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:00:08.018144 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 1323/2642] 10.222.0.13 () {60 vars in 1153 bytes} [Wed Aug 31 10:00:07 2022] POST /identity/v3/projects => generated 381 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 10:00:08.023074 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2793eaf-7105-4ef6-8732-cac783f7675a admin admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:00:08.033176 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2793eaf-7105-4ef6-8732-cac783f7675a admin admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:00:08.091632 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e72a23d5-81fa-40ac-add0-f808c9dfb695 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:00:08.092239 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72a23d5-81fa-40ac-add0-f808c9dfb695 admin admin] REQUEST_METHOD: `POST` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:00:08.092520 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72a23d5-81fa-40ac-add0-f808c9dfb695 admin admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:00:08.092791 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72a23d5-81fa-40ac-add0-f808c9dfb695 admin admin] PATH_INFO: `/v3/users` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:00:08.093469 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e72a23d5-81fa-40ac-add0-f808c9dfb695 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:00:08.095427 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e72a23d5-81fa-40ac-add0-f808c9dfb695 admin admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:00:08.120609 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 1320/2643] 10.222.0.13 () {60 vars in 1144 bytes} [Wed Aug 31 10:00:08 2022] POST /identity/v3/users => generated 291 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 10:00:08.121877 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef749e5a-16a0-4c13-b1cb-9c43384a284c admin admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:00:08.133598 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef749e5a-16a0-4c13-b1cb-9c43384a284c admin admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:00:08.138498 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec8f7d1-e359-4fcd-bcb2-cc593f259e22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:00:08.139100 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec8f7d1-e359-4fcd-bcb2-cc593f259e22 admin admin] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:00:08.139377 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec8f7d1-e359-4fcd-bcb2-cc593f259e22 admin admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:00:08.139647 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec8f7d1-e359-4fcd-bcb2-cc593f259e22 admin admin] PATH_INFO: `/v3/roles` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:00:08.140341 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ec8f7d1-e359-4fcd-bcb2-cc593f259e22 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:00:08.141904 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ec8f7d1-e359-4fcd-bcb2-cc593f259e22 admin admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:00:08.148683 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:DELETE=14,keystone:SELECT=83,keystone:INSERT=3 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:00:08.157685 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 1324/2644] 10.222.0.13 () {58 vars in 1122 bytes} [Wed Aug 31 10:00:08 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 10:00:08.159233 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e72a23d5-81fa-40ac-add0-f808c9dfb695 admin admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:00:08.170069 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e72a23d5-81fa-40ac-add0-f808c9dfb695 admin admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:00:08.176359 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d216f8-828c-4a52-9747-5fe4bf11d8d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:00:08.176990 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d216f8-828c-4a52-9747-5fe4bf11d8d7 admin admin] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:00:08.177360 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d216f8-828c-4a52-9747-5fe4bf11d8d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:00:08.177589 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d216f8-828c-4a52-9747-5fe4bf11d8d7 admin admin] PATH_INFO: `/v3/roles` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:00:08.178304 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82d216f8-828c-4a52-9747-5fe4bf11d8d7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:00:08.179900 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82d216f8-828c-4a52-9747-5fe4bf11d8d7 admin admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:00:08.197719 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 1321/2645] 10.222.0.13 () {58 vars in 1122 bytes} [Wed Aug 31 10:00:08 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 10:00:08.199784 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec8f7d1-e359-4fcd-bcb2-cc593f259e22 admin admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:00:08.211260 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec8f7d1-e359-4fcd-bcb2-cc593f259e22 admin admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:00:08.216522 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adaeaa4c-0fd9-4ab9-93d1-d03832b834aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:00:08.217277 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adaeaa4c-0fd9-4ab9-93d1-d03832b834aa admin admin] REQUEST_METHOD: `PUT` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:00:08.217932 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adaeaa4c-0fd9-4ab9-93d1-d03832b834aa admin admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:00:08.217932 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adaeaa4c-0fd9-4ab9-93d1-d03832b834aa admin admin] PATH_INFO: `/v3/projects/dab9f0edb74f4f58b73e5b95f29d0492/users/034791535bad4c65b9a168f2d0a65659/roles/6c99f5d9f1f243eaa134d8540f21095d` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:00:08.234649 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adaeaa4c-0fd9-4ab9-93d1-d03832b834aa admin admin] RBAC: Authorizing `identity:create_grant(project_id=dab9f0edb74f4f58b73e5b95f29d0492, user_id=034791535bad4c65b9a168f2d0a65659, role_id=6c99f5d9f1f243eaa134d8540f21095d)` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:00:08.237331 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adaeaa4c-0fd9-4ab9-93d1-d03832b834aa admin admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:00:08.251938 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 1325/2646] 10.222.0.13 () {60 vars in 1483 bytes} [Wed Aug 31 10:00:08 2022] PUT /identity/v3/projects/dab9f0edb74f4f58b73e5b95f29d0492/users/034791535bad4c65b9a168f2d0a65659/roles/6c99f5d9f1f243eaa134d8540f21095d => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 10:00:08.264136 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c93884-4fc2-4120-a6b5-afc23b2f7b4d None None] REQUEST_METHOD: `POST` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:00:08.264678 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c93884-4fc2-4120-a6b5-afc23b2f7b4d None None] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:00:08.264678 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c93884-4fc2-4120-a6b5-afc23b2f7b4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:00:08.297661 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.auth.core [None req-c8c93884-4fc2-4120-a6b5-afc23b2f7b4d None None] MFA Rules not processed for user `034791535bad4c65b9a168f2d0a65659`. Rule list: `[]` (Enabled: `True`). {{(pid=88113) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 10:00:08.299115 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.fernet_utils [None req-c8c93884-4fc2-4120-a6b5-afc23b2f7b4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:00:08.324855 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=93,keystone:DELETE=3,keystone:INSERT=4 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:00:08.376529 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 1322/2647] 10.222.0.13 () {58 vars in 958 bytes} [Wed Aug 31 10:00:08 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 10:00:08.381145 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adaeaa4c-0fd9-4ab9-93d1-d03832b834aa admin admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:00:08.448960 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adaeaa4c-0fd9-4ab9-93d1-d03832b834aa admin admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:00:08.453822 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c40aa7-ebc9-4c18-980a-70a17500693d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:00:08.454450 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c40aa7-ebc9-4c18-980a-70a17500693d admin admin] REQUEST_METHOD: `POST` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:00:08.454730 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c40aa7-ebc9-4c18-980a-70a17500693d admin admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:00:08.455025 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c40aa7-ebc9-4c18-980a-70a17500693d admin admin] PATH_INFO: `/v3/projects` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:00:08.455651 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27c40aa7-ebc9-4c18-980a-70a17500693d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:00:08.457896 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27c40aa7-ebc9-4c18-980a-70a17500693d admin admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:00:08.497443 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 1326/2648] 10.222.0.13 () {60 vars in 1153 bytes} [Wed Aug 31 10:00:08 2022] POST /identity/v3/projects => generated 383 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 10:00:08.502151 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c93884-4fc2-4120-a6b5-afc23b2f7b4d None None] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:00:08.520836 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c93884-4fc2-4120-a6b5-afc23b2f7b4d None None] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:00:08.595345 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c354acd0-82af-4a25-867a-5fcc601c5446 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:00:08.595977 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c354acd0-82af-4a25-867a-5fcc601c5446 admin admin] REQUEST_METHOD: `POST` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:00:08.596260 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c354acd0-82af-4a25-867a-5fcc601c5446 admin admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:00:08.596535 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c354acd0-82af-4a25-867a-5fcc601c5446 admin admin] PATH_INFO: `/v3/users` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:00:08.597840 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c354acd0-82af-4a25-867a-5fcc601c5446 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:00:08.599820 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c354acd0-82af-4a25-867a-5fcc601c5446 admin admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:00:08.627668 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 1323/2649] 10.222.0.13 () {60 vars in 1144 bytes} [Wed Aug 31 10:00:08 2022] POST /identity/v3/users => generated 298 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 10:00:08.629049 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c40aa7-ebc9-4c18-980a-70a17500693d admin admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:00:08.641477 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c40aa7-ebc9-4c18-980a-70a17500693d admin admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:00:08.646965 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b3cfa6e-7b59-412a-8f5a-7f5d8a1fbcd2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:00:08.647565 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3cfa6e-7b59-412a-8f5a-7f5d8a1fbcd2 admin admin] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:00:08.647843 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3cfa6e-7b59-412a-8f5a-7f5d8a1fbcd2 admin admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:00:08.648111 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3cfa6e-7b59-412a-8f5a-7f5d8a1fbcd2 admin admin] PATH_INFO: `/v3/roles` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:00:08.648757 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b3cfa6e-7b59-412a-8f5a-7f5d8a1fbcd2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:00:08.650372 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b3cfa6e-7b59-412a-8f5a-7f5d8a1fbcd2 admin admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:00:08.662134 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 1327/2650] 10.222.0.13 () {58 vars in 1122 bytes} [Wed Aug 31 10:00:08 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 10:00:08.667210 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c354acd0-82af-4a25-867a-5fcc601c5446 admin admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:00:08.679358 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c354acd0-82af-4a25-867a-5fcc601c5446 admin admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:00:08.685062 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1011862-08f9-48f6-8c0a-391f131baa1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:00:08.685890 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1011862-08f9-48f6-8c0a-391f131baa1c admin admin] REQUEST_METHOD: `PUT` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:00:08.686364 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1011862-08f9-48f6-8c0a-391f131baa1c admin admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:00:08.686757 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1011862-08f9-48f6-8c0a-391f131baa1c admin admin] PATH_INFO: `/v3/projects/82742be415734d858f0d43c33e1907e6/users/c4a2d164bdad4e83a298453ce59d3712/roles/5d4ee63a3add47fc9a6b8b16d39e0a1f` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:00:08.705881 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1011862-08f9-48f6-8c0a-391f131baa1c admin admin] RBAC: Authorizing `identity:create_grant(project_id=82742be415734d858f0d43c33e1907e6, user_id=c4a2d164bdad4e83a298453ce59d3712, role_id=5d4ee63a3add47fc9a6b8b16d39e0a1f)` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:00:08.710025 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1011862-08f9-48f6-8c0a-391f131baa1c admin admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:00:08.726131 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 1324/2651] 10.222.0.13 () {60 vars in 1483 bytes} [Wed Aug 31 10:00:08 2022] PUT /identity/v3/projects/82742be415734d858f0d43c33e1907e6/users/c4a2d164bdad4e83a298453ce59d3712/roles/5d4ee63a3add47fc9a6b8b16d39e0a1f => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 10:00:08.730222 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b3cfa6e-7b59-412a-8f5a-7f5d8a1fbcd2 admin admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:00:08.747868 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b3cfa6e-7b59-412a-8f5a-7f5d8a1fbcd2 admin admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:00:08.807115 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e25ba3-882a-4280-bd0e-aa14dda99b47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:00:08.807772 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e25ba3-882a-4280-bd0e-aa14dda99b47 admin admin] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:00:08.808053 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e25ba3-882a-4280-bd0e-aa14dda99b47 admin admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:00:08.808318 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e25ba3-882a-4280-bd0e-aa14dda99b47 admin admin] PATH_INFO: `/v3/roles` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:00:08.808952 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61e25ba3-882a-4280-bd0e-aa14dda99b47 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:00:08.810553 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61e25ba3-882a-4280-bd0e-aa14dda99b47 admin admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:00:08.821717 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 1328/2652] 10.222.0.13 () {58 vars in 1122 bytes} [Wed Aug 31 10:00:08 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 10:00:08.825877 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1011862-08f9-48f6-8c0a-391f131baa1c admin admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:00:08.837062 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1011862-08f9-48f6-8c0a-391f131baa1c admin admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:00:08.842401 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a992a7ee-ec81-4ad5-b0ce-f99e4d6a5b21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:00:08.843066 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a992a7ee-ec81-4ad5-b0ce-f99e4d6a5b21 admin admin] REQUEST_METHOD: `PUT` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:00:08.843341 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a992a7ee-ec81-4ad5-b0ce-f99e4d6a5b21 admin admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:00:08.843606 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a992a7ee-ec81-4ad5-b0ce-f99e4d6a5b21 admin admin] PATH_INFO: `/v3/projects/82742be415734d858f0d43c33e1907e6/users/c4a2d164bdad4e83a298453ce59d3712/roles/5d4ee63a3add47fc9a6b8b16d39e0a1f` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:00:08.845596 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a992a7ee-ec81-4ad5-b0ce-f99e4d6a5b21 admin admin] RBAC: Authorizing `identity:create_grant(project_id=82742be415734d858f0d43c33e1907e6, user_id=c4a2d164bdad4e83a298453ce59d3712, role_id=5d4ee63a3add47fc9a6b8b16d39e0a1f)` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:00:08.848403 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a992a7ee-ec81-4ad5-b0ce-f99e4d6a5b21 admin admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:00:08.859384 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 1325/2653] 10.222.0.13 () {60 vars in 1483 bytes} [Wed Aug 31 10:00:08 2022] PUT /identity/v3/projects/82742be415734d858f0d43c33e1907e6/users/c4a2d164bdad4e83a298453ce59d3712/roles/5d4ee63a3add47fc9a6b8b16d39e0a1f => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 10:00:08.870947 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84606f5-b144-4837-9b42-aed51cd08a19 None None] REQUEST_METHOD: `POST` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:00:08.871327 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84606f5-b144-4837-9b42-aed51cd08a19 None None] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:00:08.871609 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84606f5-b144-4837-9b42-aed51cd08a19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:00:08.900723 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.auth.core [None req-a84606f5-b144-4837-9b42-aed51cd08a19 None None] MFA Rules not processed for user `c4a2d164bdad4e83a298453ce59d3712`. Rule list: `[]` (Enabled: `True`). {{(pid=88114) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 10:00:08.901965 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.fernet_utils [None req-a84606f5-b144-4837-9b42-aed51cd08a19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88114) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:00:08.936050 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:00:08.974515 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 1329/2654] 10.222.0.13 () {58 vars in 958 bytes} [Wed Aug 31 10:00:08 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 10:00:08.986693 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a992a7ee-ec81-4ad5-b0ce-f99e4d6a5b21 admin admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:00:08.988485 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.fernet_utils [None req-a992a7ee-ec81-4ad5-b0ce-f99e4d6a5b21 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:00:09.051899 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a992a7ee-ec81-4ad5-b0ce-f99e4d6a5b21 admin admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:00:09.055708 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac368ddd-3d5d-4e1a-81ba-be5083a4222c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d52b69c8e94441dacfcff944326f169', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '49548eb44d62415582af820ab66decee', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:00:09.056332 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac368ddd-3d5d-4e1a-81ba-be5083a4222c service cinder] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:00:09.056616 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac368ddd-3d5d-4e1a-81ba-be5083a4222c service cinder] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:00:09.056897 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac368ddd-3d5d-4e1a-81ba-be5083a4222c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:00:09.070309 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac368ddd-3d5d-4e1a-81ba-be5083a4222c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:00:09.071796 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac368ddd-3d5d-4e1a-81ba-be5083a4222c service cinder] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:00:09.085335 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 1326/2655] 10.222.0.13 () {60 vars in 1365 bytes} [Wed Aug 31 10:00:08 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 10:00:09.159533 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a84606f5-b144-4837-9b42-aed51cd08a19 None None] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:00:09.169675 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a84606f5-b144-4837-9b42-aed51cd08a19 None None] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:00:09.174439 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79fa16c2-7287-4640-a860-20252bbeed7b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d52b69c8e94441dacfcff944326f169', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '49548eb44d62415582af820ab66decee', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:00:09.176765 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79fa16c2-7287-4640-a860-20252bbeed7b service cinder] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:00:09.177042 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79fa16c2-7287-4640-a860-20252bbeed7b service cinder] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:00:09.177306 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79fa16c2-7287-4640-a860-20252bbeed7b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:00:09.240028 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79fa16c2-7287-4640-a860-20252bbeed7b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:00:09.241437 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79fa16c2-7287-4640-a860-20252bbeed7b service cinder] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:00:09.250554 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 1330/2656] 10.222.0.13 () {60 vars in 1365 bytes} [Wed Aug 31 10:00:09 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 10:00:09.547333 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac368ddd-3d5d-4e1a-81ba-be5083a4222c service cinder] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:00:09.567941 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:00:09.614797 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac368ddd-3d5d-4e1a-81ba-be5083a4222c service cinder] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:00:09.619389 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-237f84ad-2c21-4af2-8fe1-9f43155d6e4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:00:09.620021 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237f84ad-2c21-4af2-8fe1-9f43155d6e4c admin admin] REQUEST_METHOD: `DELETE` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:00:09.620298 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237f84ad-2c21-4af2-8fe1-9f43155d6e4c admin admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:00:09.620569 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237f84ad-2c21-4af2-8fe1-9f43155d6e4c admin admin] PATH_INFO: `/v3/users/034791535bad4c65b9a168f2d0a65659` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:00:09.621728 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-237f84ad-2c21-4af2-8fe1-9f43155d6e4c admin admin] RBAC: Authorizing `identity:delete_user(user_id=034791535bad4c65b9a168f2d0a65659)` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:00:09.623702 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-237f84ad-2c21-4af2-8fe1-9f43155d6e4c admin admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:00:09.671562 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.notifications [None req-237f84ad-2c21-4af2-8fe1-9f43155d6e4c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '034791535bad4c65b9a168f2d0a65659'} {{(pid=88113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 10:00:09.679574 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.notifications [None req-237f84ad-2c21-4af2-8fe1-9f43155d6e4c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '034791535bad4c65b9a168f2d0a65659'} {{(pid=88113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 10:00:09.691596 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.notifications [None req-237f84ad-2c21-4af2-8fe1-9f43155d6e4c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '034791535bad4c65b9a168f2d0a65659'} {{(pid=88113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 10:00:09.692303 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.notifications [None req-237f84ad-2c21-4af2-8fe1-9f43155d6e4c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '034791535bad4c65b9a168f2d0a65659'} {{(pid=88113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 10:00:09.699766 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.notifications [None req-237f84ad-2c21-4af2-8fe1-9f43155d6e4c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '034791535bad4c65b9a168f2d0a65659'} {{(pid=88113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 10:00:09.705781 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 1327/2657] 10.222.0.13 () {58 vars in 1224 bytes} [Wed Aug 31 10:00:09 2022] DELETE /identity/v3/users/034791535bad4c65b9a168f2d0a65659 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 10:00:09.710266 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51eb11ba-1935-4527-a478-e8162b173732 None None] REQUEST_METHOD: `POST` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:00:09.710553 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51eb11ba-1935-4527-a478-e8162b173732 None None] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:00:09.710826 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51eb11ba-1935-4527-a478-e8162b173732 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:00:09.731514 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.auth.core [None req-51eb11ba-1935-4527-a478-e8162b173732 None None] MFA Rules not processed for user `fd836cf590a145df9acf19cf3ae2ac56`. Rule list: `[]` (Enabled: `True`). {{(pid=88114) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 10:00:09.732734 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.fernet_utils [None req-51eb11ba-1935-4527-a478-e8162b173732 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88114) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:00:09.795189 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 1331/2658] 10.222.0.13 () {58 vars in 958 bytes} [Wed Aug 31 10:00:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 10:00:09.805422 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-237f84ad-2c21-4af2-8fe1-9f43155d6e4c admin admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:00:09.807494 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.fernet_utils [None req-237f84ad-2c21-4af2-8fe1-9f43155d6e4c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:00:09.865912 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-237f84ad-2c21-4af2-8fe1-9f43155d6e4c admin admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:00:09.869976 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-898eeec0-1c28-45e1-a69c-3a958338a561 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9bdaacd597442c2bcde2c555f2cc140', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '49548eb44d62415582af820ab66decee', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:00:09.870584 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898eeec0-1c28-45e1-a69c-3a958338a561 service neutron] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:00:09.870863 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898eeec0-1c28-45e1-a69c-3a958338a561 service neutron] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:00:09.871129 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898eeec0-1c28-45e1-a69c-3a958338a561 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:00:09.881545 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-898eeec0-1c28-45e1-a69c-3a958338a561 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:00:09.883045 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-898eeec0-1c28-45e1-a69c-3a958338a561 service neutron] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:00:09.894153 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 1328/2659] 10.222.0.13 () {60 vars in 1365 bytes} [Wed Aug 31 10:00:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 10:00:10.172693 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51eb11ba-1935-4527-a478-e8162b173732 None None] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:00:10.174842 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.fernet_utils [None req-51eb11ba-1935-4527-a478-e8162b173732 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88114) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:00:10.187024 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51eb11ba-1935-4527-a478-e8162b173732 None None] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:00:10.192255 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b8ca4a-0789-48b1-9a45-12abe58dc570 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:00:10.192890 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b8ca4a-0789-48b1-9a45-12abe58dc570 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:00:10.193258 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b8ca4a-0789-48b1-9a45-12abe58dc570 admin admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:00:10.193569 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b8ca4a-0789-48b1-9a45-12abe58dc570 admin admin] PATH_INFO: `/v3/projects/dab9f0edb74f4f58b73e5b95f29d0492` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:00:10.194777 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59b8ca4a-0789-48b1-9a45-12abe58dc570 admin admin] RBAC: Authorizing `identity:delete_project(project_id=dab9f0edb74f4f58b73e5b95f29d0492)` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:00:10.196810 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59b8ca4a-0789-48b1-9a45-12abe58dc570 admin admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:00:10.247071 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.notifications [None req-59b8ca4a-0789-48b1-9a45-12abe58dc570 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dab9f0edb74f4f58b73e5b95f29d0492', 'request_id': 'req-59b8ca4a-0789-48b1-9a45-12abe58dc570'} {{(pid=88114) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 10:00:10.253790 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.notifications [None req-59b8ca4a-0789-48b1-9a45-12abe58dc570 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dab9f0edb74f4f58b73e5b95f29d0492', 'request_id': 'req-59b8ca4a-0789-48b1-9a45-12abe58dc570'} {{(pid=88114) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 10:00:10.259835 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.notifications [None req-59b8ca4a-0789-48b1-9a45-12abe58dc570 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dab9f0edb74f4f58b73e5b95f29d0492', 'request_id': 'req-59b8ca4a-0789-48b1-9a45-12abe58dc570'} {{(pid=88114) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 10:00:10.262332 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=93,keystone:DELETE=6,keystone:INSERT=1 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:00:10.275430 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.notifications [None req-59b8ca4a-0789-48b1-9a45-12abe58dc570 admin admin] The token cache is being invalidate because project dab9f0edb74f4f58b73e5b95f29d0492 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88114) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 10:00:10.275723 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.notifications [None req-59b8ca4a-0789-48b1-9a45-12abe58dc570 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88114) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 10:00:10.276353 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.notifications [None req-59b8ca4a-0789-48b1-9a45-12abe58dc570 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88114) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 10:00:10.277880 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 1332/2660] 10.222.0.13 () {58 vars in 1233 bytes} [Wed Aug 31 10:00:10 2022] DELETE /identity/v3/projects/dab9f0edb74f4f58b73e5b95f29d0492 => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 10:00:10.281828 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-898eeec0-1c28-45e1-a69c-3a958338a561 service neutron] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:00:10.291669 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-898eeec0-1c28-45e1-a69c-3a958338a561 service neutron] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:00:10.332675 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:00:10.353215 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df495bb-6b71-4dcc-90c4-164689e750be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:00:10.353872 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df495bb-6b71-4dcc-90c4-164689e750be admin admin] REQUEST_METHOD: `DELETE` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:00:10.354146 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df495bb-6b71-4dcc-90c4-164689e750be admin admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:00:10.354415 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df495bb-6b71-4dcc-90c4-164689e750be admin admin] PATH_INFO: `/v3/users/c4a2d164bdad4e83a298453ce59d3712` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:00:10.355541 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4df495bb-6b71-4dcc-90c4-164689e750be admin admin] RBAC: Authorizing `identity:delete_user(user_id=c4a2d164bdad4e83a298453ce59d3712)` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:00:10.357476 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4df495bb-6b71-4dcc-90c4-164689e750be admin admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:00:10.404236 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.notifications [None req-4df495bb-6b71-4dcc-90c4-164689e750be admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c4a2d164bdad4e83a298453ce59d3712'} {{(pid=88113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 10:00:10.411622 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.notifications [None req-4df495bb-6b71-4dcc-90c4-164689e750be admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c4a2d164bdad4e83a298453ce59d3712'} {{(pid=88113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 10:00:10.423528 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.notifications [None req-4df495bb-6b71-4dcc-90c4-164689e750be admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c4a2d164bdad4e83a298453ce59d3712'} {{(pid=88113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 10:00:10.424232 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.notifications [None req-4df495bb-6b71-4dcc-90c4-164689e750be admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c4a2d164bdad4e83a298453ce59d3712'} {{(pid=88113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 10:00:10.432759 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.notifications [None req-4df495bb-6b71-4dcc-90c4-164689e750be admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c4a2d164bdad4e83a298453ce59d3712'} {{(pid=88113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 10:00:10.438938 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 1329/2661] 10.222.0.13 () {58 vars in 1224 bytes} [Wed Aug 31 10:00:10 2022] DELETE /identity/v3/users/c4a2d164bdad4e83a298453ce59d3712 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 10:00:10.722916 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b8ca4a-0789-48b1-9a45-12abe58dc570 admin admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:00:10.724403 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.fernet_utils [None req-59b8ca4a-0789-48b1-9a45-12abe58dc570 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88114) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:00:10.738391 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b8ca4a-0789-48b1-9a45-12abe58dc570 admin admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:00:10.807159 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13382d12-db3e-4d49-82ac-4189a2fc09ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:00:10.807817 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13382d12-db3e-4d49-82ac-4189a2fc09ae admin admin] REQUEST_METHOD: `DELETE` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:00:10.808110 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13382d12-db3e-4d49-82ac-4189a2fc09ae admin admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:00:10.808370 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13382d12-db3e-4d49-82ac-4189a2fc09ae admin admin] PATH_INFO: `/v3/projects/82742be415734d858f0d43c33e1907e6` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:00:10.809612 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13382d12-db3e-4d49-82ac-4189a2fc09ae admin admin] RBAC: Authorizing `identity:delete_project(project_id=82742be415734d858f0d43c33e1907e6)` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:00:10.811528 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13382d12-db3e-4d49-82ac-4189a2fc09ae admin admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:00:10.858570 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.notifications [None req-13382d12-db3e-4d49-82ac-4189a2fc09ae admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '82742be415734d858f0d43c33e1907e6', 'request_id': 'req-13382d12-db3e-4d49-82ac-4189a2fc09ae'} {{(pid=88114) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 10:00:10.865543 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.notifications [None req-13382d12-db3e-4d49-82ac-4189a2fc09ae admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '82742be415734d858f0d43c33e1907e6', 'request_id': 'req-13382d12-db3e-4d49-82ac-4189a2fc09ae'} {{(pid=88114) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 10:00:10.871741 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.notifications [None req-13382d12-db3e-4d49-82ac-4189a2fc09ae admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '82742be415734d858f0d43c33e1907e6', 'request_id': 'req-13382d12-db3e-4d49-82ac-4189a2fc09ae'} {{(pid=88114) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 10:00:10.884074 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.notifications [None req-13382d12-db3e-4d49-82ac-4189a2fc09ae admin admin] The token cache is being invalidate because project 82742be415734d858f0d43c33e1907e6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88114) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 10:00:10.884361 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.notifications [None req-13382d12-db3e-4d49-82ac-4189a2fc09ae admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88114) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 10:00:10.884985 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.notifications [None req-13382d12-db3e-4d49-82ac-4189a2fc09ae admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88114) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 10:00:10.886481 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 1333/2662] 10.222.0.13 () {58 vars in 1233 bytes} [Wed Aug 31 10:00:10 2022] DELETE /identity/v3/projects/82742be415734d858f0d43c33e1907e6 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 10:00:10.891393 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e047ff-191b-4a10-a6d6-f3a1652d1d0d None None] REQUEST_METHOD: `POST` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:00:10.891863 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e047ff-191b-4a10-a6d6-f3a1652d1d0d None None] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:00:10.891961 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e047ff-191b-4a10-a6d6-f3a1652d1d0d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:00:10.911720 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.auth.core [None req-89e047ff-191b-4a10-a6d6-f3a1652d1d0d None None] MFA Rules not processed for user `fd836cf590a145df9acf19cf3ae2ac56`. Rule list: `[]` (Enabled: `True`). {{(pid=88113) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 10:00:10.912916 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.fernet_utils [None req-89e047ff-191b-4a10-a6d6-f3a1652d1d0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:00:10.973317 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 1330/2663] 10.222.0.13 () {58 vars in 958 bytes} [Wed Aug 31 10:00:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 10:00:10.981147 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332694c6-1449-48b4-b418-c3699f344389 None None] REQUEST_METHOD: `POST` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:00:10.981483 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332694c6-1449-48b4-b418-c3699f344389 None None] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:00:10.981751 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332694c6-1449-48b4-b418-c3699f344389 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:00:11.002135 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.auth.core [None req-332694c6-1449-48b4-b418-c3699f344389 None None] MFA Rules not processed for user `fd836cf590a145df9acf19cf3ae2ac56`. Rule list: `[]` (Enabled: `True`). {{(pid=88114) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 10:00:11.003392 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.fernet_utils [None req-332694c6-1449-48b4-b418-c3699f344389 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88114) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:00:11.011510 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 1334/2664] 10.222.0.13 () {58 vars in 958 bytes} [Wed Aug 31 10:00:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 10:00:11.015906 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e047ff-191b-4a10-a6d6-f3a1652d1d0d None None] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:00:11.026375 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e047ff-191b-4a10-a6d6-f3a1652d1d0d None None] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:00:11.030988 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffae5f68-ba23-42dc-9398-30a2d7e444b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:00:11.031585 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffae5f68-ba23-42dc-9398-30a2d7e444b3 admin admin] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:00:11.031862 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffae5f68-ba23-42dc-9398-30a2d7e444b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:00:11.032135 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffae5f68-ba23-42dc-9398-30a2d7e444b3 admin admin] PATH_INFO: `/v3/domains` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:00:11.032791 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffae5f68-ba23-42dc-9398-30a2d7e444b3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88113) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:00:11.033115 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffae5f68-ba23-42dc-9398-30a2d7e444b3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:00:11.035644 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffae5f68-ba23-42dc-9398-30a2d7e444b3 admin admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:00:11.047243 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 1331/2665] 10.222.0.13 () {58 vars in 1153 bytes} [Wed Aug 31 10:00:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:00:11.051851 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-332694c6-1449-48b4-b418-c3699f344389 None None] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:00:11.061225 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-332694c6-1449-48b4-b418-c3699f344389 None None] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:00:11.066572 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-107118d4-81cb-49b0-be1a-60fb9139ba7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:00:11.067176 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107118d4-81cb-49b0-be1a-60fb9139ba7b admin admin] REQUEST_METHOD: `POST` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:00:11.067550 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107118d4-81cb-49b0-be1a-60fb9139ba7b admin admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:00:11.067824 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107118d4-81cb-49b0-be1a-60fb9139ba7b admin admin] PATH_INFO: `/v3/projects` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:00:11.068458 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-107118d4-81cb-49b0-be1a-60fb9139ba7b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:00:11.070389 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-107118d4-81cb-49b0-be1a-60fb9139ba7b admin admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:00:11.098584 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 1335/2666] 10.222.0.13 () {60 vars in 1153 bytes} [Wed Aug 31 10:00:11 2022] POST /identity/v3/projects => generated 371 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 10:00:11.102966 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffae5f68-ba23-42dc-9398-30a2d7e444b3 admin admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:00:11.114066 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffae5f68-ba23-42dc-9398-30a2d7e444b3 admin admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:00:11.157664 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:00:11.171519 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b337c1d2-5418-4115-ae73-d25e51f6cc5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:00:11.172117 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b337c1d2-5418-4115-ae73-d25e51f6cc5d admin admin] REQUEST_METHOD: `POST` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:00:11.172393 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b337c1d2-5418-4115-ae73-d25e51f6cc5d admin admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:00:11.172661 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b337c1d2-5418-4115-ae73-d25e51f6cc5d admin admin] PATH_INFO: `/v3/users` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:00:11.173284 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b337c1d2-5418-4115-ae73-d25e51f6cc5d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:00:11.175423 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b337c1d2-5418-4115-ae73-d25e51f6cc5d admin admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:00:11.197238 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 1332/2667] 10.222.0.13 () {60 vars in 1144 bytes} [Wed Aug 31 10:00:11 2022] POST /identity/v3/users => generated 286 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 10:00:11.201538 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-107118d4-81cb-49b0-be1a-60fb9139ba7b admin admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:00:11.212578 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-107118d4-81cb-49b0-be1a-60fb9139ba7b admin admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:00:11.217191 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ebcea54-d7a6-40f0-bcb9-dc223f2dc9dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:00:11.217849 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ebcea54-d7a6-40f0-bcb9-dc223f2dc9dd admin admin] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:00:11.218124 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ebcea54-d7a6-40f0-bcb9-dc223f2dc9dd admin admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:00:11.218393 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ebcea54-d7a6-40f0-bcb9-dc223f2dc9dd admin admin] PATH_INFO: `/v3/roles` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:00:11.219029 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ebcea54-d7a6-40f0-bcb9-dc223f2dc9dd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:00:11.220602 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ebcea54-d7a6-40f0-bcb9-dc223f2dc9dd admin admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:00:11.231489 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 1336/2668] 10.222.0.13 () {58 vars in 1122 bytes} [Wed Aug 31 10:00:11 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 10:00:11.235477 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b337c1d2-5418-4115-ae73-d25e51f6cc5d admin admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:00:11.246665 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b337c1d2-5418-4115-ae73-d25e51f6cc5d admin admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:00:11.251288 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea19041-6b16-43a0-9549-0f04ab880e3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:00:11.251891 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea19041-6b16-43a0-9549-0f04ab880e3e admin admin] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:00:11.252170 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea19041-6b16-43a0-9549-0f04ab880e3e admin admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:00:11.252447 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea19041-6b16-43a0-9549-0f04ab880e3e admin admin] PATH_INFO: `/v3/roles` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:00:11.253088 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ea19041-6b16-43a0-9549-0f04ab880e3e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:00:11.254877 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ea19041-6b16-43a0-9549-0f04ab880e3e admin admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:00:11.265664 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 1333/2669] 10.222.0.13 () {58 vars in 1122 bytes} [Wed Aug 31 10:00:11 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 10:00:11.270020 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ebcea54-d7a6-40f0-bcb9-dc223f2dc9dd admin admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:00:11.279890 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ebcea54-d7a6-40f0-bcb9-dc223f2dc9dd admin admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:00:11.285168 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-587da1fe-1006-471c-8b4b-04b9863c90f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:00:11.285891 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587da1fe-1006-471c-8b4b-04b9863c90f6 admin admin] REQUEST_METHOD: `PUT` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:00:11.286169 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587da1fe-1006-471c-8b4b-04b9863c90f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:00:11.286438 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587da1fe-1006-471c-8b4b-04b9863c90f6 admin admin] PATH_INFO: `/v3/projects/92d95ab2b5c84b5d81b5fb4096abb0de/users/ad8d41feb37248529df9e5cae5d142d2/roles/6c99f5d9f1f243eaa134d8540f21095d` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:00:11.301699 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-587da1fe-1006-471c-8b4b-04b9863c90f6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=92d95ab2b5c84b5d81b5fb4096abb0de, user_id=ad8d41feb37248529df9e5cae5d142d2, role_id=6c99f5d9f1f243eaa134d8540f21095d)` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:00:11.304310 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-587da1fe-1006-471c-8b4b-04b9863c90f6 admin admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:00:11.317940 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 1337/2670] 10.222.0.13 () {60 vars in 1483 bytes} [Wed Aug 31 10:00:11 2022] PUT /identity/v3/projects/92d95ab2b5c84b5d81b5fb4096abb0de/users/ad8d41feb37248529df9e5cae5d142d2/roles/6c99f5d9f1f243eaa134d8540f21095d => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 10:00:11.328095 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3925563-3d1a-4332-82cc-a10a0d778de3 None None] REQUEST_METHOD: `POST` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:00:11.328383 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3925563-3d1a-4332-82cc-a10a0d778de3 None None] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:00:11.328656 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3925563-3d1a-4332-82cc-a10a0d778de3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:00:11.358698 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.auth.core [None req-e3925563-3d1a-4332-82cc-a10a0d778de3 None None] MFA Rules not processed for user `ad8d41feb37248529df9e5cae5d142d2`. Rule list: `[]` (Enabled: `True`). {{(pid=88113) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 10:00:11.359913 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.fernet_utils [None req-e3925563-3d1a-4332-82cc-a10a0d778de3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:00:11.425368 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 1334/2671] 10.222.0.13 () {58 vars in 958 bytes} [Wed Aug 31 10:00:11 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 10:00:11.436570 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-587da1fe-1006-471c-8b4b-04b9863c90f6 admin admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:00:11.438317 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.fernet_utils [None req-587da1fe-1006-471c-8b4b-04b9863c90f6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88114) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:00:11.461892 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:DELETE=8,keystone:SELECT=89,keystone:INSERT=3 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:00:11.499582 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-587da1fe-1006-471c-8b4b-04b9863c90f6 admin admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:00:11.503680 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0963c95-62ff-44c9-aaa7-0eb5613a6647 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d52b69c8e94441dacfcff944326f169', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '49548eb44d62415582af820ab66decee', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:00:11.504273 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0963c95-62ff-44c9-aaa7-0eb5613a6647 service cinder] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:00:11.504547 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0963c95-62ff-44c9-aaa7-0eb5613a6647 service cinder] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:00:11.504811 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0963c95-62ff-44c9-aaa7-0eb5613a6647 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:00:11.512921 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0963c95-62ff-44c9-aaa7-0eb5613a6647 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:00:11.514362 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0963c95-62ff-44c9-aaa7-0eb5613a6647 service cinder] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:00:11.523918 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 1338/2672] 10.222.0.13 () {60 vars in 1365 bytes} [Wed Aug 31 10:00:11 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 10:00:11.545642 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3925563-3d1a-4332-82cc-a10a0d778de3 None None] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:00:11.608715 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3925563-3d1a-4332-82cc-a10a0d778de3 None None] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:00:11.614863 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e1ce9cb-3c21-4059-9ee2-50e6b4170b15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:00:11.615506 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1ce9cb-3c21-4059-9ee2-50e6b4170b15 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:00:11.615814 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1ce9cb-3c21-4059-9ee2-50e6b4170b15 admin admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:00:11.616114 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1ce9cb-3c21-4059-9ee2-50e6b4170b15 admin admin] PATH_INFO: `/v3/users/ad8d41feb37248529df9e5cae5d142d2` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:00:11.617293 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e1ce9cb-3c21-4059-9ee2-50e6b4170b15 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ad8d41feb37248529df9e5cae5d142d2)` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:00:11.619304 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e1ce9cb-3c21-4059-9ee2-50e6b4170b15 admin admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:00:11.667637 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.notifications [None req-2e1ce9cb-3c21-4059-9ee2-50e6b4170b15 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ad8d41feb37248529df9e5cae5d142d2'} {{(pid=88113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 10:00:11.674824 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.notifications [None req-2e1ce9cb-3c21-4059-9ee2-50e6b4170b15 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ad8d41feb37248529df9e5cae5d142d2'} {{(pid=88113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 10:00:11.676193 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:00:11.690765 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.notifications [None req-2e1ce9cb-3c21-4059-9ee2-50e6b4170b15 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ad8d41feb37248529df9e5cae5d142d2'} {{(pid=88113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 10:00:11.691666 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.notifications [None req-2e1ce9cb-3c21-4059-9ee2-50e6b4170b15 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ad8d41feb37248529df9e5cae5d142d2'} {{(pid=88113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 10:00:11.700172 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.notifications [None req-2e1ce9cb-3c21-4059-9ee2-50e6b4170b15 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ad8d41feb37248529df9e5cae5d142d2'} {{(pid=88113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 10:00:11.706898 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 1335/2673] 10.222.0.13 () {58 vars in 1224 bytes} [Wed Aug 31 10:00:11 2022] DELETE /identity/v3/users/ad8d41feb37248529df9e5cae5d142d2 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 10:00:11.711394 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb15ed7c-eeaa-45b0-ad0c-387ef6673c8e None None] REQUEST_METHOD: `POST` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:00:11.711682 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb15ed7c-eeaa-45b0-ad0c-387ef6673c8e None None] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:00:11.711955 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb15ed7c-eeaa-45b0-ad0c-387ef6673c8e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:00:11.731577 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.auth.core [None req-bb15ed7c-eeaa-45b0-ad0c-387ef6673c8e None None] MFA Rules not processed for user `fd836cf590a145df9acf19cf3ae2ac56`. Rule list: `[]` (Enabled: `True`). {{(pid=88114) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 10:00:11.732771 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.fernet_utils [None req-bb15ed7c-eeaa-45b0-ad0c-387ef6673c8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88114) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:00:11.797622 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 1339/2674] 10.222.0.13 () {58 vars in 958 bytes} [Wed Aug 31 10:00:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 10:00:11.807672 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e1ce9cb-3c21-4059-9ee2-50e6b4170b15 admin admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:00:11.809136 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.fernet_utils [None req-2e1ce9cb-3c21-4059-9ee2-50e6b4170b15 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:00:11.866655 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e1ce9cb-3c21-4059-9ee2-50e6b4170b15 admin admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:00:11.870299 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a866e5-3c43-4279-abb4-7ecdbe3b69be service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9bdaacd597442c2bcde2c555f2cc140', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '49548eb44d62415582af820ab66decee', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:00:11.870899 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a866e5-3c43-4279-abb4-7ecdbe3b69be service neutron] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:00:11.871170 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a866e5-3c43-4279-abb4-7ecdbe3b69be service neutron] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:00:11.871439 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a866e5-3c43-4279-abb4-7ecdbe3b69be service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:00:11.880960 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56a866e5-3c43-4279-abb4-7ecdbe3b69be service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:00:11.882391 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56a866e5-3c43-4279-abb4-7ecdbe3b69be service neutron] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:00:11.891771 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 1336/2675] 10.222.0.13 () {60 vars in 1365 bytes} [Wed Aug 31 10:00:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 10:00:12.165074 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb15ed7c-eeaa-45b0-ad0c-387ef6673c8e None None] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:00:12.166617 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.fernet_utils [None req-bb15ed7c-eeaa-45b0-ad0c-387ef6673c8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88114) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:00:12.180618 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb15ed7c-eeaa-45b0-ad0c-387ef6673c8e None None] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:00:12.186122 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23269a69-d90f-4a6b-8685-f2e2a923f981 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:00:12.187108 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23269a69-d90f-4a6b-8685-f2e2a923f981 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:00:12.187395 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23269a69-d90f-4a6b-8685-f2e2a923f981 admin admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:00:12.187762 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23269a69-d90f-4a6b-8685-f2e2a923f981 admin admin] PATH_INFO: `/v3/projects/92d95ab2b5c84b5d81b5fb4096abb0de` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:00:12.188879 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23269a69-d90f-4a6b-8685-f2e2a923f981 admin admin] RBAC: Authorizing `identity:delete_project(project_id=92d95ab2b5c84b5d81b5fb4096abb0de)` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:00:12.192535 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23269a69-d90f-4a6b-8685-f2e2a923f981 admin admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:00:12.240656 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.notifications [None req-23269a69-d90f-4a6b-8685-f2e2a923f981 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '92d95ab2b5c84b5d81b5fb4096abb0de', 'request_id': 'req-23269a69-d90f-4a6b-8685-f2e2a923f981'} {{(pid=88114) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 10:00:12.246678 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.notifications [None req-23269a69-d90f-4a6b-8685-f2e2a923f981 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '92d95ab2b5c84b5d81b5fb4096abb0de', 'request_id': 'req-23269a69-d90f-4a6b-8685-f2e2a923f981'} {{(pid=88114) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 10:00:12.252338 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.notifications [None req-23269a69-d90f-4a6b-8685-f2e2a923f981 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '92d95ab2b5c84b5d81b5fb4096abb0de', 'request_id': 'req-23269a69-d90f-4a6b-8685-f2e2a923f981'} {{(pid=88114) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 10:00:12.266657 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.notifications [None req-23269a69-d90f-4a6b-8685-f2e2a923f981 admin admin] The token cache is being invalidate because project 92d95ab2b5c84b5d81b5fb4096abb0de was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88114) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 10:00:12.266942 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.notifications [None req-23269a69-d90f-4a6b-8685-f2e2a923f981 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88114) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 10:00:12.267582 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.notifications [None req-23269a69-d90f-4a6b-8685-f2e2a923f981 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88114) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 10:00:12.270172 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 1340/2676] 10.222.0.13 () {58 vars in 1233 bytes} [Wed Aug 31 10:00:12 2022] DELETE /identity/v3/projects/92d95ab2b5c84b5d81b5fb4096abb0de => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 10:00:12.274985 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f047d633-d9b0-4d68-bba1-8bbc11da8c36 None None] REQUEST_METHOD: `POST` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:00:12.275267 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f047d633-d9b0-4d68-bba1-8bbc11da8c36 None None] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:00:12.275540 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f047d633-d9b0-4d68-bba1-8bbc11da8c36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:00:12.295792 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.auth.core [None req-f047d633-d9b0-4d68-bba1-8bbc11da8c36 None None] MFA Rules not processed for user `fd836cf590a145df9acf19cf3ae2ac56`. Rule list: `[]` (Enabled: `True`). {{(pid=88113) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 10:00:12.296979 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.fernet_utils [None req-f047d633-d9b0-4d68-bba1-8bbc11da8c36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:00:12.356496 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 1337/2677] 10.222.0.13 () {58 vars in 958 bytes} [Wed Aug 31 10:00:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 10:00:12.366145 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d453a0-ee5b-40bb-b9c9-07bdcbe3721f None None] REQUEST_METHOD: `POST` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:00:12.366428 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d453a0-ee5b-40bb-b9c9-07bdcbe3721f None None] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:00:12.366699 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d453a0-ee5b-40bb-b9c9-07bdcbe3721f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:00:12.387039 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.auth.core [None req-88d453a0-ee5b-40bb-b9c9-07bdcbe3721f None None] MFA Rules not processed for user `fd836cf590a145df9acf19cf3ae2ac56`. Rule list: `[]` (Enabled: `True`). {{(pid=88114) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 10:00:12.388245 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.fernet_utils [None req-88d453a0-ee5b-40bb-b9c9-07bdcbe3721f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88114) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:00:12.396681 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 1341/2678] 10.222.0.13 () {58 vars in 958 bytes} [Wed Aug 31 10:00:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 10:00:12.401141 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f047d633-d9b0-4d68-bba1-8bbc11da8c36 None None] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:00:12.411132 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f047d633-d9b0-4d68-bba1-8bbc11da8c36 None None] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:00:12.416820 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f927ee7-543d-4f56-ad91-12d8bb1d34fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:00:12.417441 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f927ee7-543d-4f56-ad91-12d8bb1d34fc admin admin] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:00:12.417720 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f927ee7-543d-4f56-ad91-12d8bb1d34fc admin admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:00:12.417987 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f927ee7-543d-4f56-ad91-12d8bb1d34fc admin admin] PATH_INFO: `/v3/domains` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:00:12.418630 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f927ee7-543d-4f56-ad91-12d8bb1d34fc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88113) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:00:12.418929 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f927ee7-543d-4f56-ad91-12d8bb1d34fc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:00:12.420521 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f927ee7-543d-4f56-ad91-12d8bb1d34fc admin admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:00:12.434779 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 1338/2679] 10.222.0.13 () {58 vars in 1153 bytes} [Wed Aug 31 10:00:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:00:12.439638 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d453a0-ee5b-40bb-b9c9-07bdcbe3721f None None] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:00:12.450018 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d453a0-ee5b-40bb-b9c9-07bdcbe3721f None None] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:00:12.455092 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e9e353-cc67-41e4-b067-211fea1096a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:00:12.455687 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e9e353-cc67-41e4-b067-211fea1096a5 admin admin] REQUEST_METHOD: `POST` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:00:12.456124 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e9e353-cc67-41e4-b067-211fea1096a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:00:12.456438 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e9e353-cc67-41e4-b067-211fea1096a5 admin admin] PATH_INFO: `/v3/projects` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:00:12.457080 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2e9e353-cc67-41e4-b067-211fea1096a5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:00:12.458995 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2e9e353-cc67-41e4-b067-211fea1096a5 admin admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:00:12.487244 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 1342/2680] 10.222.0.13 () {60 vars in 1153 bytes} [Wed Aug 31 10:00:12 2022] POST /identity/v3/projects => generated 375 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 10:00:12.492300 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f927ee7-543d-4f56-ad91-12d8bb1d34fc admin admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:00:12.502185 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f927ee7-543d-4f56-ad91-12d8bb1d34fc admin admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:00:12.560029 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-352222d6-8873-441e-ad4a-6968727f2121 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:00:12.560630 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352222d6-8873-441e-ad4a-6968727f2121 admin admin] REQUEST_METHOD: `POST` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:00:12.560905 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352222d6-8873-441e-ad4a-6968727f2121 admin admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:00:12.561174 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352222d6-8873-441e-ad4a-6968727f2121 admin admin] PATH_INFO: `/v3/users` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:00:12.561836 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-352222d6-8873-441e-ad4a-6968727f2121 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:00:12.563749 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-352222d6-8873-441e-ad4a-6968727f2121 admin admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:00:12.573856 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=94,keystone:DELETE=3,keystone:INSERT=3 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:00:12.585814 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 1339/2681] 10.222.0.13 () {60 vars in 1144 bytes} [Wed Aug 31 10:00:12 2022] POST /identity/v3/users => generated 288 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 10:00:12.590059 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e9e353-cc67-41e4-b067-211fea1096a5 admin admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:00:12.601107 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e9e353-cc67-41e4-b067-211fea1096a5 admin admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:00:12.605872 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bbe30c1-4a1a-4013-9dbc-cba13b8bce7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:00:12.606463 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bbe30c1-4a1a-4013-9dbc-cba13b8bce7f admin admin] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:00:12.606733 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bbe30c1-4a1a-4013-9dbc-cba13b8bce7f admin admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:00:12.606995 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bbe30c1-4a1a-4013-9dbc-cba13b8bce7f admin admin] PATH_INFO: `/v3/roles` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:00:12.607623 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bbe30c1-4a1a-4013-9dbc-cba13b8bce7f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:00:12.609188 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bbe30c1-4a1a-4013-9dbc-cba13b8bce7f admin admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:00:12.620207 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 1343/2682] 10.222.0.13 () {58 vars in 1122 bytes} [Wed Aug 31 10:00:12 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 10:00:12.624591 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-352222d6-8873-441e-ad4a-6968727f2121 admin admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:00:12.636787 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-352222d6-8873-441e-ad4a-6968727f2121 admin admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:00:12.641584 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad27a01-dd49-42b9-8703-b999af4d9feb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:00:12.642189 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad27a01-dd49-42b9-8703-b999af4d9feb admin admin] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:00:12.642469 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad27a01-dd49-42b9-8703-b999af4d9feb admin admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:00:12.642743 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad27a01-dd49-42b9-8703-b999af4d9feb admin admin] PATH_INFO: `/v3/roles` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:00:12.643389 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ad27a01-dd49-42b9-8703-b999af4d9feb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:00:12.644990 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ad27a01-dd49-42b9-8703-b999af4d9feb admin admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:00:12.656760 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 1340/2683] 10.222.0.13 () {58 vars in 1122 bytes} [Wed Aug 31 10:00:12 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 10:00:12.661633 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bbe30c1-4a1a-4013-9dbc-cba13b8bce7f admin admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:00:12.672200 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bbe30c1-4a1a-4013-9dbc-cba13b8bce7f admin admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:00:12.677685 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf674c25-bcd1-4ac0-850e-2e9e7aac137c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:00:12.678318 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf674c25-bcd1-4ac0-850e-2e9e7aac137c admin admin] REQUEST_METHOD: `PUT` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:00:12.678590 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf674c25-bcd1-4ac0-850e-2e9e7aac137c admin admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:00:12.678857 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf674c25-bcd1-4ac0-850e-2e9e7aac137c admin admin] PATH_INFO: `/v3/projects/4240431c61eb4c238296cd2934e1bb69/users/c2f2f9cb647343689a55e175eee6c5d9/roles/6c99f5d9f1f243eaa134d8540f21095d` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:00:12.681040 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:00:12.695958 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf674c25-bcd1-4ac0-850e-2e9e7aac137c admin admin] RBAC: Authorizing `identity:create_grant(project_id=4240431c61eb4c238296cd2934e1bb69, user_id=c2f2f9cb647343689a55e175eee6c5d9, role_id=6c99f5d9f1f243eaa134d8540f21095d)` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:00:12.698610 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf674c25-bcd1-4ac0-850e-2e9e7aac137c admin admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:00:12.711984 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 1344/2684] 10.222.0.13 () {60 vars in 1483 bytes} [Wed Aug 31 10:00:12 2022] PUT /identity/v3/projects/4240431c61eb4c238296cd2934e1bb69/users/c2f2f9cb647343689a55e175eee6c5d9/roles/6c99f5d9f1f243eaa134d8540f21095d => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 10:00:12.722734 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c03a940-a415-4f08-a66f-5f586f953979 None None] REQUEST_METHOD: `POST` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:00:12.723034 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c03a940-a415-4f08-a66f-5f586f953979 None None] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:00:12.723311 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c03a940-a415-4f08-a66f-5f586f953979 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:00:12.752689 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.auth.core [None req-6c03a940-a415-4f08-a66f-5f586f953979 None None] MFA Rules not processed for user `c2f2f9cb647343689a55e175eee6c5d9`. Rule list: `[]` (Enabled: `True`). {{(pid=88113) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 10:00:12.754437 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.fernet_utils [None req-6c03a940-a415-4f08-a66f-5f586f953979 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:00:12.821100 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 1341/2685] 10.222.0.13 () {58 vars in 958 bytes} [Wed Aug 31 10:00:12 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 10:00:12.832689 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf674c25-bcd1-4ac0-850e-2e9e7aac137c admin admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:00:12.834367 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.fernet_utils [None req-bf674c25-bcd1-4ac0-850e-2e9e7aac137c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88114) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:00:12.891561 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf674c25-bcd1-4ac0-850e-2e9e7aac137c admin admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:00:12.895318 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d79b79ad-ebfb-495a-92c4-66516627cabc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d52b69c8e94441dacfcff944326f169', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '49548eb44d62415582af820ab66decee', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:00:12.895907 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79b79ad-ebfb-495a-92c4-66516627cabc service cinder] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:00:12.896178 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79b79ad-ebfb-495a-92c4-66516627cabc service cinder] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:00:12.896439 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79b79ad-ebfb-495a-92c4-66516627cabc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:00:12.904549 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d79b79ad-ebfb-495a-92c4-66516627cabc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:00:12.905992 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d79b79ad-ebfb-495a-92c4-66516627cabc service cinder] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:00:12.915153 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 1345/2686] 10.222.0.13 () {60 vars in 1365 bytes} [Wed Aug 31 10:00:12 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 10:00:15.958602 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c03a940-a415-4f08-a66f-5f586f953979 None None] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:00:16.021599 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c03a940-a415-4f08-a66f-5f586f953979 None None] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:00:16.026177 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c9538d0-e419-4b89-9782-7423e733ea5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:00:16.026803 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9538d0-e419-4b89-9782-7423e733ea5a admin admin] REQUEST_METHOD: `DELETE` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:00:16.027078 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9538d0-e419-4b89-9782-7423e733ea5a admin admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:00:16.027348 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9538d0-e419-4b89-9782-7423e733ea5a admin admin] PATH_INFO: `/v3/users/c2f2f9cb647343689a55e175eee6c5d9` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:00:16.028449 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c9538d0-e419-4b89-9782-7423e733ea5a admin admin] RBAC: Authorizing `identity:delete_user(user_id=c2f2f9cb647343689a55e175eee6c5d9)` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:00:16.030405 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c9538d0-e419-4b89-9782-7423e733ea5a admin admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:00:16.080432 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.notifications [None req-3c9538d0-e419-4b89-9782-7423e733ea5a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c2f2f9cb647343689a55e175eee6c5d9'} {{(pid=88113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 10:00:16.087730 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.notifications [None req-3c9538d0-e419-4b89-9782-7423e733ea5a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c2f2f9cb647343689a55e175eee6c5d9'} {{(pid=88113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 10:00:16.099979 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.notifications [None req-3c9538d0-e419-4b89-9782-7423e733ea5a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c2f2f9cb647343689a55e175eee6c5d9'} {{(pid=88113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 10:00:16.100676 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.notifications [None req-3c9538d0-e419-4b89-9782-7423e733ea5a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c2f2f9cb647343689a55e175eee6c5d9'} {{(pid=88113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 10:00:16.102368 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:INSERT=1,keystone:SELECT=90,keystone:DELETE=9 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:00:16.108917 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.notifications [None req-3c9538d0-e419-4b89-9782-7423e733ea5a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c2f2f9cb647343689a55e175eee6c5d9'} {{(pid=88113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 10:00:16.116719 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 1342/2687] 10.222.0.13 () {58 vars in 1224 bytes} [Wed Aug 31 10:00:15 2022] DELETE /identity/v3/users/c2f2f9cb647343689a55e175eee6c5d9 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 10:00:16.121118 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b2a6b4-d821-4805-b7e1-c31ea912f236 None None] REQUEST_METHOD: `POST` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:00:16.121634 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b2a6b4-d821-4805-b7e1-c31ea912f236 None None] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:00:16.121737 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b2a6b4-d821-4805-b7e1-c31ea912f236 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:00:16.140263 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.auth.core [None req-74b2a6b4-d821-4805-b7e1-c31ea912f236 None None] MFA Rules not processed for user `fd836cf590a145df9acf19cf3ae2ac56`. Rule list: `[]` (Enabled: `True`). {{(pid=88114) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 10:00:16.141487 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.fernet_utils [None req-74b2a6b4-d821-4805-b7e1-c31ea912f236 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88114) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:00:16.198884 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 1346/2688] 10.222.0.13 () {58 vars in 958 bytes} [Wed Aug 31 10:00:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 10:00:16.209145 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c9538d0-e419-4b89-9782-7423e733ea5a admin admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:00:16.210623 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.fernet_utils [None req-3c9538d0-e419-4b89-9782-7423e733ea5a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:00:16.269857 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c9538d0-e419-4b89-9782-7423e733ea5a admin admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:00:16.273700 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2289ab2f-4a09-48bc-b3df-8f2301596c35 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9bdaacd597442c2bcde2c555f2cc140', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '49548eb44d62415582af820ab66decee', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:00:16.274318 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2289ab2f-4a09-48bc-b3df-8f2301596c35 service neutron] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:00:16.274598 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2289ab2f-4a09-48bc-b3df-8f2301596c35 service neutron] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:00:16.274872 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2289ab2f-4a09-48bc-b3df-8f2301596c35 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:00:16.284011 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2289ab2f-4a09-48bc-b3df-8f2301596c35 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:00:16.285486 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2289ab2f-4a09-48bc-b3df-8f2301596c35 service neutron] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:00:16.295275 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 1343/2689] 10.222.0.13 () {60 vars in 1365 bytes} [Wed Aug 31 10:00:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 10:00:16.574336 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b2a6b4-d821-4805-b7e1-c31ea912f236 None None] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:00:16.575793 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.fernet_utils [None req-74b2a6b4-d821-4805-b7e1-c31ea912f236 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88114) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:00:16.593339 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b2a6b4-d821-4805-b7e1-c31ea912f236 None None] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:00:16.597929 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca706a6-1eb6-41ef-9db6-c7bbd5ba2c9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:00:16.599362 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca706a6-1eb6-41ef-9db6-c7bbd5ba2c9d admin admin] REQUEST_METHOD: `DELETE` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:00:16.599620 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca706a6-1eb6-41ef-9db6-c7bbd5ba2c9d admin admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:00:16.599948 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca706a6-1eb6-41ef-9db6-c7bbd5ba2c9d admin admin] PATH_INFO: `/v3/projects/4240431c61eb4c238296cd2934e1bb69` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:00:16.601133 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fca706a6-1eb6-41ef-9db6-c7bbd5ba2c9d admin admin] RBAC: Authorizing `identity:delete_project(project_id=4240431c61eb4c238296cd2934e1bb69)` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:00:16.603070 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fca706a6-1eb6-41ef-9db6-c7bbd5ba2c9d admin admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:00:16.662578 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.notifications [None req-fca706a6-1eb6-41ef-9db6-c7bbd5ba2c9d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4240431c61eb4c238296cd2934e1bb69', 'request_id': 'req-fca706a6-1eb6-41ef-9db6-c7bbd5ba2c9d'} {{(pid=88114) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 10:00:16.668260 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.notifications [None req-fca706a6-1eb6-41ef-9db6-c7bbd5ba2c9d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4240431c61eb4c238296cd2934e1bb69', 'request_id': 'req-fca706a6-1eb6-41ef-9db6-c7bbd5ba2c9d'} {{(pid=88114) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 10:00:16.673879 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.notifications [None req-fca706a6-1eb6-41ef-9db6-c7bbd5ba2c9d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4240431c61eb4c238296cd2934e1bb69', 'request_id': 'req-fca706a6-1eb6-41ef-9db6-c7bbd5ba2c9d'} {{(pid=88114) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 10:00:16.687033 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.notifications [None req-fca706a6-1eb6-41ef-9db6-c7bbd5ba2c9d admin admin] The token cache is being invalidate because project 4240431c61eb4c238296cd2934e1bb69 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88114) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 10:00:16.687400 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.notifications [None req-fca706a6-1eb6-41ef-9db6-c7bbd5ba2c9d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88114) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 10:00:16.688082 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.notifications [None req-fca706a6-1eb6-41ef-9db6-c7bbd5ba2c9d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88114) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 10:00:16.689633 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 1347/2690] 10.222.0.13 () {58 vars in 1233 bytes} [Wed Aug 31 10:00:16 2022] DELETE /identity/v3/projects/4240431c61eb4c238296cd2934e1bb69 => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 10:00:16.695585 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7317f1-7c64-49c1-b0af-3841e4e1e81f None None] REQUEST_METHOD: `POST` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:00:16.695878 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7317f1-7c64-49c1-b0af-3841e4e1e81f None None] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:00:16.696152 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7317f1-7c64-49c1-b0af-3841e4e1e81f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:00:16.716828 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.auth.core [None req-ac7317f1-7c64-49c1-b0af-3841e4e1e81f None None] MFA Rules not processed for user `fd836cf590a145df9acf19cf3ae2ac56`. Rule list: `[]` (Enabled: `True`). {{(pid=88113) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 10:00:16.717975 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.fernet_utils [None req-ac7317f1-7c64-49c1-b0af-3841e4e1e81f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:00:16.779678 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 1344/2691] 10.222.0.13 () {58 vars in 958 bytes} [Wed Aug 31 10:00:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 10:00:16.787509 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ecca47b-3b43-40f8-8ec5-eb55a7a94fc4 None None] REQUEST_METHOD: `POST` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:00:16.787788 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ecca47b-3b43-40f8-8ec5-eb55a7a94fc4 None None] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:00:16.788060 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ecca47b-3b43-40f8-8ec5-eb55a7a94fc4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:00:16.807151 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.auth.core [None req-1ecca47b-3b43-40f8-8ec5-eb55a7a94fc4 None None] MFA Rules not processed for user `fd836cf590a145df9acf19cf3ae2ac56`. Rule list: `[]` (Enabled: `True`). {{(pid=88114) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 10:00:16.808340 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.fernet_utils [None req-1ecca47b-3b43-40f8-8ec5-eb55a7a94fc4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88114) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:00:16.816133 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 1348/2692] 10.222.0.13 () {58 vars in 958 bytes} [Wed Aug 31 10:00:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 10:00:16.820878 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac7317f1-7c64-49c1-b0af-3841e4e1e81f None None] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:00:16.830440 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac7317f1-7c64-49c1-b0af-3841e4e1e81f None None] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:00:16.835773 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ab5fa0-b2b5-444a-98df-ec1430d6bd87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:00:16.836362 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ab5fa0-b2b5-444a-98df-ec1430d6bd87 admin admin] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:00:16.836634 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ab5fa0-b2b5-444a-98df-ec1430d6bd87 admin admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:00:16.836899 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ab5fa0-b2b5-444a-98df-ec1430d6bd87 admin admin] PATH_INFO: `/v3/domains` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:00:16.837569 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64ab5fa0-b2b5-444a-98df-ec1430d6bd87 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88113) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:00:16.837864 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64ab5fa0-b2b5-444a-98df-ec1430d6bd87 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:00:16.839450 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64ab5fa0-b2b5-444a-98df-ec1430d6bd87 admin admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:00:16.851156 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 1345/2693] 10.222.0.13 () {58 vars in 1153 bytes} [Wed Aug 31 10:00:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:00:16.855449 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ecca47b-3b43-40f8-8ec5-eb55a7a94fc4 None None] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:00:16.859824 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:00:16.866998 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ecca47b-3b43-40f8-8ec5-eb55a7a94fc4 None None] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:00:16.873258 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a4e2d3f-4d9f-4754-86c2-f426e2aace54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:00:16.873882 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4e2d3f-4d9f-4754-86c2-f426e2aace54 admin admin] REQUEST_METHOD: `POST` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:00:16.874157 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4e2d3f-4d9f-4754-86c2-f426e2aace54 admin admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:00:16.874425 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4e2d3f-4d9f-4754-86c2-f426e2aace54 admin admin] PATH_INFO: `/v3/projects` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:00:16.875059 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a4e2d3f-4d9f-4754-86c2-f426e2aace54 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:00:16.876980 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a4e2d3f-4d9f-4754-86c2-f426e2aace54 admin admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:00:16.904931 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 1349/2694] 10.222.0.13 () {60 vars in 1153 bytes} [Wed Aug 31 10:00:16 2022] POST /identity/v3/projects => generated 383 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 10:00:16.909130 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ab5fa0-b2b5-444a-98df-ec1430d6bd87 admin admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:00:16.919952 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ab5fa0-b2b5-444a-98df-ec1430d6bd87 admin admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:00:17.103764 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e6b7231-af4d-44a0-a3e3-39a040cb6ee3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:00:17.104553 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6b7231-af4d-44a0-a3e3-39a040cb6ee3 admin admin] REQUEST_METHOD: `POST` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:00:17.104658 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6b7231-af4d-44a0-a3e3-39a040cb6ee3 admin admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:00:17.104882 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6b7231-af4d-44a0-a3e3-39a040cb6ee3 admin admin] PATH_INFO: `/v3/users` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:00:17.105545 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e6b7231-af4d-44a0-a3e3-39a040cb6ee3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:00:17.107467 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e6b7231-af4d-44a0-a3e3-39a040cb6ee3 admin admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:00:17.131773 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 1346/2695] 10.222.0.13 () {60 vars in 1144 bytes} [Wed Aug 31 10:00:16 2022] POST /identity/v3/users => generated 292 bytes in 223 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 10:00:17.136726 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a4e2d3f-4d9f-4754-86c2-f426e2aace54 admin admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:00:17.147255 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a4e2d3f-4d9f-4754-86c2-f426e2aace54 admin admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:00:17.151715 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e54b4e-003b-4c3a-bf89-948a72192345 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:00:17.152305 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e54b4e-003b-4c3a-bf89-948a72192345 admin admin] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:00:17.152578 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e54b4e-003b-4c3a-bf89-948a72192345 admin admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:00:17.152841 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e54b4e-003b-4c3a-bf89-948a72192345 admin admin] PATH_INFO: `/v3/roles` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:00:17.153517 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4e54b4e-003b-4c3a-bf89-948a72192345 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:00:17.155074 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4e54b4e-003b-4c3a-bf89-948a72192345 admin admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:00:17.165182 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 1350/2696] 10.222.0.13 () {58 vars in 1122 bytes} [Wed Aug 31 10:00:17 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 10:00:17.169316 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e6b7231-af4d-44a0-a3e3-39a040cb6ee3 admin admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:00:17.182264 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e6b7231-af4d-44a0-a3e3-39a040cb6ee3 admin admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:00:17.187302 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d06dda2-a9d2-42bc-b8a5-4af88007cc8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:00:17.187900 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d06dda2-a9d2-42bc-b8a5-4af88007cc8e admin admin] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:00:17.188183 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d06dda2-a9d2-42bc-b8a5-4af88007cc8e admin admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:00:17.188452 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d06dda2-a9d2-42bc-b8a5-4af88007cc8e admin admin] PATH_INFO: `/v3/roles` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:00:17.189107 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d06dda2-a9d2-42bc-b8a5-4af88007cc8e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:00:17.190706 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d06dda2-a9d2-42bc-b8a5-4af88007cc8e admin admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:00:17.197177 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:INSERT=4,keystone:DELETE=1,keystone:SELECT=95 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:00:17.202695 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 1347/2697] 10.222.0.13 () {58 vars in 1122 bytes} [Wed Aug 31 10:00:17 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 10:00:17.206757 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e54b4e-003b-4c3a-bf89-948a72192345 admin admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:00:17.217049 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e54b4e-003b-4c3a-bf89-948a72192345 admin admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:00:17.221571 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-304932df-8e25-46a5-8057-226340bb094d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:00:17.222248 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304932df-8e25-46a5-8057-226340bb094d admin admin] REQUEST_METHOD: `PUT` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:00:17.222522 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304932df-8e25-46a5-8057-226340bb094d admin admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:00:17.222790 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304932df-8e25-46a5-8057-226340bb094d admin admin] PATH_INFO: `/v3/projects/142cb7349c874330be15b30d765aa589/users/1c1304d413a64725a6c4e95d4bb07ecc/roles/6c99f5d9f1f243eaa134d8540f21095d` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:00:17.237814 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-304932df-8e25-46a5-8057-226340bb094d admin admin] RBAC: Authorizing `identity:create_grant(project_id=142cb7349c874330be15b30d765aa589, user_id=1c1304d413a64725a6c4e95d4bb07ecc, role_id=6c99f5d9f1f243eaa134d8540f21095d)` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:00:17.241245 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-304932df-8e25-46a5-8057-226340bb094d admin admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:00:17.254038 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 1351/2698] 10.222.0.13 () {60 vars in 1483 bytes} [Wed Aug 31 10:00:17 2022] PUT /identity/v3/projects/142cb7349c874330be15b30d765aa589/users/1c1304d413a64725a6c4e95d4bb07ecc/roles/6c99f5d9f1f243eaa134d8540f21095d => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 10:00:17.320571 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06083d80-72ff-4119-9ceb-ec46ad5eeac8 None None] REQUEST_METHOD: `POST` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:00:17.320858 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06083d80-72ff-4119-9ceb-ec46ad5eeac8 None None] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:00:17.321164 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06083d80-72ff-4119-9ceb-ec46ad5eeac8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:00:17.350252 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.auth.core [None req-06083d80-72ff-4119-9ceb-ec46ad5eeac8 None None] MFA Rules not processed for user `1c1304d413a64725a6c4e95d4bb07ecc`. Rule list: `[]` (Enabled: `True`). {{(pid=88113) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 10:00:17.351462 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.fernet_utils [None req-06083d80-72ff-4119-9ceb-ec46ad5eeac8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:00:17.420902 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 1348/2699] 10.222.0.13 () {58 vars in 958 bytes} [Wed Aug 31 10:00:17 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 10:00:17.425503 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-304932df-8e25-46a5-8057-226340bb094d admin admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:00:17.488565 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-304932df-8e25-46a5-8057-226340bb094d admin admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:00:17.494364 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9196ee0-b0f7-416d-aea2-d5a4cbcb4b06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:00:17.494958 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9196ee0-b0f7-416d-aea2-d5a4cbcb4b06 admin admin] REQUEST_METHOD: `POST` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:00:17.495233 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9196ee0-b0f7-416d-aea2-d5a4cbcb4b06 admin admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:00:17.495502 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9196ee0-b0f7-416d-aea2-d5a4cbcb4b06 admin admin] PATH_INFO: `/v3/projects` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:00:17.496124 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9196ee0-b0f7-416d-aea2-d5a4cbcb4b06 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:00:17.498073 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9196ee0-b0f7-416d-aea2-d5a4cbcb4b06 admin admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:00:17.524206 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 1352/2700] 10.222.0.13 () {60 vars in 1153 bytes} [Wed Aug 31 10:00:17 2022] POST /identity/v3/projects => generated 383 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 10:00:17.528529 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06083d80-72ff-4119-9ceb-ec46ad5eeac8 None None] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:00:17.539939 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06083d80-72ff-4119-9ceb-ec46ad5eeac8 None None] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:00:17.594850 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eabd8ed-1c06-4001-980f-113154d0884e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:00:17.595484 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eabd8ed-1c06-4001-980f-113154d0884e admin admin] REQUEST_METHOD: `POST` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:00:17.595779 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eabd8ed-1c06-4001-980f-113154d0884e admin admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:00:17.596043 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eabd8ed-1c06-4001-980f-113154d0884e admin admin] PATH_INFO: `/v3/users` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:00:17.596666 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eabd8ed-1c06-4001-980f-113154d0884e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:00:17.599226 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eabd8ed-1c06-4001-980f-113154d0884e admin admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:00:17.621495 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 1349/2701] 10.222.0.13 () {60 vars in 1144 bytes} [Wed Aug 31 10:00:17 2022] POST /identity/v3/users => generated 292 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 10:00:17.625456 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9196ee0-b0f7-416d-aea2-d5a4cbcb4b06 admin admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:00:17.635980 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9196ee0-b0f7-416d-aea2-d5a4cbcb4b06 admin admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:00:17.640746 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f6bfa4-0f47-427b-856c-3d20bb7361d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:00:17.641349 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f6bfa4-0f47-427b-856c-3d20bb7361d2 admin admin] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:00:17.641663 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f6bfa4-0f47-427b-856c-3d20bb7361d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:00:17.641931 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f6bfa4-0f47-427b-856c-3d20bb7361d2 admin admin] PATH_INFO: `/v3/roles` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:00:17.642585 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26f6bfa4-0f47-427b-856c-3d20bb7361d2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:00:17.644156 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26f6bfa4-0f47-427b-856c-3d20bb7361d2 admin admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:00:17.654934 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 1353/2702] 10.222.0.13 () {58 vars in 1122 bytes} [Wed Aug 31 10:00:17 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 10:00:17.659369 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eabd8ed-1c06-4001-980f-113154d0884e admin admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:00:17.670105 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eabd8ed-1c06-4001-980f-113154d0884e admin admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:00:17.674725 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a08a2013-6002-46e5-9e34-8a9e675095cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:00:17.675288 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08a2013-6002-46e5-9e34-8a9e675095cc admin admin] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:00:17.675563 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08a2013-6002-46e5-9e34-8a9e675095cc admin admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:00:17.675828 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08a2013-6002-46e5-9e34-8a9e675095cc admin admin] PATH_INFO: `/v3/roles` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:00:17.677117 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a08a2013-6002-46e5-9e34-8a9e675095cc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:00:17.678751 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a08a2013-6002-46e5-9e34-8a9e675095cc admin admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:00:17.688841 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 1350/2703] 10.222.0.13 () {58 vars in 1122 bytes} [Wed Aug 31 10:00:17 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 10:00:17.693212 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f6bfa4-0f47-427b-856c-3d20bb7361d2 admin admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:00:17.703173 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f6bfa4-0f47-427b-856c-3d20bb7361d2 admin admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:00:17.708277 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c94df7db-dbe0-4414-8da7-f99824c0ddd8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:00:17.708915 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94df7db-dbe0-4414-8da7-f99824c0ddd8 admin admin] REQUEST_METHOD: `PUT` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:00:17.709190 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94df7db-dbe0-4414-8da7-f99824c0ddd8 admin admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:00:17.709493 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94df7db-dbe0-4414-8da7-f99824c0ddd8 admin admin] PATH_INFO: `/v3/projects/a291c39ed1204cc9ac3e3da45151dd99/users/faa896c3fe8945fda1833285ae5d6954/roles/6c99f5d9f1f243eaa134d8540f21095d` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:00:17.724290 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c94df7db-dbe0-4414-8da7-f99824c0ddd8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a291c39ed1204cc9ac3e3da45151dd99, user_id=faa896c3fe8945fda1833285ae5d6954, role_id=6c99f5d9f1f243eaa134d8540f21095d)` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:00:17.726975 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c94df7db-dbe0-4414-8da7-f99824c0ddd8 admin admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:00:17.740190 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 1354/2704] 10.222.0.13 () {60 vars in 1483 bytes} [Wed Aug 31 10:00:17 2022] PUT /identity/v3/projects/a291c39ed1204cc9ac3e3da45151dd99/users/faa896c3fe8945fda1833285ae5d6954/roles/6c99f5d9f1f243eaa134d8540f21095d => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 10:00:17.750617 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6118231-06bc-44dd-a03a-0e6f1bea39b8 None None] REQUEST_METHOD: `POST` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:00:17.750991 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6118231-06bc-44dd-a03a-0e6f1bea39b8 None None] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:00:17.751264 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6118231-06bc-44dd-a03a-0e6f1bea39b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:00:17.780321 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.auth.core [None req-e6118231-06bc-44dd-a03a-0e6f1bea39b8 None None] MFA Rules not processed for user `faa896c3fe8945fda1833285ae5d6954`. Rule list: `[]` (Enabled: `True`). {{(pid=88113) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 10:00:17.781609 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.fernet_utils [None req-e6118231-06bc-44dd-a03a-0e6f1bea39b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:00:17.840320 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:00:17.846836 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 1351/2705] 10.222.0.13 () {58 vars in 958 bytes} [Wed Aug 31 10:00:17 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 10:00:17.851945 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c94df7db-dbe0-4414-8da7-f99824c0ddd8 admin admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:00:17.909539 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:00:17.916118 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c94df7db-dbe0-4414-8da7-f99824c0ddd8 admin admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:00:17.921129 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-000c9b5c-28af-4ee1-b890-c9baacf8daa4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:00:17.921756 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000c9b5c-28af-4ee1-b890-c9baacf8daa4 admin admin] REQUEST_METHOD: `POST` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:00:17.922027 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000c9b5c-28af-4ee1-b890-c9baacf8daa4 admin admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:00:17.922323 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000c9b5c-28af-4ee1-b890-c9baacf8daa4 admin admin] PATH_INFO: `/v3/projects` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:00:17.922983 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-000c9b5c-28af-4ee1-b890-c9baacf8daa4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:00:17.924864 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-000c9b5c-28af-4ee1-b890-c9baacf8daa4 admin admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:00:17.951386 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 1355/2706] 10.222.0.13 () {60 vars in 1153 bytes} [Wed Aug 31 10:00:17 2022] POST /identity/v3/projects => generated 385 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 10:00:17.955498 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6118231-06bc-44dd-a03a-0e6f1bea39b8 None None] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:00:17.967086 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6118231-06bc-44dd-a03a-0e6f1bea39b8 None None] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:00:18.026106 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dad3029-5d39-47dd-84e2-815972521cc0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:00:18.026638 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dad3029-5d39-47dd-84e2-815972521cc0 admin admin] REQUEST_METHOD: `POST` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:00:18.026912 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dad3029-5d39-47dd-84e2-815972521cc0 admin admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:00:18.027178 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dad3029-5d39-47dd-84e2-815972521cc0 admin admin] PATH_INFO: `/v3/users` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:00:18.027848 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dad3029-5d39-47dd-84e2-815972521cc0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:00:18.029801 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dad3029-5d39-47dd-84e2-815972521cc0 admin admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:00:18.052586 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 1352/2707] 10.222.0.13 () {60 vars in 1144 bytes} [Wed Aug 31 10:00:17 2022] POST /identity/v3/users => generated 299 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 10:00:18.057595 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-000c9b5c-28af-4ee1-b890-c9baacf8daa4 admin admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:00:18.068820 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-000c9b5c-28af-4ee1-b890-c9baacf8daa4 admin admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:00:18.073503 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed694d8-f7a5-48f6-805b-e9c11a599fd7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:00:18.074105 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed694d8-f7a5-48f6-805b-e9c11a599fd7 admin admin] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:00:18.074381 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed694d8-f7a5-48f6-805b-e9c11a599fd7 admin admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:00:18.074645 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed694d8-f7a5-48f6-805b-e9c11a599fd7 admin admin] PATH_INFO: `/v3/roles` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:00:18.075283 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ed694d8-f7a5-48f6-805b-e9c11a599fd7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:00:18.076854 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ed694d8-f7a5-48f6-805b-e9c11a599fd7 admin admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:00:18.088686 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 1356/2708] 10.222.0.13 () {58 vars in 1122 bytes} [Wed Aug 31 10:00:18 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 10:00:18.092885 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dad3029-5d39-47dd-84e2-815972521cc0 admin admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:00:18.103622 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dad3029-5d39-47dd-84e2-815972521cc0 admin admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:00:18.108204 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26a32da3-4efe-476e-8fa4-f413735ddd29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:00:18.109147 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a32da3-4efe-476e-8fa4-f413735ddd29 admin admin] REQUEST_METHOD: `PUT` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:00:18.109457 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a32da3-4efe-476e-8fa4-f413735ddd29 admin admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:00:18.109727 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a32da3-4efe-476e-8fa4-f413735ddd29 admin admin] PATH_INFO: `/v3/projects/0c466f3a96df4d0dbd5e4b3c28805f52/users/ca02b1b36d614340841faa4715c7b526/roles/5d4ee63a3add47fc9a6b8b16d39e0a1f` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:00:18.124426 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26a32da3-4efe-476e-8fa4-f413735ddd29 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0c466f3a96df4d0dbd5e4b3c28805f52, user_id=ca02b1b36d614340841faa4715c7b526, role_id=5d4ee63a3add47fc9a6b8b16d39e0a1f)` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:00:18.127087 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26a32da3-4efe-476e-8fa4-f413735ddd29 admin admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:00:18.140445 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 1353/2709] 10.222.0.13 () {60 vars in 1483 bytes} [Wed Aug 31 10:00:18 2022] PUT /identity/v3/projects/0c466f3a96df4d0dbd5e4b3c28805f52/users/ca02b1b36d614340841faa4715c7b526/roles/5d4ee63a3add47fc9a6b8b16d39e0a1f => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 10:00:18.144364 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed694d8-f7a5-48f6-805b-e9c11a599fd7 admin admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:00:18.155188 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed694d8-f7a5-48f6-805b-e9c11a599fd7 admin admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:00:18.213461 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d9c826e-4d08-4c73-8521-52583cd6ce82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:00:18.214193 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9c826e-4d08-4c73-8521-52583cd6ce82 admin admin] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:00:18.214316 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9c826e-4d08-4c73-8521-52583cd6ce82 admin admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:00:18.214579 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9c826e-4d08-4c73-8521-52583cd6ce82 admin admin] PATH_INFO: `/v3/roles` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:00:18.215210 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d9c826e-4d08-4c73-8521-52583cd6ce82 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:00:18.216761 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d9c826e-4d08-4c73-8521-52583cd6ce82 admin admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:00:18.226513 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 1357/2710] 10.222.0.13 () {58 vars in 1122 bytes} [Wed Aug 31 10:00:18 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 10:00:18.230815 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26a32da3-4efe-476e-8fa4-f413735ddd29 admin admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:00:18.241626 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26a32da3-4efe-476e-8fa4-f413735ddd29 admin admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:00:18.246244 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc82d27d-499f-447a-8b7c-990006300b04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:00:18.246884 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc82d27d-499f-447a-8b7c-990006300b04 admin admin] REQUEST_METHOD: `PUT` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:00:18.247161 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc82d27d-499f-447a-8b7c-990006300b04 admin admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:00:18.247431 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc82d27d-499f-447a-8b7c-990006300b04 admin admin] PATH_INFO: `/v3/projects/0c466f3a96df4d0dbd5e4b3c28805f52/users/ca02b1b36d614340841faa4715c7b526/roles/5d4ee63a3add47fc9a6b8b16d39e0a1f` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:00:18.249362 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc82d27d-499f-447a-8b7c-990006300b04 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0c466f3a96df4d0dbd5e4b3c28805f52, user_id=ca02b1b36d614340841faa4715c7b526, role_id=5d4ee63a3add47fc9a6b8b16d39e0a1f)` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:00:18.252011 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc82d27d-499f-447a-8b7c-990006300b04 admin admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:00:18.263605 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 1354/2711] 10.222.0.13 () {60 vars in 1483 bytes} [Wed Aug 31 10:00:18 2022] PUT /identity/v3/projects/0c466f3a96df4d0dbd5e4b3c28805f52/users/ca02b1b36d614340841faa4715c7b526/roles/5d4ee63a3add47fc9a6b8b16d39e0a1f => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 10:00:18.274718 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b662c0d7-dbc6-44ef-996f-b6ec70c5d27e None None] REQUEST_METHOD: `POST` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:00:18.274999 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b662c0d7-dbc6-44ef-996f-b6ec70c5d27e None None] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:00:18.275300 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b662c0d7-dbc6-44ef-996f-b6ec70c5d27e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:00:18.302602 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.auth.core [None req-b662c0d7-dbc6-44ef-996f-b6ec70c5d27e None None] MFA Rules not processed for user `ca02b1b36d614340841faa4715c7b526`. Rule list: `[]` (Enabled: `True`). {{(pid=88114) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 10:00:18.303966 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.fernet_utils [None req-b662c0d7-dbc6-44ef-996f-b6ec70c5d27e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88114) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:00:18.373255 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 1358/2712] 10.222.0.13 () {58 vars in 958 bytes} [Wed Aug 31 10:00:18 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 10:00:18.384964 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc82d27d-499f-447a-8b7c-990006300b04 admin admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:00:18.386682 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.fernet_utils [None req-dc82d27d-499f-447a-8b7c-990006300b04 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:00:18.446647 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc82d27d-499f-447a-8b7c-990006300b04 admin admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:00:18.450342 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-893ed90c-7757-4047-bcc8-8d8ebbdd73d1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d52b69c8e94441dacfcff944326f169', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '49548eb44d62415582af820ab66decee', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:00:18.450947 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893ed90c-7757-4047-bcc8-8d8ebbdd73d1 service cinder] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:00:18.451224 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893ed90c-7757-4047-bcc8-8d8ebbdd73d1 service cinder] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:00:18.451495 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893ed90c-7757-4047-bcc8-8d8ebbdd73d1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:00:18.510118 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-893ed90c-7757-4047-bcc8-8d8ebbdd73d1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:00:18.511550 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-893ed90c-7757-4047-bcc8-8d8ebbdd73d1 service cinder] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:00:18.521278 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 1355/2713] 10.222.0.13 () {60 vars in 1365 bytes} [Wed Aug 31 10:00:18 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 10:00:20.040749 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b662c0d7-dbc6-44ef-996f-b6ec70c5d27e None None] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:00:20.042125 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.fernet_utils [None req-b662c0d7-dbc6-44ef-996f-b6ec70c5d27e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88114) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:00:20.055243 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b662c0d7-dbc6-44ef-996f-b6ec70c5d27e None None] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:00:20.059240 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f72b8d-9f85-4682-a836-1f08dfc49aac service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d52b69c8e94441dacfcff944326f169', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '49548eb44d62415582af820ab66decee', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:00:20.060001 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f72b8d-9f85-4682-a836-1f08dfc49aac service cinder] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:00:20.060378 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f72b8d-9f85-4682-a836-1f08dfc49aac service cinder] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:00:20.060751 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f72b8d-9f85-4682-a836-1f08dfc49aac service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:00:20.110600 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:00:20.122277 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66f72b8d-9f85-4682-a836-1f08dfc49aac service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:00:20.123699 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66f72b8d-9f85-4682-a836-1f08dfc49aac service cinder] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:00:20.133523 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 1359/2714] 10.222.0.13 () {60 vars in 1365 bytes} [Wed Aug 31 10:00:20 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 10:00:20.679068 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-893ed90c-7757-4047-bcc8-8d8ebbdd73d1 service cinder] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:00:20.684301 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:00:20.688066 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-893ed90c-7757-4047-bcc8-8d8ebbdd73d1 service cinder] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:00:20.693173 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb4a34c-3fbe-45ce-b330-dd3ccb4647d3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d52b69c8e94441dacfcff944326f169', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '49548eb44d62415582af820ab66decee', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:00:20.693799 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb4a34c-3fbe-45ce-b330-dd3ccb4647d3 service cinder] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:00:20.694077 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb4a34c-3fbe-45ce-b330-dd3ccb4647d3 service cinder] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:00:20.694344 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb4a34c-3fbe-45ce-b330-dd3ccb4647d3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:00:20.703688 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efb4a34c-3fbe-45ce-b330-dd3ccb4647d3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:00:20.705107 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efb4a34c-3fbe-45ce-b330-dd3ccb4647d3 service cinder] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:00:20.715322 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 1356/2715] 10.222.0.13 () {60 vars in 1365 bytes} [Wed Aug 31 10:00:20 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 10:00:24.600380 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f72b8d-9f85-4682-a836-1f08dfc49aac service cinder] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:00:24.665901 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f72b8d-9f85-4682-a836-1f08dfc49aac service cinder] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:00:24.670384 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c75de4b-e8ed-4117-8907-1cf3f49356f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:00:24.671002 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c75de4b-e8ed-4117-8907-1cf3f49356f6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:00:24.671279 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c75de4b-e8ed-4117-8907-1cf3f49356f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:00:24.671551 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c75de4b-e8ed-4117-8907-1cf3f49356f6 admin admin] PATH_INFO: `/v3/users/1c1304d413a64725a6c4e95d4bb07ecc` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:00:24.672658 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c75de4b-e8ed-4117-8907-1cf3f49356f6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1c1304d413a64725a6c4e95d4bb07ecc)` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:00:24.674603 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c75de4b-e8ed-4117-8907-1cf3f49356f6 admin admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:00:24.719763 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.notifications [None req-5c75de4b-e8ed-4117-8907-1cf3f49356f6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1c1304d413a64725a6c4e95d4bb07ecc'} {{(pid=88114) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 10:00:24.726654 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.notifications [None req-5c75de4b-e8ed-4117-8907-1cf3f49356f6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1c1304d413a64725a6c4e95d4bb07ecc'} {{(pid=88114) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 10:00:24.734201 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.notifications [None req-5c75de4b-e8ed-4117-8907-1cf3f49356f6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1c1304d413a64725a6c4e95d4bb07ecc'} {{(pid=88114) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 10:00:24.734996 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.notifications [None req-5c75de4b-e8ed-4117-8907-1cf3f49356f6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1c1304d413a64725a6c4e95d4bb07ecc'} {{(pid=88114) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 10:00:24.747867 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.notifications [None req-5c75de4b-e8ed-4117-8907-1cf3f49356f6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1c1304d413a64725a6c4e95d4bb07ecc'} {{(pid=88114) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 10:00:24.753683 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 1360/2716] 10.222.0.13 () {58 vars in 1224 bytes} [Wed Aug 31 10:00:24 2022] DELETE /identity/v3/users/1c1304d413a64725a6c4e95d4bb07ecc => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 10:00:24.759301 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9694f10-4036-4bb0-b5a2-6f5b124f350a None None] REQUEST_METHOD: `POST` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:00:24.759793 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9694f10-4036-4bb0-b5a2-6f5b124f350a None None] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:00:24.759915 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9694f10-4036-4bb0-b5a2-6f5b124f350a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:00:24.779155 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.auth.core [None req-b9694f10-4036-4bb0-b5a2-6f5b124f350a None None] MFA Rules not processed for user `fd836cf590a145df9acf19cf3ae2ac56`. Rule list: `[]` (Enabled: `True`). {{(pid=88113) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 10:00:24.780344 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.fernet_utils [None req-b9694f10-4036-4bb0-b5a2-6f5b124f350a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:00:24.839396 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 1357/2717] 10.222.0.13 () {58 vars in 958 bytes} [Wed Aug 31 10:00:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 10:00:24.849147 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c75de4b-e8ed-4117-8907-1cf3f49356f6 admin admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:00:24.851217 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.fernet_utils [None req-5c75de4b-e8ed-4117-8907-1cf3f49356f6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88114) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:00:24.907612 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c75de4b-e8ed-4117-8907-1cf3f49356f6 admin admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:00:24.917226 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a7838a-6b7a-4a82-be27-930138aa3777 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9bdaacd597442c2bcde2c555f2cc140', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '49548eb44d62415582af820ab66decee', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:00:24.917869 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a7838a-6b7a-4a82-be27-930138aa3777 service neutron] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:00:24.918146 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a7838a-6b7a-4a82-be27-930138aa3777 service neutron] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:00:24.918413 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a7838a-6b7a-4a82-be27-930138aa3777 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:00:24.928789 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96a7838a-6b7a-4a82-be27-930138aa3777 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:00:24.930204 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96a7838a-6b7a-4a82-be27-930138aa3777 service neutron] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:00:24.939969 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 1361/2718] 10.222.0.13 () {60 vars in 1365 bytes} [Wed Aug 31 10:00:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 10:00:25.216775 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9694f10-4036-4bb0-b5a2-6f5b124f350a None None] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:00:25.218231 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.fernet_utils [None req-b9694f10-4036-4bb0-b5a2-6f5b124f350a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:00:25.231114 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9694f10-4036-4bb0-b5a2-6f5b124f350a None None] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:00:25.235770 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f3d8b65-293f-4c37-865d-264df1274223 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:00:25.236407 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3d8b65-293f-4c37-865d-264df1274223 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:00:25.236681 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3d8b65-293f-4c37-865d-264df1274223 admin admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:00:25.236952 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3d8b65-293f-4c37-865d-264df1274223 admin admin] PATH_INFO: `/v3/projects/142cb7349c874330be15b30d765aa589` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:00:25.238086 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f3d8b65-293f-4c37-865d-264df1274223 admin admin] RBAC: Authorizing `identity:delete_project(project_id=142cb7349c874330be15b30d765aa589)` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:00:25.240335 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f3d8b65-293f-4c37-865d-264df1274223 admin admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:00:25.286608 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.notifications [None req-8f3d8b65-293f-4c37-865d-264df1274223 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '142cb7349c874330be15b30d765aa589', 'request_id': 'req-8f3d8b65-293f-4c37-865d-264df1274223'} {{(pid=88113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 10:00:25.293157 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.notifications [None req-8f3d8b65-293f-4c37-865d-264df1274223 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '142cb7349c874330be15b30d765aa589', 'request_id': 'req-8f3d8b65-293f-4c37-865d-264df1274223'} {{(pid=88113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 10:00:25.300611 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.notifications [None req-8f3d8b65-293f-4c37-865d-264df1274223 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '142cb7349c874330be15b30d765aa589', 'request_id': 'req-8f3d8b65-293f-4c37-865d-264df1274223'} {{(pid=88113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 10:00:25.312017 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.notifications [None req-8f3d8b65-293f-4c37-865d-264df1274223 admin admin] The token cache is being invalidate because project 142cb7349c874330be15b30d765aa589 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88113) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 10:00:25.312299 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.notifications [None req-8f3d8b65-293f-4c37-865d-264df1274223 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 10:00:25.312939 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.notifications [None req-8f3d8b65-293f-4c37-865d-264df1274223 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 10:00:25.314440 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 1358/2719] 10.222.0.13 () {58 vars in 1233 bytes} [Wed Aug 31 10:00:25 2022] DELETE /identity/v3/projects/142cb7349c874330be15b30d765aa589 => generated 0 bytes in 98 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 10:00:25.318505 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a7838a-6b7a-4a82-be27-930138aa3777 service neutron] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:00:25.328056 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a7838a-6b7a-4a82-be27-930138aa3777 service neutron] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:00:25.336500 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:00:25.392975 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3199fad-fa23-46dd-859e-de76dcc505d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:00:25.393614 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3199fad-fa23-46dd-859e-de76dcc505d1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:00:25.393898 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3199fad-fa23-46dd-859e-de76dcc505d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:00:25.394170 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3199fad-fa23-46dd-859e-de76dcc505d1 admin admin] PATH_INFO: `/v3/users/faa896c3fe8945fda1833285ae5d6954` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:00:25.395374 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3199fad-fa23-46dd-859e-de76dcc505d1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=faa896c3fe8945fda1833285ae5d6954)` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:00:25.397865 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3199fad-fa23-46dd-859e-de76dcc505d1 admin admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:00:25.459104 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.notifications [None req-a3199fad-fa23-46dd-859e-de76dcc505d1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'faa896c3fe8945fda1833285ae5d6954'} {{(pid=88114) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 10:00:25.467533 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.notifications [None req-a3199fad-fa23-46dd-859e-de76dcc505d1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'faa896c3fe8945fda1833285ae5d6954'} {{(pid=88114) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 10:00:25.474682 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.notifications [None req-a3199fad-fa23-46dd-859e-de76dcc505d1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'faa896c3fe8945fda1833285ae5d6954'} {{(pid=88114) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 10:00:25.475315 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.notifications [None req-a3199fad-fa23-46dd-859e-de76dcc505d1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'faa896c3fe8945fda1833285ae5d6954'} {{(pid=88114) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 10:00:25.488935 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.notifications [None req-a3199fad-fa23-46dd-859e-de76dcc505d1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'faa896c3fe8945fda1833285ae5d6954'} {{(pid=88114) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 10:00:25.496438 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 1362/2720] 10.222.0.13 () {58 vars in 1224 bytes} [Wed Aug 31 10:00:25 2022] DELETE /identity/v3/users/faa896c3fe8945fda1833285ae5d6954 => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 10:00:25.771108 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f3d8b65-293f-4c37-865d-264df1274223 admin admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:00:25.773449 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.fernet_utils [None req-8f3d8b65-293f-4c37-865d-264df1274223 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:00:25.791926 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f3d8b65-293f-4c37-865d-264df1274223 admin admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:00:25.855121 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e441934-a6d8-422d-95b5-e96f87fa2704 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:00:25.855766 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e441934-a6d8-422d-95b5-e96f87fa2704 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:00:25.856045 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e441934-a6d8-422d-95b5-e96f87fa2704 admin admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:00:25.856318 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e441934-a6d8-422d-95b5-e96f87fa2704 admin admin] PATH_INFO: `/v3/projects/a291c39ed1204cc9ac3e3da45151dd99` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:00:25.857479 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e441934-a6d8-422d-95b5-e96f87fa2704 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a291c39ed1204cc9ac3e3da45151dd99)` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:00:25.859394 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e441934-a6d8-422d-95b5-e96f87fa2704 admin admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:00:25.876294 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:00:25.907793 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.notifications [None req-0e441934-a6d8-422d-95b5-e96f87fa2704 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a291c39ed1204cc9ac3e3da45151dd99', 'request_id': 'req-0e441934-a6d8-422d-95b5-e96f87fa2704'} {{(pid=88113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 10:00:25.913704 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.notifications [None req-0e441934-a6d8-422d-95b5-e96f87fa2704 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a291c39ed1204cc9ac3e3da45151dd99', 'request_id': 'req-0e441934-a6d8-422d-95b5-e96f87fa2704'} {{(pid=88113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 10:00:25.920934 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.notifications [None req-0e441934-a6d8-422d-95b5-e96f87fa2704 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a291c39ed1204cc9ac3e3da45151dd99', 'request_id': 'req-0e441934-a6d8-422d-95b5-e96f87fa2704'} {{(pid=88113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 10:00:25.932772 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.notifications [None req-0e441934-a6d8-422d-95b5-e96f87fa2704 admin admin] The token cache is being invalidate because project a291c39ed1204cc9ac3e3da45151dd99 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88113) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 10:00:25.933062 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.notifications [None req-0e441934-a6d8-422d-95b5-e96f87fa2704 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 10:00:25.933716 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.notifications [None req-0e441934-a6d8-422d-95b5-e96f87fa2704 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 10:00:25.935206 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 1359/2721] 10.222.0.13 () {58 vars in 1233 bytes} [Wed Aug 31 10:00:25 2022] DELETE /identity/v3/projects/a291c39ed1204cc9ac3e3da45151dd99 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 10:00:25.939328 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3199fad-fa23-46dd-859e-de76dcc505d1 admin admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:00:25.940820 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.fernet_utils [None req-a3199fad-fa23-46dd-859e-de76dcc505d1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88114) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:00:25.953797 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3199fad-fa23-46dd-859e-de76dcc505d1 admin admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:00:26.011522 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d30d02-2d6b-4922-9ced-393bdd0ca220 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:00:26.012143 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d30d02-2d6b-4922-9ced-393bdd0ca220 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:00:26.012416 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d30d02-2d6b-4922-9ced-393bdd0ca220 admin admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:00:26.012685 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d30d02-2d6b-4922-9ced-393bdd0ca220 admin admin] PATH_INFO: `/v3/users/ca02b1b36d614340841faa4715c7b526` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:00:26.013811 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01d30d02-2d6b-4922-9ced-393bdd0ca220 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ca02b1b36d614340841faa4715c7b526)` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:00:26.015693 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01d30d02-2d6b-4922-9ced-393bdd0ca220 admin admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:00:26.060908 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:00:26.076137 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.notifications [None req-01d30d02-2d6b-4922-9ced-393bdd0ca220 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ca02b1b36d614340841faa4715c7b526'} {{(pid=88114) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 10:00:26.084155 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.notifications [None req-01d30d02-2d6b-4922-9ced-393bdd0ca220 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ca02b1b36d614340841faa4715c7b526'} {{(pid=88114) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 10:00:26.092806 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.notifications [None req-01d30d02-2d6b-4922-9ced-393bdd0ca220 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ca02b1b36d614340841faa4715c7b526'} {{(pid=88114) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 10:00:26.093448 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.notifications [None req-01d30d02-2d6b-4922-9ced-393bdd0ca220 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ca02b1b36d614340841faa4715c7b526'} {{(pid=88114) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 10:00:26.107234 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.notifications [None req-01d30d02-2d6b-4922-9ced-393bdd0ca220 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ca02b1b36d614340841faa4715c7b526'} {{(pid=88114) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 10:00:26.113524 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 1363/2722] 10.222.0.13 () {58 vars in 1224 bytes} [Wed Aug 31 10:00:25 2022] DELETE /identity/v3/users/ca02b1b36d614340841faa4715c7b526 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 10:00:26.386996 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e441934-a6d8-422d-95b5-e96f87fa2704 admin admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:00:26.388481 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.fernet_utils [None req-0e441934-a6d8-422d-95b5-e96f87fa2704 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:00:26.405870 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e441934-a6d8-422d-95b5-e96f87fa2704 admin admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:00:26.463734 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74766d63-45a9-41b2-8550-80ff958746ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:00:26.464275 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74766d63-45a9-41b2-8550-80ff958746ca admin admin] REQUEST_METHOD: `DELETE` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:00:26.464567 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74766d63-45a9-41b2-8550-80ff958746ca admin admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:00:26.464814 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74766d63-45a9-41b2-8550-80ff958746ca admin admin] PATH_INFO: `/v3/projects/0c466f3a96df4d0dbd5e4b3c28805f52` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:00:26.465942 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74766d63-45a9-41b2-8550-80ff958746ca admin admin] RBAC: Authorizing `identity:delete_project(project_id=0c466f3a96df4d0dbd5e4b3c28805f52)` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:00:26.467830 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74766d63-45a9-41b2-8550-80ff958746ca admin admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:00:26.518399 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.notifications [None req-74766d63-45a9-41b2-8550-80ff958746ca admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0c466f3a96df4d0dbd5e4b3c28805f52', 'request_id': 'req-74766d63-45a9-41b2-8550-80ff958746ca'} {{(pid=88113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 10:00:26.523894 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.notifications [None req-74766d63-45a9-41b2-8550-80ff958746ca admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0c466f3a96df4d0dbd5e4b3c28805f52', 'request_id': 'req-74766d63-45a9-41b2-8550-80ff958746ca'} {{(pid=88113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 10:00:26.531091 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.notifications [None req-74766d63-45a9-41b2-8550-80ff958746ca admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0c466f3a96df4d0dbd5e4b3c28805f52', 'request_id': 'req-74766d63-45a9-41b2-8550-80ff958746ca'} {{(pid=88113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 10:00:26.542870 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.notifications [None req-74766d63-45a9-41b2-8550-80ff958746ca admin admin] The token cache is being invalidate because project 0c466f3a96df4d0dbd5e4b3c28805f52 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88113) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 10:00:26.543152 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.notifications [None req-74766d63-45a9-41b2-8550-80ff958746ca admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 10:00:26.543784 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.notifications [None req-74766d63-45a9-41b2-8550-80ff958746ca admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 10:00:26.545261 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 1360/2723] 10.222.0.13 () {58 vars in 1233 bytes} [Wed Aug 31 10:00:26 2022] DELETE /identity/v3/projects/0c466f3a96df4d0dbd5e4b3c28805f52 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 10:00:26.550313 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289a000f-c18d-41ea-a7df-ee246d7dbcaf None None] REQUEST_METHOD: `POST` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:00:26.550606 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289a000f-c18d-41ea-a7df-ee246d7dbcaf None None] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:00:26.550911 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289a000f-c18d-41ea-a7df-ee246d7dbcaf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:00:26.571434 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.auth.core [None req-289a000f-c18d-41ea-a7df-ee246d7dbcaf None None] MFA Rules not processed for user `fd836cf590a145df9acf19cf3ae2ac56`. Rule list: `[]` (Enabled: `True`). {{(pid=88114) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 10:00:26.572667 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.fernet_utils [None req-289a000f-c18d-41ea-a7df-ee246d7dbcaf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88114) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:00:26.637508 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 1364/2724] 10.222.0.13 () {58 vars in 958 bytes} [Wed Aug 31 10:00:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 10:00:26.646677 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69ac640-5298-445c-8869-40fa7de32c92 None None] REQUEST_METHOD: `POST` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:00:26.646971 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69ac640-5298-445c-8869-40fa7de32c92 None None] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:00:26.647322 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69ac640-5298-445c-8869-40fa7de32c92 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:00:26.668887 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.auth.core [None req-c69ac640-5298-445c-8869-40fa7de32c92 None None] MFA Rules not processed for user `fd836cf590a145df9acf19cf3ae2ac56`. Rule list: `[]` (Enabled: `True`). {{(pid=88113) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 10:00:26.670137 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.fernet_utils [None req-c69ac640-5298-445c-8869-40fa7de32c92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:00:26.678919 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 1361/2725] 10.222.0.13 () {58 vars in 958 bytes} [Wed Aug 31 10:00:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 10:00:26.682927 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-289a000f-c18d-41ea-a7df-ee246d7dbcaf None None] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:00:26.692849 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-289a000f-c18d-41ea-a7df-ee246d7dbcaf None None] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:00:26.698317 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73bfec91-ea6f-4b7d-a9fe-ed09c5b3f27b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:00:26.698919 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73bfec91-ea6f-4b7d-a9fe-ed09c5b3f27b admin admin] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:00:26.699192 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73bfec91-ea6f-4b7d-a9fe-ed09c5b3f27b admin admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:00:26.699459 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73bfec91-ea6f-4b7d-a9fe-ed09c5b3f27b admin admin] PATH_INFO: `/v3/domains` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:00:26.700120 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73bfec91-ea6f-4b7d-a9fe-ed09c5b3f27b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88114) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:00:26.700560 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73bfec91-ea6f-4b7d-a9fe-ed09c5b3f27b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:00:26.702158 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73bfec91-ea6f-4b7d-a9fe-ed09c5b3f27b admin admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:00:26.714410 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 1365/2726] 10.222.0.13 () {58 vars in 1153 bytes} [Wed Aug 31 10:00:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:00:26.718797 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c69ac640-5298-445c-8869-40fa7de32c92 None None] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:00:26.728417 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c69ac640-5298-445c-8869-40fa7de32c92 None None] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:00:26.733289 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f9f6b39-0890-409f-bc5a-21e843326f93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:00:26.733940 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9f6b39-0890-409f-bc5a-21e843326f93 admin admin] REQUEST_METHOD: `POST` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:00:26.734221 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9f6b39-0890-409f-bc5a-21e843326f93 admin admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:00:26.734491 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9f6b39-0890-409f-bc5a-21e843326f93 admin admin] PATH_INFO: `/v3/projects` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:00:26.735130 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f9f6b39-0890-409f-bc5a-21e843326f93 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:00:26.737045 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f9f6b39-0890-409f-bc5a-21e843326f93 admin admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:00:26.765614 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 1362/2727] 10.222.0.13 () {60 vars in 1153 bytes} [Wed Aug 31 10:00:26 2022] POST /identity/v3/projects => generated 375 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 10:00:26.769971 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73bfec91-ea6f-4b7d-a9fe-ed09c5b3f27b admin admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:00:26.781123 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73bfec91-ea6f-4b7d-a9fe-ed09c5b3f27b admin admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:00:26.837483 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-244d8e5f-b23a-40d8-8e20-9c7333476d67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:00:26.838082 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244d8e5f-b23a-40d8-8e20-9c7333476d67 admin admin] REQUEST_METHOD: `POST` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:00:26.838361 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244d8e5f-b23a-40d8-8e20-9c7333476d67 admin admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:00:26.838631 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244d8e5f-b23a-40d8-8e20-9c7333476d67 admin admin] PATH_INFO: `/v3/users` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:00:26.839254 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-244d8e5f-b23a-40d8-8e20-9c7333476d67 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:00:26.841164 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-244d8e5f-b23a-40d8-8e20-9c7333476d67 admin admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:00:26.862644 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 1366/2728] 10.222.0.13 () {60 vars in 1144 bytes} [Wed Aug 31 10:00:26 2022] POST /identity/v3/users => generated 288 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 10:00:26.867574 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f9f6b39-0890-409f-bc5a-21e843326f93 admin admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:00:26.877879 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f9f6b39-0890-409f-bc5a-21e843326f93 admin admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:00:26.883250 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca5e3eb4-2a19-4b87-9b12-fd9cddcb1f19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:00:26.883822 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5e3eb4-2a19-4b87-9b12-fd9cddcb1f19 admin admin] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:00:26.884096 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5e3eb4-2a19-4b87-9b12-fd9cddcb1f19 admin admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:00:26.884367 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5e3eb4-2a19-4b87-9b12-fd9cddcb1f19 admin admin] PATH_INFO: `/v3/roles` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:00:26.884995 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca5e3eb4-2a19-4b87-9b12-fd9cddcb1f19 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:00:26.886604 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca5e3eb4-2a19-4b87-9b12-fd9cddcb1f19 admin admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:00:26.887838 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:00:26.897614 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 1363/2729] 10.222.0.13 () {58 vars in 1122 bytes} [Wed Aug 31 10:00:26 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 10:00:26.901939 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-244d8e5f-b23a-40d8-8e20-9c7333476d67 admin admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:00:26.911969 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-244d8e5f-b23a-40d8-8e20-9c7333476d67 admin admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:00:26.916634 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-903cf406-bb9f-456c-a117-6ac87432914c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:00:26.917221 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903cf406-bb9f-456c-a117-6ac87432914c admin admin] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:00:26.917539 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903cf406-bb9f-456c-a117-6ac87432914c admin admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:00:26.917810 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903cf406-bb9f-456c-a117-6ac87432914c admin admin] PATH_INFO: `/v3/roles` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:00:26.918450 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-903cf406-bb9f-456c-a117-6ac87432914c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:00:26.920037 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-903cf406-bb9f-456c-a117-6ac87432914c admin admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:00:26.931470 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 1367/2730] 10.222.0.13 () {58 vars in 1122 bytes} [Wed Aug 31 10:00:26 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 10:00:26.935607 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca5e3eb4-2a19-4b87-9b12-fd9cddcb1f19 admin admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:00:26.946342 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca5e3eb4-2a19-4b87-9b12-fd9cddcb1f19 admin admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:00:26.950905 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e453997-7a74-41a0-a020-61f17ce42b2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:00:26.951540 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e453997-7a74-41a0-a020-61f17ce42b2b admin admin] REQUEST_METHOD: `PUT` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:00:26.951815 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e453997-7a74-41a0-a020-61f17ce42b2b admin admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:00:26.952084 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e453997-7a74-41a0-a020-61f17ce42b2b admin admin] PATH_INFO: `/v3/projects/cdafb29648f34d9cbad378da051ac5c5/users/e98df7d1938343f081cd4e003fbf43a3/roles/6c99f5d9f1f243eaa134d8540f21095d` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:00:26.967237 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e453997-7a74-41a0-a020-61f17ce42b2b admin admin] RBAC: Authorizing `identity:create_grant(project_id=cdafb29648f34d9cbad378da051ac5c5, user_id=e98df7d1938343f081cd4e003fbf43a3, role_id=6c99f5d9f1f243eaa134d8540f21095d)` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:00:26.969838 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e453997-7a74-41a0-a020-61f17ce42b2b admin admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:00:26.983885 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 1364/2731] 10.222.0.13 () {60 vars in 1483 bytes} [Wed Aug 31 10:00:26 2022] PUT /identity/v3/projects/cdafb29648f34d9cbad378da051ac5c5/users/e98df7d1938343f081cd4e003fbf43a3/roles/6c99f5d9f1f243eaa134d8540f21095d => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 10:00:26.995254 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94f5e03-575b-46f2-bd5c-3a3089c9c57c None None] REQUEST_METHOD: `POST` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:00:26.995768 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94f5e03-575b-46f2-bd5c-3a3089c9c57c None None] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:00:26.995895 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94f5e03-575b-46f2-bd5c-3a3089c9c57c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:00:27.024320 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.auth.core [None req-a94f5e03-575b-46f2-bd5c-3a3089c9c57c None None] MFA Rules not processed for user `e98df7d1938343f081cd4e003fbf43a3`. Rule list: `[]` (Enabled: `True`). {{(pid=88114) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 10:00:27.025565 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.fernet_utils [None req-a94f5e03-575b-46f2-bd5c-3a3089c9c57c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88114) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:00:27.029564 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:00:27.093808 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 1368/2732] 10.222.0.13 () {58 vars in 958 bytes} [Wed Aug 31 10:00:26 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 10:00:27.105529 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e453997-7a74-41a0-a020-61f17ce42b2b admin admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:00:27.107170 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.fernet_utils [None req-1e453997-7a74-41a0-a020-61f17ce42b2b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:00:27.164749 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e453997-7a74-41a0-a020-61f17ce42b2b admin admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:00:27.168431 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0adde671-7f62-4ddf-8916-876cd6e40dd3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d52b69c8e94441dacfcff944326f169', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '49548eb44d62415582af820ab66decee', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:00:27.169028 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0adde671-7f62-4ddf-8916-876cd6e40dd3 service cinder] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:00:27.169302 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0adde671-7f62-4ddf-8916-876cd6e40dd3 service cinder] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:00:27.169598 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0adde671-7f62-4ddf-8916-876cd6e40dd3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:00:27.178001 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0adde671-7f62-4ddf-8916-876cd6e40dd3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:00:27.180197 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0adde671-7f62-4ddf-8916-876cd6e40dd3 service cinder] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:00:27.188983 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 1365/2733] 10.222.0.13 () {60 vars in 1365 bytes} [Wed Aug 31 10:00:27 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 10:00:28.548958 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a197fda1-331e-432c-9985-681d61e5a21b None None] REQUEST_METHOD: `POST` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:00:28.549551 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a197fda1-331e-432c-9985-681d61e5a21b None None] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:00:28.549551 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a197fda1-331e-432c-9985-681d61e5a21b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:00:28.568856 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.auth.core [None req-a197fda1-331e-432c-9985-681d61e5a21b None None] MFA Rules not processed for user `e98df7d1938343f081cd4e003fbf43a3`. Rule list: `[]` (Enabled: `True`). {{(pid=88114) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 10:00:28.571407 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.fernet_utils [None req-a197fda1-331e-432c-9985-681d61e5a21b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88114) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:00:28.579997 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 1369/2734] 10.222.0.13 () {58 vars in 958 bytes} [Wed Aug 31 10:00:28 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 10:00:37.184960 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=38,keystone:INSERT=1 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:00:38.563180 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=28 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:00:40.896928 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483e6528-dbb2-48f9-8a74-f72a57afc815 None None] REQUEST_METHOD: `POST` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:00:40.898032 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483e6528-dbb2-48f9-8a74-f72a57afc815 None None] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:00:40.898240 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483e6528-dbb2-48f9-8a74-f72a57afc815 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:00:40.919767 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.auth.core [None req-483e6528-dbb2-48f9-8a74-f72a57afc815 None None] MFA Rules not processed for user `6dc6f70f3b1b40d9be1f9d01da0de1f1`. Rule list: `[]` (Enabled: `True`). {{(pid=88113) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 10:00:40.921122 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.fernet_utils [None req-483e6528-dbb2-48f9-8a74-f72a57afc815 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:00:40.983002 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 1366/2735] 10.222.0.13 () {58 vars in 958 bytes} [Wed Aug 31 10:00:40 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 10:00:48.063248 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db04b7b0-874a-42ed-ade4-6ad5a3a07e40 None None] REQUEST_METHOD: `POST` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:00:48.063815 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db04b7b0-874a-42ed-ade4-6ad5a3a07e40 None None] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:00:48.063815 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db04b7b0-874a-42ed-ade4-6ad5a3a07e40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:00:48.087052 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.auth.core [None req-db04b7b0-874a-42ed-ade4-6ad5a3a07e40 None None] MFA Rules not processed for user `768cb67c1ef24251aaa113c12d81a250`. Rule list: `[]` (Enabled: `True`). {{(pid=88114) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 10:00:48.088973 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.fernet_utils [None req-db04b7b0-874a-42ed-ade4-6ad5a3a07e40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88114) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:00:48.158680 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 1370/2736] 10.222.0.13 () {58 vars in 1009 bytes} [Wed Aug 31 10:00:48 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 10:00:48.211381 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-483e6528-dbb2-48f9-8a74-f72a57afc815 None None] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:00:48.212946 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.fernet_utils [None req-483e6528-dbb2-48f9-8a74-f72a57afc815 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:00:48.284729 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-483e6528-dbb2-48f9-8a74-f72a57afc815 None None] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:00:48.290184 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f96e8482-e601-4e90-a430-b6f0f7f9656b service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '202bb9db7a8a4977ba788a290ed9a3fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '49548eb44d62415582af820ab66decee', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:00:48.290787 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96e8482-e601-4e90-a430-b6f0f7f9656b service swift] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:00:48.291060 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96e8482-e601-4e90-a430-b6f0f7f9656b service swift] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:00:48.291327 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96e8482-e601-4e90-a430-b6f0f7f9656b service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:00:48.299693 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f96e8482-e601-4e90-a430-b6f0f7f9656b service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:00:48.301281 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f96e8482-e601-4e90-a430-b6f0f7f9656b service swift] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:00:48.310526 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 1367/2737] 10.222.0.13 () {60 vars in 1384 bytes} [Wed Aug 31 10:00:48 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 31 10:00:48.893978 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836e6a25-d525-4c87-9c27-7495600623bb None None] REQUEST_METHOD: `POST` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:00:48.894263 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836e6a25-d525-4c87-9c27-7495600623bb None None] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:00:48.894614 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836e6a25-d525-4c87-9c27-7495600623bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:00:48.916188 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.auth.core [None req-836e6a25-d525-4c87-9c27-7495600623bb None None] MFA Rules not processed for user `768cb67c1ef24251aaa113c12d81a250`. Rule list: `[]` (Enabled: `True`). {{(pid=88114) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 10:00:48.917447 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.fernet_utils [None req-836e6a25-d525-4c87-9c27-7495600623bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88114) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:00:48.924867 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 1371/2738] 10.222.0.13 () {58 vars in 967 bytes} [Wed Aug 31 10:00:48 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 10:00:48.938181 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f96e8482-e601-4e90-a430-b6f0f7f9656b service swift] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:00:48.950083 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f96e8482-e601-4e90-a430-b6f0f7f9656b service swift] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:00:48.958097 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf55954-c529-450b-bf56-bb5b0d244af9 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '202bb9db7a8a4977ba788a290ed9a3fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '49548eb44d62415582af820ab66decee', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:00:48.958097 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf55954-c529-450b-bf56-bb5b0d244af9 service swift] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:00:48.958327 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf55954-c529-450b-bf56-bb5b0d244af9 service swift] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:00:48.958580 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf55954-c529-450b-bf56-bb5b0d244af9 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:00:48.968350 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bf55954-c529-450b-bf56-bb5b0d244af9 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:00:48.969953 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bf55954-c529-450b-bf56-bb5b0d244af9 service swift] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:00:48.979139 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 1368/2739] 10.222.0.13 () {60 vars in 1384 bytes} [Wed Aug 31 10:00:48 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 31 10:00:49.210178 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60002755-84c3-4caf-8f12-d1840e8cb525 None None] REQUEST_METHOD: `POST` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:00:49.210664 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60002755-84c3-4caf-8f12-d1840e8cb525 None None] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:00:49.210879 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60002755-84c3-4caf-8f12-d1840e8cb525 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:00:49.229957 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.auth.core [None req-60002755-84c3-4caf-8f12-d1840e8cb525 None None] MFA Rules not processed for user `768cb67c1ef24251aaa113c12d81a250`. Rule list: `[]` (Enabled: `True`). {{(pid=88114) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 10:00:49.231160 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.fernet_utils [None req-60002755-84c3-4caf-8f12-d1840e8cb525 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88114) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:00:49.238275 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 1372/2740] 10.222.0.13 () {58 vars in 1009 bytes} [Wed Aug 31 10:00:49 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 10:00:49.250915 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf55954-c529-450b-bf56-bb5b0d244af9 service swift] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:00:49.262816 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf55954-c529-450b-bf56-bb5b0d244af9 service swift] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:00:49.267978 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1619c76-6ba6-40d7-9980-8a4d0c81980f service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '202bb9db7a8a4977ba788a290ed9a3fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '49548eb44d62415582af820ab66decee', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:00:49.268557 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1619c76-6ba6-40d7-9980-8a4d0c81980f service swift] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:00:49.268836 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1619c76-6ba6-40d7-9980-8a4d0c81980f service swift] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:00:49.269128 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1619c76-6ba6-40d7-9980-8a4d0c81980f service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:00:49.277705 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1619c76-6ba6-40d7-9980-8a4d0c81980f service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:00:49.279298 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1619c76-6ba6-40d7-9980-8a4d0c81980f service swift] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:00:49.287208 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 1369/2741] 10.222.0.13 () {60 vars in 1384 bytes} [Wed Aug 31 10:00:49 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 31 10:00:49.611748 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c700d262-caee-4ada-9eca-d3b39fc6f300 None None] REQUEST_METHOD: `POST` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:00:49.612049 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c700d262-caee-4ada-9eca-d3b39fc6f300 None None] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:00:49.612320 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c700d262-caee-4ada-9eca-d3b39fc6f300 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:00:49.632756 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.auth.core [None req-c700d262-caee-4ada-9eca-d3b39fc6f300 None None] MFA Rules not processed for user `768cb67c1ef24251aaa113c12d81a250`. Rule list: `[]` (Enabled: `True`). {{(pid=88114) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 10:00:49.633980 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.fernet_utils [None req-c700d262-caee-4ada-9eca-d3b39fc6f300 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88114) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:00:49.641065 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 1373/2742] 10.222.0.13 () {58 vars in 967 bytes} [Wed Aug 31 10:00:49 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 10:00:49.656983 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1619c76-6ba6-40d7-9980-8a4d0c81980f service swift] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:00:49.668943 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1619c76-6ba6-40d7-9980-8a4d0c81980f service swift] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:00:49.673991 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d4ca90-e919-49e8-8fec-3e3413dc0d1b service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '202bb9db7a8a4977ba788a290ed9a3fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '49548eb44d62415582af820ab66decee', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:00:49.674636 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d4ca90-e919-49e8-8fec-3e3413dc0d1b service swift] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:00:49.674908 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d4ca90-e919-49e8-8fec-3e3413dc0d1b service swift] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:00:49.675174 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d4ca90-e919-49e8-8fec-3e3413dc0d1b service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:00:49.684414 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44d4ca90-e919-49e8-8fec-3e3413dc0d1b service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:00:49.686048 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44d4ca90-e919-49e8-8fec-3e3413dc0d1b service swift] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:00:49.694519 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 1370/2743] 10.222.0.13 () {60 vars in 1384 bytes} [Wed Aug 31 10:00:49 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 31 10:00:51.869699 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c700d262-caee-4ada-9eca-d3b39fc6f300 None None] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:00:51.936044 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c700d262-caee-4ada-9eca-d3b39fc6f300 None None] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:00:51.940800 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c4e593-3596-41e1-9a05-9f2c09e3c113 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:00:51.941518 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c4e593-3596-41e1-9a05-9f2c09e3c113 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:00:51.941796 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c4e593-3596-41e1-9a05-9f2c09e3c113 admin admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:00:51.942064 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c4e593-3596-41e1-9a05-9f2c09e3c113 admin admin] PATH_INFO: `/v3/users/e98df7d1938343f081cd4e003fbf43a3` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:00:51.943226 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29c4e593-3596-41e1-9a05-9f2c09e3c113 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e98df7d1938343f081cd4e003fbf43a3)` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:00:51.946270 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29c4e593-3596-41e1-9a05-9f2c09e3c113 admin admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:00:51.999237 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.notifications [None req-29c4e593-3596-41e1-9a05-9f2c09e3c113 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e98df7d1938343f081cd4e003fbf43a3'} {{(pid=88114) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 10:00:52.006760 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.notifications [None req-29c4e593-3596-41e1-9a05-9f2c09e3c113 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e98df7d1938343f081cd4e003fbf43a3'} {{(pid=88114) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 10:00:52.014545 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.notifications [None req-29c4e593-3596-41e1-9a05-9f2c09e3c113 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e98df7d1938343f081cd4e003fbf43a3'} {{(pid=88114) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 10:00:52.015199 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.notifications [None req-29c4e593-3596-41e1-9a05-9f2c09e3c113 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e98df7d1938343f081cd4e003fbf43a3'} {{(pid=88114) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 10:00:52.030287 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.notifications [None req-29c4e593-3596-41e1-9a05-9f2c09e3c113 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e98df7d1938343f081cd4e003fbf43a3'} {{(pid=88114) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 10:00:52.036839 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 1374/2744] 10.222.0.13 () {58 vars in 1224 bytes} [Wed Aug 31 10:00:51 2022] DELETE /identity/v3/users/e98df7d1938343f081cd4e003fbf43a3 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 10:00:52.041913 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a92c346-235c-4116-9896-d8c3348b3a2c None None] REQUEST_METHOD: `POST` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:00:52.042463 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a92c346-235c-4116-9896-d8c3348b3a2c None None] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:00:52.042463 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a92c346-235c-4116-9896-d8c3348b3a2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:00:52.062805 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.auth.core [None req-2a92c346-235c-4116-9896-d8c3348b3a2c None None] MFA Rules not processed for user `fd836cf590a145df9acf19cf3ae2ac56`. Rule list: `[]` (Enabled: `True`). {{(pid=88113) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 10:00:52.064194 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.fernet_utils [None req-2a92c346-235c-4116-9896-d8c3348b3a2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:00:52.119696 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=100 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:00:52.133687 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 1371/2745] 10.222.0.13 () {58 vars in 958 bytes} [Wed Aug 31 10:00:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 10:00:52.143983 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c4e593-3596-41e1-9a05-9f2c09e3c113 admin admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:00:52.145704 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.fernet_utils [None req-29c4e593-3596-41e1-9a05-9f2c09e3c113 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88114) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:00:52.153973 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:00:52.215356 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c4e593-3596-41e1-9a05-9f2c09e3c113 admin admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:00:52.219476 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc5248f-809a-408e-af37-479185464cc7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9bdaacd597442c2bcde2c555f2cc140', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '49548eb44d62415582af820ab66decee', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:00:52.220132 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc5248f-809a-408e-af37-479185464cc7 service neutron] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:00:52.220406 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc5248f-809a-408e-af37-479185464cc7 service neutron] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:00:52.220671 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc5248f-809a-408e-af37-479185464cc7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:00:52.231626 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bc5248f-809a-408e-af37-479185464cc7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:00:52.233005 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bc5248f-809a-408e-af37-479185464cc7 service neutron] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:00:52.244854 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 1375/2746] 10.222.0.13 () {60 vars in 1365 bytes} [Wed Aug 31 10:00:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 10:00:52.449041 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a92c346-235c-4116-9896-d8c3348b3a2c None None] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:00:52.450495 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.fernet_utils [None req-2a92c346-235c-4116-9896-d8c3348b3a2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:00:52.463479 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a92c346-235c-4116-9896-d8c3348b3a2c None None] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:00:52.467773 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-941266e0-5a6a-4512-bced-bfe7e7d82ae1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:00:52.468415 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941266e0-5a6a-4512-bced-bfe7e7d82ae1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:00:52.469088 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941266e0-5a6a-4512-bced-bfe7e7d82ae1 admin admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:00:52.469779 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941266e0-5a6a-4512-bced-bfe7e7d82ae1 admin admin] PATH_INFO: `/v3/projects/cdafb29648f34d9cbad378da051ac5c5` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:00:52.471051 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-941266e0-5a6a-4512-bced-bfe7e7d82ae1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cdafb29648f34d9cbad378da051ac5c5)` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:00:52.473244 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-941266e0-5a6a-4512-bced-bfe7e7d82ae1 admin admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:00:52.532522 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.notifications [None req-941266e0-5a6a-4512-bced-bfe7e7d82ae1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cdafb29648f34d9cbad378da051ac5c5', 'request_id': 'req-941266e0-5a6a-4512-bced-bfe7e7d82ae1'} {{(pid=88113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 10:00:52.541010 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.notifications [None req-941266e0-5a6a-4512-bced-bfe7e7d82ae1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cdafb29648f34d9cbad378da051ac5c5', 'request_id': 'req-941266e0-5a6a-4512-bced-bfe7e7d82ae1'} {{(pid=88113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 10:00:52.549162 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.notifications [None req-941266e0-5a6a-4512-bced-bfe7e7d82ae1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cdafb29648f34d9cbad378da051ac5c5', 'request_id': 'req-941266e0-5a6a-4512-bced-bfe7e7d82ae1'} {{(pid=88113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 10:00:52.560989 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.notifications [None req-941266e0-5a6a-4512-bced-bfe7e7d82ae1 admin admin] The token cache is being invalidate because project cdafb29648f34d9cbad378da051ac5c5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88113) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 10:00:52.561275 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.notifications [None req-941266e0-5a6a-4512-bced-bfe7e7d82ae1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 10:00:52.561977 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.notifications [None req-941266e0-5a6a-4512-bced-bfe7e7d82ae1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 10:00:52.563480 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 1372/2747] 10.222.0.13 () {58 vars in 1233 bytes} [Wed Aug 31 10:00:52 2022] DELETE /identity/v3/projects/cdafb29648f34d9cbad378da051ac5c5 => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 10:01:02.239328 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=24 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:01:02.557671 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=29,keystone:DELETE=7,keystone:INSERT=1 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:01:42.811769 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc5248f-809a-408e-af37-479185464cc7 service neutron] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:01:42.813589 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.fernet_utils [None req-9bc5248f-809a-408e-af37-479185464cc7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88114) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:01:42.824889 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc5248f-809a-408e-af37-479185464cc7 service neutron] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:01:42.873689 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21474e47-8e13-405e-a58d-1b0ecfd4dcbc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9bdaacd597442c2bcde2c555f2cc140', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '49548eb44d62415582af820ab66decee', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:01:42.874285 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21474e47-8e13-405e-a58d-1b0ecfd4dcbc service neutron] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:01:42.874593 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21474e47-8e13-405e-a58d-1b0ecfd4dcbc service neutron] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:01:42.874861 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21474e47-8e13-405e-a58d-1b0ecfd4dcbc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:01:42.876602 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.fernet_utils [None req-21474e47-8e13-405e-a58d-1b0ecfd4dcbc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88114) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:01:42.957055 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21474e47-8e13-405e-a58d-1b0ecfd4dcbc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:01:42.958522 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21474e47-8e13-405e-a58d-1b0ecfd4dcbc service neutron] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:01:42.977265 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 1376/2748] 10.222.0.13 () {60 vars in 1365 bytes} [Wed Aug 31 10:01:42 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 166 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 10:01:47.448029 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-941266e0-5a6a-4512-bced-bfe7e7d82ae1 admin admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:01:47.449504 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.fernet_utils [None req-941266e0-5a6a-4512-bced-bfe7e7d82ae1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:01:47.520549 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-941266e0-5a6a-4512-bced-bfe7e7d82ae1 admin admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:01:47.525477 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e0d6983-f819-4196-9632-aab6bfac6c8c service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ff3aff5ef444910be6dbb989d0e3194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '49548eb44d62415582af820ab66decee', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:01:47.526075 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e0d6983-f819-4196-9632-aab6bfac6c8c service placement] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:01:47.526351 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e0d6983-f819-4196-9632-aab6bfac6c8c service placement] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:01:47.526639 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e0d6983-f819-4196-9632-aab6bfac6c8c service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:01:47.528366 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.fernet_utils [None req-5e0d6983-f819-4196-9632-aab6bfac6c8c service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:01:47.540133 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e0d6983-f819-4196-9632-aab6bfac6c8c service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:01:47.541755 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e0d6983-f819-4196-9632-aab6bfac6c8c service placement] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:01:47.552184 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 1373/2749] 10.222.0.13 () {60 vars in 1365 bytes} [Wed Aug 31 10:01:47 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 10:01:52.980812 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=61 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:01:56.432209 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21474e47-8e13-405e-a58d-1b0ecfd4dcbc service neutron] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:01:56.433692 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.fernet_utils [None req-21474e47-8e13-405e-a58d-1b0ecfd4dcbc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88114) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:01:56.445894 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21474e47-8e13-405e-a58d-1b0ecfd4dcbc service neutron] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:01:56.449625 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3308bd03-915b-42c5-a8cc-03c7d2a888e3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9bdaacd597442c2bcde2c555f2cc140', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '49548eb44d62415582af820ab66decee', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:01:56.450224 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3308bd03-915b-42c5-a8cc-03c7d2a888e3 service neutron] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:01:56.450505 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3308bd03-915b-42c5-a8cc-03c7d2a888e3 service neutron] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:01:56.450773 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3308bd03-915b-42c5-a8cc-03c7d2a888e3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:01:56.453262 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.fernet_utils [None req-3308bd03-915b-42c5-a8cc-03c7d2a888e3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88114) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:01:56.517702 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3308bd03-915b-42c5-a8cc-03c7d2a888e3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:01:56.519134 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3308bd03-915b-42c5-a8cc-03c7d2a888e3 service neutron] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:01:56.529397 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 1377/2750] 10.222.0.13 () {60 vars in 1365 bytes} [Wed Aug 31 10:01:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 10:01:57.547918 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=32 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:01:59.620077 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e0d6983-f819-4196-9632-aab6bfac6c8c service placement] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:01:59.621589 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.fernet_utils [None req-5e0d6983-f819-4196-9632-aab6bfac6c8c service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:01:59.640980 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e0d6983-f819-4196-9632-aab6bfac6c8c service placement] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:01:59.646396 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c18b6f54-4d1a-46b7-a6b0-45013b50e121 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ff3aff5ef444910be6dbb989d0e3194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '49548eb44d62415582af820ab66decee', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:01:59.647002 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18b6f54-4d1a-46b7-a6b0-45013b50e121 service placement] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:01:59.647274 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18b6f54-4d1a-46b7-a6b0-45013b50e121 service placement] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:01:59.647536 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18b6f54-4d1a-46b7-a6b0-45013b50e121 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:01:59.649301 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.fernet_utils [None req-c18b6f54-4d1a-46b7-a6b0-45013b50e121 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:01:59.665017 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c18b6f54-4d1a-46b7-a6b0-45013b50e121 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:01:59.666668 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c18b6f54-4d1a-46b7-a6b0-45013b50e121 service placement] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:01:59.679851 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 1374/2751] 10.222.0.13 () {60 vars in 1365 bytes} [Wed Aug 31 10:01:59 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 60 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 10:02:02.881595 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3308bd03-915b-42c5-a8cc-03c7d2a888e3 service neutron] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:02:02.884123 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.fernet_utils [None req-3308bd03-915b-42c5-a8cc-03c7d2a888e3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88114) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:02:02.902155 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3308bd03-915b-42c5-a8cc-03c7d2a888e3 service neutron] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:02:02.908045 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-460b13c2-e684-459d-848b-31b2e7aa347c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:02:02.908704 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460b13c2-e684-459d-848b-31b2e7aa347c admin admin] REQUEST_METHOD: `DELETE` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:02:02.909490 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460b13c2-e684-459d-848b-31b2e7aa347c admin admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:02:02.909915 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460b13c2-e684-459d-848b-31b2e7aa347c admin admin] PATH_INFO: `/v3/users/1090c027746845239055973b44207556` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:02:02.911368 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-460b13c2-e684-459d-848b-31b2e7aa347c admin admin] RBAC: Authorizing `identity:delete_user(user_id=1090c027746845239055973b44207556)` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:02:02.913283 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-460b13c2-e684-459d-848b-31b2e7aa347c admin admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:02:02.960551 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.notifications [None req-460b13c2-e684-459d-848b-31b2e7aa347c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1090c027746845239055973b44207556'} {{(pid=88114) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 10:02:02.967471 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.notifications [None req-460b13c2-e684-459d-848b-31b2e7aa347c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1090c027746845239055973b44207556'} {{(pid=88114) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 10:02:02.976279 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.notifications [None req-460b13c2-e684-459d-848b-31b2e7aa347c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1090c027746845239055973b44207556'} {{(pid=88114) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 10:02:02.977006 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.notifications [None req-460b13c2-e684-459d-848b-31b2e7aa347c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1090c027746845239055973b44207556'} {{(pid=88114) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 10:02:02.994830 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.notifications [None req-460b13c2-e684-459d-848b-31b2e7aa347c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1090c027746845239055973b44207556'} {{(pid=88114) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 10:02:03.002465 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 1378/2752] 10.222.0.13 () {58 vars in 1224 bytes} [Wed Aug 31 10:02:02 2022] DELETE /identity/v3/users/1090c027746845239055973b44207556 => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 10:02:03.203700 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c18b6f54-4d1a-46b7-a6b0-45013b50e121 service placement] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:02:03.214843 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c18b6f54-4d1a-46b7-a6b0-45013b50e121 service placement] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:02:03.273399 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b37050-bcc8-4862-8bd4-c7520b0465a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:02:03.274033 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b37050-bcc8-4862-8bd4-c7520b0465a4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:02:03.274314 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b37050-bcc8-4862-8bd4-c7520b0465a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:02:03.274589 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b37050-bcc8-4862-8bd4-c7520b0465a4 admin admin] PATH_INFO: `/v3/projects/6634173c9d534fca800d394d545a9d0c` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:02:03.275777 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5b37050-bcc8-4862-8bd4-c7520b0465a4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6634173c9d534fca800d394d545a9d0c)` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:02:03.277640 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5b37050-bcc8-4862-8bd4-c7520b0465a4 admin admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:02:03.325012 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.notifications [None req-a5b37050-bcc8-4862-8bd4-c7520b0465a4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6634173c9d534fca800d394d545a9d0c', 'request_id': 'req-a5b37050-bcc8-4862-8bd4-c7520b0465a4'} {{(pid=88113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 10:02:03.331608 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.notifications [None req-a5b37050-bcc8-4862-8bd4-c7520b0465a4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6634173c9d534fca800d394d545a9d0c', 'request_id': 'req-a5b37050-bcc8-4862-8bd4-c7520b0465a4'} {{(pid=88113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 10:02:03.338923 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.notifications [None req-a5b37050-bcc8-4862-8bd4-c7520b0465a4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6634173c9d534fca800d394d545a9d0c', 'request_id': 'req-a5b37050-bcc8-4862-8bd4-c7520b0465a4'} {{(pid=88113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 10:02:03.350424 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.notifications [None req-a5b37050-bcc8-4862-8bd4-c7520b0465a4 admin admin] The token cache is being invalidate because project 6634173c9d534fca800d394d545a9d0c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88113) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 10:02:03.350707 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.notifications [None req-a5b37050-bcc8-4862-8bd4-c7520b0465a4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 10:02:03.351323 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.notifications [None req-a5b37050-bcc8-4862-8bd4-c7520b0465a4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 10:02:03.352813 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 1375/2753] 10.222.0.13 () {58 vars in 1233 bytes} [Wed Aug 31 10:02:03 2022] DELETE /identity/v3/projects/6634173c9d534fca800d394d545a9d0c => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 10:02:03.357006 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-460b13c2-e684-459d-848b-31b2e7aa347c admin admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:02:03.358530 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.fernet_utils [None req-460b13c2-e684-459d-848b-31b2e7aa347c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88114) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:02:03.371575 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-460b13c2-e684-459d-848b-31b2e7aa347c admin admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:02:03.428421 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26970030-2d6b-4441-9475-193369921420 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:02:03.429041 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26970030-2d6b-4441-9475-193369921420 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:02:03.429314 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26970030-2d6b-4441-9475-193369921420 admin admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:02:03.429615 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26970030-2d6b-4441-9475-193369921420 admin admin] PATH_INFO: `/v3/users/1e55ab86464d40dd85871673319b6986` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:02:03.440229 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:02:03.444089 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26970030-2d6b-4441-9475-193369921420 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1e55ab86464d40dd85871673319b6986)` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:02:03.446044 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26970030-2d6b-4441-9475-193369921420 admin admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:02:03.495391 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.notifications [None req-26970030-2d6b-4441-9475-193369921420 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1e55ab86464d40dd85871673319b6986'} {{(pid=88114) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 10:02:03.502535 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.notifications [None req-26970030-2d6b-4441-9475-193369921420 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1e55ab86464d40dd85871673319b6986'} {{(pid=88114) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 10:02:03.509542 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.notifications [None req-26970030-2d6b-4441-9475-193369921420 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1e55ab86464d40dd85871673319b6986'} {{(pid=88114) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 10:02:03.510176 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.notifications [None req-26970030-2d6b-4441-9475-193369921420 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1e55ab86464d40dd85871673319b6986'} {{(pid=88114) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 10:02:03.522597 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.notifications [None req-26970030-2d6b-4441-9475-193369921420 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1e55ab86464d40dd85871673319b6986'} {{(pid=88114) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 10:02:03.528418 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 1379/2754] 10.222.0.13 () {58 vars in 1224 bytes} [Wed Aug 31 10:02:03 2022] DELETE /identity/v3/users/1e55ab86464d40dd85871673319b6986 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 10:02:03.704215 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b37050-bcc8-4862-8bd4-c7520b0465a4 admin admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:02:03.705693 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.fernet_utils [None req-a5b37050-bcc8-4862-8bd4-c7520b0465a4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:02:03.721834 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b37050-bcc8-4862-8bd4-c7520b0465a4 admin admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:02:03.780664 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84cac363-85c0-45a8-aca7-e478b0773ff6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:02:03.781293 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84cac363-85c0-45a8-aca7-e478b0773ff6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:02:03.781600 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84cac363-85c0-45a8-aca7-e478b0773ff6 admin admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:02:03.781870 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84cac363-85c0-45a8-aca7-e478b0773ff6 admin admin] PATH_INFO: `/v3/projects/c5f727a6b9094171b5fd34fe23f9cc5a` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:02:03.792378 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84cac363-85c0-45a8-aca7-e478b0773ff6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c5f727a6b9094171b5fd34fe23f9cc5a)` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:02:03.794414 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84cac363-85c0-45a8-aca7-e478b0773ff6 admin admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:02:03.805798 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:02:03.843017 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.notifications [None req-84cac363-85c0-45a8-aca7-e478b0773ff6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c5f727a6b9094171b5fd34fe23f9cc5a', 'request_id': 'req-84cac363-85c0-45a8-aca7-e478b0773ff6'} {{(pid=88113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 10:02:03.848481 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.notifications [None req-84cac363-85c0-45a8-aca7-e478b0773ff6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c5f727a6b9094171b5fd34fe23f9cc5a', 'request_id': 'req-84cac363-85c0-45a8-aca7-e478b0773ff6'} {{(pid=88113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 10:02:03.856114 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.notifications [None req-84cac363-85c0-45a8-aca7-e478b0773ff6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c5f727a6b9094171b5fd34fe23f9cc5a', 'request_id': 'req-84cac363-85c0-45a8-aca7-e478b0773ff6'} {{(pid=88113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 10:02:03.868572 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.notifications [None req-84cac363-85c0-45a8-aca7-e478b0773ff6 admin admin] The token cache is being invalidate because project c5f727a6b9094171b5fd34fe23f9cc5a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88113) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 10:02:03.868942 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.notifications [None req-84cac363-85c0-45a8-aca7-e478b0773ff6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 10:02:03.869680 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.notifications [None req-84cac363-85c0-45a8-aca7-e478b0773ff6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 10:02:03.871187 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 1376/2755] 10.222.0.13 () {58 vars in 1233 bytes} [Wed Aug 31 10:02:03 2022] DELETE /identity/v3/projects/c5f727a6b9094171b5fd34fe23f9cc5a => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 10:02:13.164329 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26970030-2d6b-4441-9475-193369921420 admin admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:02:13.166440 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.fernet_utils [None req-26970030-2d6b-4441-9475-193369921420 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88114) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:02:13.247393 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26970030-2d6b-4441-9475-193369921420 admin admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:02:13.319306 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98cf81f1-6929-42c9-9555-7814a5a6579b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc076493239d4ccdbdc37c0369c49529', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '49548eb44d62415582af820ab66decee', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:02:13.319925 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98cf81f1-6929-42c9-9555-7814a5a6579b service nova] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:02:13.320207 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98cf81f1-6929-42c9-9555-7814a5a6579b service nova] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:02:13.320474 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98cf81f1-6929-42c9-9555-7814a5a6579b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:02:13.322310 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.fernet_utils [None req-98cf81f1-6929-42c9-9555-7814a5a6579b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88114) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:02:13.333147 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:02:13.384208 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98cf81f1-6929-42c9-9555-7814a5a6579b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:02:13.385830 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98cf81f1-6929-42c9-9555-7814a5a6579b service nova] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:02:13.395338 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 1380/2756] 10.222.0.13 () {60 vars in 1365 bytes} [Wed Aug 31 10:02:13 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 232 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 10:02:13.865538 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=20,keystone:DELETE=7,keystone:INSERT=1 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:02:23.390637 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=23 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:02:31.292059 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120a24e9-fe54-421a-adce-68901c1f0d6a None None] REQUEST_METHOD: `POST` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:02:31.292594 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120a24e9-fe54-421a-adce-68901c1f0d6a None None] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:02:31.292594 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120a24e9-fe54-421a-adce-68901c1f0d6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:02:31.312477 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.auth.core [None req-120a24e9-fe54-421a-adce-68901c1f0d6a None None] MFA Rules not processed for user `768cb67c1ef24251aaa113c12d81a250`. Rule list: `[]` (Enabled: `True`). {{(pid=88113) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 10:02:31.313728 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.fernet_utils [None req-120a24e9-fe54-421a-adce-68901c1f0d6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:02:31.374797 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 1377/2757] 10.222.0.13 () {58 vars in 1009 bytes} [Wed Aug 31 10:02:31 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 10:02:31.431854 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98cf81f1-6929-42c9-9555-7814a5a6579b service nova] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:02:31.433404 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.fernet_utils [None req-98cf81f1-6929-42c9-9555-7814a5a6579b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88114) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:02:31.513281 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98cf81f1-6929-42c9-9555-7814a5a6579b service nova] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:02:31.518615 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3665f944-90fe-4efc-985b-523ad44f59ba service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '202bb9db7a8a4977ba788a290ed9a3fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '49548eb44d62415582af820ab66decee', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:02:31.519218 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3665f944-90fe-4efc-985b-523ad44f59ba service swift] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:02:31.519489 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3665f944-90fe-4efc-985b-523ad44f59ba service swift] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:02:31.519756 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3665f944-90fe-4efc-985b-523ad44f59ba service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:02:31.528319 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3665f944-90fe-4efc-985b-523ad44f59ba service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:02:31.529927 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3665f944-90fe-4efc-985b-523ad44f59ba service swift] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:02:31.539030 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 1381/2758] 10.222.0.13 () {60 vars in 1384 bytes} [Wed Aug 31 10:02:31 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 31 10:02:41.367962 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=25 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:02:41.535296 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=32 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:02:45.300421 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-120a24e9-fe54-421a-adce-68901c1f0d6a None None] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:02:45.301996 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.fernet_utils [None req-120a24e9-fe54-421a-adce-68901c1f0d6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:02:45.366378 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-120a24e9-fe54-421a-adce-68901c1f0d6a None None] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:02:45.370258 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88bf7257-d97b-46e6-8f12-628f505f6750 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9bdaacd597442c2bcde2c555f2cc140', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '49548eb44d62415582af820ab66decee', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:02:45.370969 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88bf7257-d97b-46e6-8f12-628f505f6750 service neutron] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:02:45.371314 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88bf7257-d97b-46e6-8f12-628f505f6750 service neutron] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:02:45.371579 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88bf7257-d97b-46e6-8f12-628f505f6750 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:02:45.373825 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.fernet_utils [None req-88bf7257-d97b-46e6-8f12-628f505f6750 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:02:45.383700 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88bf7257-d97b-46e6-8f12-628f505f6750 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:02:45.385141 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88bf7257-d97b-46e6-8f12-628f505f6750 service neutron] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:02:45.394095 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 1378/2759] 10.222.0.13 () {60 vars in 1365 bytes} [Wed Aug 31 10:02:45 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 10:02:45.469156 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3665f944-90fe-4efc-985b-523ad44f59ba service swift] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:02:45.480777 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3665f944-90fe-4efc-985b-523ad44f59ba service swift] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:02:45.484391 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd17d3fd-05c9-4609-a0eb-0cc3efb524f4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9bdaacd597442c2bcde2c555f2cc140', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '49548eb44d62415582af820ab66decee', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:02:45.484984 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd17d3fd-05c9-4609-a0eb-0cc3efb524f4 service neutron] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:02:45.485259 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd17d3fd-05c9-4609-a0eb-0cc3efb524f4 service neutron] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:02:45.485544 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd17d3fd-05c9-4609-a0eb-0cc3efb524f4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:02:45.488470 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.fernet_utils [None req-cd17d3fd-05c9-4609-a0eb-0cc3efb524f4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88114) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:02:45.503081 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd17d3fd-05c9-4609-a0eb-0cc3efb524f4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:02:45.504586 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd17d3fd-05c9-4609-a0eb-0cc3efb524f4 service neutron] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:02:45.515526 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 1382/2760] 10.222.0.13 () {60 vars in 1365 bytes} [Wed Aug 31 10:02:45 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 10:02:47.270019 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef2d351-916e-4de5-99a4-1d75dd6c4353 None None] REQUEST_METHOD: `POST` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:02:47.270541 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef2d351-916e-4de5-99a4-1d75dd6c4353 None None] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:02:47.270541 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef2d351-916e-4de5-99a4-1d75dd6c4353 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:02:47.290269 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.auth.core [None req-eef2d351-916e-4de5-99a4-1d75dd6c4353 None None] MFA Rules not processed for user `768cb67c1ef24251aaa113c12d81a250`. Rule list: `[]` (Enabled: `True`). {{(pid=88113) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 10:02:47.291542 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.fernet_utils [None req-eef2d351-916e-4de5-99a4-1d75dd6c4353 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:02:47.298651 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 1379/2761] 10.222.0.13 () {58 vars in 1009 bytes} [Wed Aug 31 10:02:47 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 10:02:47.311385 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd17d3fd-05c9-4609-a0eb-0cc3efb524f4 service neutron] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:02:47.322826 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd17d3fd-05c9-4609-a0eb-0cc3efb524f4 service neutron] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:02:47.328942 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84cbd291-fe94-4e51-9a8d-0d18ea2b28e4 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '202bb9db7a8a4977ba788a290ed9a3fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '49548eb44d62415582af820ab66decee', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:02:47.329576 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84cbd291-fe94-4e51-9a8d-0d18ea2b28e4 service swift] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:02:47.329855 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84cbd291-fe94-4e51-9a8d-0d18ea2b28e4 service swift] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:02:47.330190 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84cbd291-fe94-4e51-9a8d-0d18ea2b28e4 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:02:47.339078 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84cbd291-fe94-4e51-9a8d-0d18ea2b28e4 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:02:47.340647 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84cbd291-fe94-4e51-9a8d-0d18ea2b28e4 service swift] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:02:47.350242 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 1383/2762] 10.222.0.13 () {60 vars in 1384 bytes} [Wed Aug 31 10:02:47 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 31 10:02:47.702677 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef2d351-916e-4de5-99a4-1d75dd6c4353 None None] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:02:47.704101 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.fernet_utils [None req-eef2d351-916e-4de5-99a4-1d75dd6c4353 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:02:47.716526 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef2d351-916e-4de5-99a4-1d75dd6c4353 None None] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:02:47.720302 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f49a6144-ad78-4c0b-a813-fdb42c13dac3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9bdaacd597442c2bcde2c555f2cc140', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '49548eb44d62415582af820ab66decee', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:02:47.721774 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49a6144-ad78-4c0b-a813-fdb42c13dac3 service neutron] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:02:47.722098 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49a6144-ad78-4c0b-a813-fdb42c13dac3 service neutron] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:02:47.722475 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49a6144-ad78-4c0b-a813-fdb42c13dac3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:02:47.724352 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.fernet_utils [None req-f49a6144-ad78-4c0b-a813-fdb42c13dac3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:02:47.754881 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f49a6144-ad78-4c0b-a813-fdb42c13dac3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:02:47.756803 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f49a6144-ad78-4c0b-a813-fdb42c13dac3 service neutron] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:02:47.773978 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 1380/2763] 10.222.0.13 () {60 vars in 1365 bytes} [Wed Aug 31 10:02:47 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 72 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 10:02:51.375061 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84cbd291-fe94-4e51-9a8d-0d18ea2b28e4 service swift] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:02:51.385721 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84cbd291-fe94-4e51-9a8d-0d18ea2b28e4 service swift] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:02:51.390008 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e88cf1-b211-42b4-b4cc-80dc03eb9609 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9bdaacd597442c2bcde2c555f2cc140', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '49548eb44d62415582af820ab66decee', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:02:51.391064 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e88cf1-b211-42b4-b4cc-80dc03eb9609 service neutron] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:02:51.391395 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e88cf1-b211-42b4-b4cc-80dc03eb9609 service neutron] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:02:51.391744 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e88cf1-b211-42b4-b4cc-80dc03eb9609 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:02:51.393837 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.fernet_utils [None req-46e88cf1-b211-42b4-b4cc-80dc03eb9609 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88114) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:02:51.411964 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46e88cf1-b211-42b4-b4cc-80dc03eb9609 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:02:51.413728 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46e88cf1-b211-42b4-b4cc-80dc03eb9609 service neutron] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:02:51.429369 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 1384/2764] 10.222.0.13 () {60 vars in 1365 bytes} [Wed Aug 31 10:02:51 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 10:02:57.765565 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=35 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:03:01.422361 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=18 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:03:07.274676 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a435c470-8213-4e6d-8720-b6dcedfa1a18 None None] REQUEST_METHOD: `POST` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:03:07.275181 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a435c470-8213-4e6d-8720-b6dcedfa1a18 None None] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:03:07.275293 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a435c470-8213-4e6d-8720-b6dcedfa1a18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:03:07.295216 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.auth.core [None req-a435c470-8213-4e6d-8720-b6dcedfa1a18 None None] MFA Rules not processed for user `768cb67c1ef24251aaa113c12d81a250`. Rule list: `[]` (Enabled: `True`). {{(pid=88113) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 10:03:07.296429 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.fernet_utils [None req-a435c470-8213-4e6d-8720-b6dcedfa1a18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:03:07.304421 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 1381/2765] 10.222.0.13 () {58 vars in 967 bytes} [Wed Aug 31 10:03:07 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 10:03:07.317665 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e88cf1-b211-42b4-b4cc-80dc03eb9609 service neutron] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:03:07.328886 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e88cf1-b211-42b4-b4cc-80dc03eb9609 service neutron] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:03:07.334970 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c081ef2e-f247-4ac6-bbbe-90a686cba6a7 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '202bb9db7a8a4977ba788a290ed9a3fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '49548eb44d62415582af820ab66decee', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:03:07.335568 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c081ef2e-f247-4ac6-bbbe-90a686cba6a7 service swift] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:03:07.335845 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c081ef2e-f247-4ac6-bbbe-90a686cba6a7 service swift] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:03:07.336119 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c081ef2e-f247-4ac6-bbbe-90a686cba6a7 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:03:07.344537 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c081ef2e-f247-4ac6-bbbe-90a686cba6a7 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:03:07.346144 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c081ef2e-f247-4ac6-bbbe-90a686cba6a7 service swift] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:03:07.354439 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 1385/2766] 10.222.0.13 () {60 vars in 1384 bytes} [Wed Aug 31 10:03:07 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 31 10:03:17.290123 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=3 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:03:17.351098 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=6 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:04:10.650467 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a435c470-8213-4e6d-8720-b6dcedfa1a18 None None] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:04:10.660576 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a435c470-8213-4e6d-8720-b6dcedfa1a18 None None] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:04:10.664200 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca7a005-03d3-4148-9139-dfa13cf97a44 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9bdaacd597442c2bcde2c555f2cc140', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '49548eb44d62415582af820ab66decee', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:04:10.664798 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca7a005-03d3-4148-9139-dfa13cf97a44 service neutron] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:04:10.665075 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca7a005-03d3-4148-9139-dfa13cf97a44 service neutron] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:04:10.665347 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca7a005-03d3-4148-9139-dfa13cf97a44 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:04:10.667771 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.fernet_utils [None req-3ca7a005-03d3-4148-9139-dfa13cf97a44 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:04:10.730709 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ca7a005-03d3-4148-9139-dfa13cf97a44 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:04:10.732139 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ca7a005-03d3-4148-9139-dfa13cf97a44 service neutron] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:04:10.741799 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 1382/2767] 10.222.0.13 () {60 vars in 1365 bytes} [Wed Aug 31 10:04:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 10:04:13.951011 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c081ef2e-f247-4ac6-bbbe-90a686cba6a7 service swift] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:04:13.952571 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.fernet_utils [None req-c081ef2e-f247-4ac6-bbbe-90a686cba6a7 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88114) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:04:13.967741 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c081ef2e-f247-4ac6-bbbe-90a686cba6a7 service swift] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:04:13.973210 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54414868-535f-46d7-a2bc-b20df4a76ff3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:04:13.973977 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54414868-535f-46d7-a2bc-b20df4a76ff3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:04:13.974258 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54414868-535f-46d7-a2bc-b20df4a76ff3 admin admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:04:13.974525 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54414868-535f-46d7-a2bc-b20df4a76ff3 admin admin] PATH_INFO: `/v3/users/6dc6f70f3b1b40d9be1f9d01da0de1f1` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:04:13.975673 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54414868-535f-46d7-a2bc-b20df4a76ff3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6dc6f70f3b1b40d9be1f9d01da0de1f1)` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:04:13.977623 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54414868-535f-46d7-a2bc-b20df4a76ff3 admin admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:04:14.026883 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.notifications [None req-54414868-535f-46d7-a2bc-b20df4a76ff3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6dc6f70f3b1b40d9be1f9d01da0de1f1'} {{(pid=88114) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 10:04:14.033881 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.notifications [None req-54414868-535f-46d7-a2bc-b20df4a76ff3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6dc6f70f3b1b40d9be1f9d01da0de1f1'} {{(pid=88114) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 10:04:14.041441 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.notifications [None req-54414868-535f-46d7-a2bc-b20df4a76ff3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6dc6f70f3b1b40d9be1f9d01da0de1f1'} {{(pid=88114) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 10:04:14.042117 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.notifications [None req-54414868-535f-46d7-a2bc-b20df4a76ff3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6dc6f70f3b1b40d9be1f9d01da0de1f1'} {{(pid=88114) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 10:04:14.060262 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.notifications [None req-54414868-535f-46d7-a2bc-b20df4a76ff3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6dc6f70f3b1b40d9be1f9d01da0de1f1'} {{(pid=88114) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 10:04:14.066268 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 1386/2768] 10.222.0.13 () {58 vars in 1224 bytes} [Wed Aug 31 10:04:13 2022] DELETE /identity/v3/users/6dc6f70f3b1b40d9be1f9d01da0de1f1 => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 10:04:14.243924 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca7a005-03d3-4148-9139-dfa13cf97a44 service neutron] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:04:14.255481 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca7a005-03d3-4148-9139-dfa13cf97a44 service neutron] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:04:14.322641 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c639ad41-5ae4-4a54-8ec9-cbd88807bf24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:04:14.323626 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c639ad41-5ae4-4a54-8ec9-cbd88807bf24 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:04:14.323765 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c639ad41-5ae4-4a54-8ec9-cbd88807bf24 admin admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:04:14.323991 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c639ad41-5ae4-4a54-8ec9-cbd88807bf24 admin admin] PATH_INFO: `/v3/projects/a68bf51dcd404f559e9ae2efeb185712` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:04:14.325120 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c639ad41-5ae4-4a54-8ec9-cbd88807bf24 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a68bf51dcd404f559e9ae2efeb185712)` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:04:14.327065 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c639ad41-5ae4-4a54-8ec9-cbd88807bf24 admin admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:04:14.375750 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.notifications [None req-c639ad41-5ae4-4a54-8ec9-cbd88807bf24 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a68bf51dcd404f559e9ae2efeb185712', 'request_id': 'req-c639ad41-5ae4-4a54-8ec9-cbd88807bf24'} {{(pid=88113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 10:04:14.381195 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.notifications [None req-c639ad41-5ae4-4a54-8ec9-cbd88807bf24 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a68bf51dcd404f559e9ae2efeb185712', 'request_id': 'req-c639ad41-5ae4-4a54-8ec9-cbd88807bf24'} {{(pid=88113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 10:04:14.388076 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.notifications [None req-c639ad41-5ae4-4a54-8ec9-cbd88807bf24 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a68bf51dcd404f559e9ae2efeb185712', 'request_id': 'req-c639ad41-5ae4-4a54-8ec9-cbd88807bf24'} {{(pid=88113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 10:04:14.399132 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.notifications [None req-c639ad41-5ae4-4a54-8ec9-cbd88807bf24 admin admin] The token cache is being invalidate because project a68bf51dcd404f559e9ae2efeb185712 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88113) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 10:04:14.399420 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.notifications [None req-c639ad41-5ae4-4a54-8ec9-cbd88807bf24 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 10:04:14.400055 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.notifications [None req-c639ad41-5ae4-4a54-8ec9-cbd88807bf24 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 10:04:14.402342 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 1383/2769] 10.222.0.13 () {58 vars in 1233 bytes} [Wed Aug 31 10:04:14 2022] DELETE /identity/v3/projects/a68bf51dcd404f559e9ae2efeb185712 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 10:04:14.407395 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18bdcaa3-3f49-4b5d-9975-8729c40b2d33 None None] REQUEST_METHOD: `POST` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:04:14.407688 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18bdcaa3-3f49-4b5d-9975-8729c40b2d33 None None] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:04:14.407956 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18bdcaa3-3f49-4b5d-9975-8729c40b2d33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:04:14.428065 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.auth.core [None req-18bdcaa3-3f49-4b5d-9975-8729c40b2d33 None None] MFA Rules not processed for user `fd836cf590a145df9acf19cf3ae2ac56`. Rule list: `[]` (Enabled: `True`). {{(pid=88114) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 10:04:14.429308 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.fernet_utils [None req-18bdcaa3-3f49-4b5d-9975-8729c40b2d33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88114) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:04:14.488141 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 1387/2770] 10.222.0.13 () {58 vars in 958 bytes} [Wed Aug 31 10:04:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 10:04:14.497842 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a284b6-5d5b-48b8-a62c-9f468c59eecf None None] REQUEST_METHOD: `POST` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:04:14.498125 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a284b6-5d5b-48b8-a62c-9f468c59eecf None None] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:04:14.498397 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a284b6-5d5b-48b8-a62c-9f468c59eecf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:04:14.518899 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.auth.core [None req-39a284b6-5d5b-48b8-a62c-9f468c59eecf None None] MFA Rules not processed for user `fd836cf590a145df9acf19cf3ae2ac56`. Rule list: `[]` (Enabled: `True`). {{(pid=88113) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 10:04:14.520103 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.fernet_utils [None req-39a284b6-5d5b-48b8-a62c-9f468c59eecf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:04:14.528402 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 1384/2771] 10.222.0.13 () {58 vars in 958 bytes} [Wed Aug 31 10:04:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 10:04:14.532695 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18bdcaa3-3f49-4b5d-9975-8729c40b2d33 None None] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:04:14.543480 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18bdcaa3-3f49-4b5d-9975-8729c40b2d33 None None] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:04:14.548219 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee4614c-6960-43e6-afc0-d39aa1b029ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:04:14.548822 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee4614c-6960-43e6-afc0-d39aa1b029ff admin admin] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:04:14.549123 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee4614c-6960-43e6-afc0-d39aa1b029ff admin admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:04:14.549405 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee4614c-6960-43e6-afc0-d39aa1b029ff admin admin] PATH_INFO: `/v3/domains` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:04:14.550106 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fee4614c-6960-43e6-afc0-d39aa1b029ff admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88114) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:04:14.550357 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fee4614c-6960-43e6-afc0-d39aa1b029ff admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:04:14.551949 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fee4614c-6960-43e6-afc0-d39aa1b029ff admin admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:04:14.564946 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 1388/2772] 10.222.0.13 () {58 vars in 1153 bytes} [Wed Aug 31 10:04:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:04:14.569501 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a284b6-5d5b-48b8-a62c-9f468c59eecf None None] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:04:14.579889 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a284b6-5d5b-48b8-a62c-9f468c59eecf None None] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:04:14.585067 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8444bfdd-490d-4250-8765-b33d7092bc3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:04:14.585736 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8444bfdd-490d-4250-8765-b33d7092bc3b admin admin] REQUEST_METHOD: `POST` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:04:14.586230 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8444bfdd-490d-4250-8765-b33d7092bc3b admin admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:04:14.586698 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8444bfdd-490d-4250-8765-b33d7092bc3b admin admin] PATH_INFO: `/v3/projects` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:04:14.587379 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8444bfdd-490d-4250-8765-b33d7092bc3b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:04:14.589988 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8444bfdd-490d-4250-8765-b33d7092bc3b admin admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:04:14.607720 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:04:14.623304 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 1385/2773] 10.222.0.13 () {60 vars in 1153 bytes} [Wed Aug 31 10:04:14 2022] POST /identity/v3/projects => generated 371 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 10:04:14.627311 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee4614c-6960-43e6-afc0-d39aa1b029ff admin admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:04:14.637304 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee4614c-6960-43e6-afc0-d39aa1b029ff admin admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:04:14.694550 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3cb1ac4-aa9b-4dcf-bee1-7d2fecc59225 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:04:14.695152 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3cb1ac4-aa9b-4dcf-bee1-7d2fecc59225 admin admin] REQUEST_METHOD: `POST` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:04:14.695431 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3cb1ac4-aa9b-4dcf-bee1-7d2fecc59225 admin admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:04:14.695700 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3cb1ac4-aa9b-4dcf-bee1-7d2fecc59225 admin admin] PATH_INFO: `/v3/users` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:04:14.696336 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3cb1ac4-aa9b-4dcf-bee1-7d2fecc59225 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:04:14.698777 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3cb1ac4-aa9b-4dcf-bee1-7d2fecc59225 admin admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:04:14.706691 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:04:14.721168 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 1389/2774] 10.222.0.13 () {60 vars in 1144 bytes} [Wed Aug 31 10:04:14 2022] POST /identity/v3/users => generated 286 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 10:04:14.725315 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8444bfdd-490d-4250-8765-b33d7092bc3b admin admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:04:14.737254 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8444bfdd-490d-4250-8765-b33d7092bc3b admin admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:04:14.742371 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b750e68-325a-46c1-afe6-2119dae02801 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:04:14.742968 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b750e68-325a-46c1-afe6-2119dae02801 admin admin] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:04:14.743264 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b750e68-325a-46c1-afe6-2119dae02801 admin admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:04:14.743514 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b750e68-325a-46c1-afe6-2119dae02801 admin admin] PATH_INFO: `/v3/roles` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:04:14.744156 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b750e68-325a-46c1-afe6-2119dae02801 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:04:14.746389 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b750e68-325a-46c1-afe6-2119dae02801 admin admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:04:14.756944 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 1386/2775] 10.222.0.13 () {58 vars in 1122 bytes} [Wed Aug 31 10:04:14 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 10:04:14.761651 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3cb1ac4-aa9b-4dcf-bee1-7d2fecc59225 admin admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:04:14.771785 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3cb1ac4-aa9b-4dcf-bee1-7d2fecc59225 admin admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:04:14.777319 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-358d3652-a2b7-42fc-a379-094a923bf5d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:04:14.777927 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358d3652-a2b7-42fc-a379-094a923bf5d5 admin admin] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:04:14.778201 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358d3652-a2b7-42fc-a379-094a923bf5d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:04:14.778469 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358d3652-a2b7-42fc-a379-094a923bf5d5 admin admin] PATH_INFO: `/v3/roles` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:04:14.779108 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-358d3652-a2b7-42fc-a379-094a923bf5d5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:04:14.780673 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-358d3652-a2b7-42fc-a379-094a923bf5d5 admin admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:04:14.791182 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 1390/2776] 10.222.0.13 () {58 vars in 1122 bytes} [Wed Aug 31 10:04:14 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 10:04:14.795318 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b750e68-325a-46c1-afe6-2119dae02801 admin admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:04:14.805209 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b750e68-325a-46c1-afe6-2119dae02801 admin admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:04:14.810415 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4066dd28-c2b5-49c6-ace1-dee49662b1ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:04:14.811059 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4066dd28-c2b5-49c6-ace1-dee49662b1ed admin admin] REQUEST_METHOD: `PUT` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:04:14.811335 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4066dd28-c2b5-49c6-ace1-dee49662b1ed admin admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:04:14.811604 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4066dd28-c2b5-49c6-ace1-dee49662b1ed admin admin] PATH_INFO: `/v3/projects/afa99c4bd4f848d2b54c1a20dd82ebea/users/6cf128be7aa140d6a53736b605956935/roles/6c99f5d9f1f243eaa134d8540f21095d` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:04:14.827973 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4066dd28-c2b5-49c6-ace1-dee49662b1ed admin admin] RBAC: Authorizing `identity:create_grant(project_id=afa99c4bd4f848d2b54c1a20dd82ebea, user_id=6cf128be7aa140d6a53736b605956935, role_id=6c99f5d9f1f243eaa134d8540f21095d)` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:04:14.830679 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4066dd28-c2b5-49c6-ace1-dee49662b1ed admin admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:04:14.844356 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 1387/2777] 10.222.0.13 () {60 vars in 1483 bytes} [Wed Aug 31 10:04:14 2022] PUT /identity/v3/projects/afa99c4bd4f848d2b54c1a20dd82ebea/users/6cf128be7aa140d6a53736b605956935/roles/6c99f5d9f1f243eaa134d8540f21095d => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 10:04:14.854974 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10315d7f-b911-4411-a388-cf4c8fc5cb47 None None] REQUEST_METHOD: `POST` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:04:14.854974 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10315d7f-b911-4411-a388-cf4c8fc5cb47 None None] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:04:14.855319 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10315d7f-b911-4411-a388-cf4c8fc5cb47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:04:14.883176 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.auth.core [None req-10315d7f-b911-4411-a388-cf4c8fc5cb47 None None] MFA Rules not processed for user `6cf128be7aa140d6a53736b605956935`. Rule list: `[]` (Enabled: `True`). {{(pid=88114) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 10:04:14.884386 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.fernet_utils [None req-10315d7f-b911-4411-a388-cf4c8fc5cb47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88114) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:04:14.950701 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 1391/2778] 10.222.0.13 () {58 vars in 958 bytes} [Wed Aug 31 10:04:14 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 10:04:14.955297 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4066dd28-c2b5-49c6-ace1-dee49662b1ed admin admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:04:15.022599 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4066dd28-c2b5-49c6-ace1-dee49662b1ed admin admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:04:15.027679 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a273416-4325-41c3-a1de-9d274d099ac4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:04:15.028280 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a273416-4325-41c3-a1de-9d274d099ac4 admin admin] REQUEST_METHOD: `POST` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:04:15.028558 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a273416-4325-41c3-a1de-9d274d099ac4 admin admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:04:15.028829 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a273416-4325-41c3-a1de-9d274d099ac4 admin admin] PATH_INFO: `/v3/projects` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:04:15.029506 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a273416-4325-41c3-a1de-9d274d099ac4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:04:15.031401 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a273416-4325-41c3-a1de-9d274d099ac4 admin admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:04:15.067165 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 1388/2779] 10.222.0.13 () {60 vars in 1153 bytes} [Wed Aug 31 10:04:14 2022] POST /identity/v3/projects => generated 371 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 10:04:15.072019 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10315d7f-b911-4411-a388-cf4c8fc5cb47 None None] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:04:15.085460 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10315d7f-b911-4411-a388-cf4c8fc5cb47 None None] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:04:15.149071 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcedd221-5956-4476-9ef8-d1dc044634ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:04:15.149786 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcedd221-5956-4476-9ef8-d1dc044634ff admin admin] REQUEST_METHOD: `POST` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:04:15.150061 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcedd221-5956-4476-9ef8-d1dc044634ff admin admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:04:15.150333 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcedd221-5956-4476-9ef8-d1dc044634ff admin admin] PATH_INFO: `/v3/users` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:04:15.150963 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcedd221-5956-4476-9ef8-d1dc044634ff admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:04:15.153984 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcedd221-5956-4476-9ef8-d1dc044634ff admin admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:04:15.177725 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 1392/2780] 10.222.0.13 () {60 vars in 1144 bytes} [Wed Aug 31 10:04:15 2022] POST /identity/v3/users => generated 292 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 10:04:15.182003 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a273416-4325-41c3-a1de-9d274d099ac4 admin admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:04:15.192953 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a273416-4325-41c3-a1de-9d274d099ac4 admin admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:04:15.197550 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46461bbd-d619-4d9f-b1fb-a8a228b39a3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:04:15.198201 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46461bbd-d619-4d9f-b1fb-a8a228b39a3f admin admin] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:04:15.198477 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46461bbd-d619-4d9f-b1fb-a8a228b39a3f admin admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:04:15.198745 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46461bbd-d619-4d9f-b1fb-a8a228b39a3f admin admin] PATH_INFO: `/v3/roles` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:04:15.199381 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46461bbd-d619-4d9f-b1fb-a8a228b39a3f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:04:15.200960 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46461bbd-d619-4d9f-b1fb-a8a228b39a3f admin admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:04:15.211495 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 1389/2781] 10.222.0.13 () {58 vars in 1122 bytes} [Wed Aug 31 10:04:15 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 10:04:15.216767 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcedd221-5956-4476-9ef8-d1dc044634ff admin admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:04:15.226728 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcedd221-5956-4476-9ef8-d1dc044634ff admin admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:04:15.232203 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39514bb8-f96c-4593-a3c1-b6fdbb138289 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:04:15.232900 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39514bb8-f96c-4593-a3c1-b6fdbb138289 admin admin] REQUEST_METHOD: `PUT` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:04:15.233199 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39514bb8-f96c-4593-a3c1-b6fdbb138289 admin admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:04:15.233503 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39514bb8-f96c-4593-a3c1-b6fdbb138289 admin admin] PATH_INFO: `/v3/projects/3671d2bceeea4f1abf3a5b112bae3bd0/users/ac1c4af3baee4a5895ee1e8080f4081c/roles/5d4ee63a3add47fc9a6b8b16d39e0a1f` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:04:15.248825 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39514bb8-f96c-4593-a3c1-b6fdbb138289 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3671d2bceeea4f1abf3a5b112bae3bd0, user_id=ac1c4af3baee4a5895ee1e8080f4081c, role_id=5d4ee63a3add47fc9a6b8b16d39e0a1f)` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:04:15.251473 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39514bb8-f96c-4593-a3c1-b6fdbb138289 admin admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:04:15.264594 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 1393/2782] 10.222.0.13 () {60 vars in 1483 bytes} [Wed Aug 31 10:04:15 2022] PUT /identity/v3/projects/3671d2bceeea4f1abf3a5b112bae3bd0/users/ac1c4af3baee4a5895ee1e8080f4081c/roles/5d4ee63a3add47fc9a6b8b16d39e0a1f => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 10:04:15.268690 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46461bbd-d619-4d9f-b1fb-a8a228b39a3f admin admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:04:15.279585 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46461bbd-d619-4d9f-b1fb-a8a228b39a3f admin admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:04:15.334864 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4590e219-1323-4191-83d0-4506c42a0f0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:04:15.335576 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4590e219-1323-4191-83d0-4506c42a0f0b admin admin] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:04:15.335858 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4590e219-1323-4191-83d0-4506c42a0f0b admin admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:04:15.336164 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4590e219-1323-4191-83d0-4506c42a0f0b admin admin] PATH_INFO: `/v3/roles` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:04:15.336811 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4590e219-1323-4191-83d0-4506c42a0f0b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:04:15.338502 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4590e219-1323-4191-83d0-4506c42a0f0b admin admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:04:15.349076 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 1390/2783] 10.222.0.13 () {58 vars in 1122 bytes} [Wed Aug 31 10:04:15 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 10:04:15.353652 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39514bb8-f96c-4593-a3c1-b6fdbb138289 admin admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:04:15.364275 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39514bb8-f96c-4593-a3c1-b6fdbb138289 admin admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:04:15.368858 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bddbfa4-39f9-443f-bcd3-8a1030d15706 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:04:15.369533 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bddbfa4-39f9-443f-bcd3-8a1030d15706 admin admin] REQUEST_METHOD: `PUT` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:04:15.369806 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bddbfa4-39f9-443f-bcd3-8a1030d15706 admin admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:04:15.370073 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bddbfa4-39f9-443f-bcd3-8a1030d15706 admin admin] PATH_INFO: `/v3/projects/3671d2bceeea4f1abf3a5b112bae3bd0/users/ac1c4af3baee4a5895ee1e8080f4081c/roles/5d4ee63a3add47fc9a6b8b16d39e0a1f` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:04:15.371997 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bddbfa4-39f9-443f-bcd3-8a1030d15706 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3671d2bceeea4f1abf3a5b112bae3bd0, user_id=ac1c4af3baee4a5895ee1e8080f4081c, role_id=5d4ee63a3add47fc9a6b8b16d39e0a1f)` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:04:15.374631 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bddbfa4-39f9-443f-bcd3-8a1030d15706 admin admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:04:15.384884 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 1394/2784] 10.222.0.13 () {60 vars in 1483 bytes} [Wed Aug 31 10:04:15 2022] PUT /identity/v3/projects/3671d2bceeea4f1abf3a5b112bae3bd0/users/ac1c4af3baee4a5895ee1e8080f4081c/roles/5d4ee63a3add47fc9a6b8b16d39e0a1f => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 10:04:15.395840 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e542387-7a4d-42c0-9daf-5e170983f77f None None] REQUEST_METHOD: `POST` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:04:15.396127 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e542387-7a4d-42c0-9daf-5e170983f77f None None] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:04:15.396399 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e542387-7a4d-42c0-9daf-5e170983f77f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:04:15.424869 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.auth.core [None req-9e542387-7a4d-42c0-9daf-5e170983f77f None None] MFA Rules not processed for user `ac1c4af3baee4a5895ee1e8080f4081c`. Rule list: `[]` (Enabled: `True`). {{(pid=88113) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 10:04:15.426124 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.fernet_utils [None req-9e542387-7a4d-42c0-9daf-5e170983f77f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:04:15.444378 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:04:15.496048 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 1391/2785] 10.222.0.13 () {58 vars in 958 bytes} [Wed Aug 31 10:04:15 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 10:04:15.507361 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bddbfa4-39f9-443f-bcd3-8a1030d15706 admin admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:04:15.509057 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.fernet_utils [None req-4bddbfa4-39f9-443f-bcd3-8a1030d15706 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88114) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:04:15.532661 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:INSERT=8,keystone:SELECT=92 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:04:15.568173 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bddbfa4-39f9-443f-bcd3-8a1030d15706 admin admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:04:15.571795 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93783bfd-7f5a-4976-ba37-5799e9d3494e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d52b69c8e94441dacfcff944326f169', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '49548eb44d62415582af820ab66decee', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:04:15.572389 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93783bfd-7f5a-4976-ba37-5799e9d3494e service cinder] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:04:15.572659 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93783bfd-7f5a-4976-ba37-5799e9d3494e service cinder] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:04:15.572922 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93783bfd-7f5a-4976-ba37-5799e9d3494e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:04:15.585375 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93783bfd-7f5a-4976-ba37-5799e9d3494e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:04:15.588179 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93783bfd-7f5a-4976-ba37-5799e9d3494e service cinder] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:04:15.601076 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 1395/2786] 10.222.0.13 () {60 vars in 1365 bytes} [Wed Aug 31 10:04:15 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 10:04:15.677661 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e542387-7a4d-42c0-9daf-5e170983f77f None None] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:04:15.687915 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e542387-7a4d-42c0-9daf-5e170983f77f None None] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:04:15.691500 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da20602-c46f-43c7-86a5-c8f523bce393 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d52b69c8e94441dacfcff944326f169', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '49548eb44d62415582af820ab66decee', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:04:15.692098 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da20602-c46f-43c7-86a5-c8f523bce393 service cinder] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:04:15.692375 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da20602-c46f-43c7-86a5-c8f523bce393 service cinder] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:04:15.692644 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da20602-c46f-43c7-86a5-c8f523bce393 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:04:15.753537 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7da20602-c46f-43c7-86a5-c8f523bce393 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:04:15.755125 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7da20602-c46f-43c7-86a5-c8f523bce393 service cinder] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:04:15.764848 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 1392/2787] 10.222.0.13 () {60 vars in 1365 bytes} [Wed Aug 31 10:04:15 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 10:04:25.596605 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=17 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:04:25.760063 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=48 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:05:00.418785 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93783bfd-7f5a-4976-ba37-5799e9d3494e service cinder] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:05:00.481867 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93783bfd-7f5a-4976-ba37-5799e9d3494e service cinder] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:05:00.487675 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cfde193-3311-448e-9a84-b15a8279c42e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:05:00.488236 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cfde193-3311-448e-9a84-b15a8279c42e admin admin] REQUEST_METHOD: `DELETE` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:05:00.488596 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cfde193-3311-448e-9a84-b15a8279c42e admin admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:05:00.488802 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cfde193-3311-448e-9a84-b15a8279c42e admin admin] PATH_INFO: `/v3/users/6cf128be7aa140d6a53736b605956935` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:05:00.489939 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cfde193-3311-448e-9a84-b15a8279c42e admin admin] RBAC: Authorizing `identity:delete_user(user_id=6cf128be7aa140d6a53736b605956935)` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:05:00.491843 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cfde193-3311-448e-9a84-b15a8279c42e admin admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:05:00.539367 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.notifications [None req-9cfde193-3311-448e-9a84-b15a8279c42e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6cf128be7aa140d6a53736b605956935'} {{(pid=88114) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 10:05:00.547041 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.notifications [None req-9cfde193-3311-448e-9a84-b15a8279c42e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6cf128be7aa140d6a53736b605956935'} {{(pid=88114) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 10:05:00.554156 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.notifications [None req-9cfde193-3311-448e-9a84-b15a8279c42e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6cf128be7aa140d6a53736b605956935'} {{(pid=88114) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 10:05:00.554843 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.notifications [None req-9cfde193-3311-448e-9a84-b15a8279c42e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6cf128be7aa140d6a53736b605956935'} {{(pid=88114) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 10:05:00.569527 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.notifications [None req-9cfde193-3311-448e-9a84-b15a8279c42e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6cf128be7aa140d6a53736b605956935'} {{(pid=88114) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 10:05:00.575834 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 1396/2788] 10.222.0.13 () {58 vars in 1224 bytes} [Wed Aug 31 10:05:00 2022] DELETE /identity/v3/users/6cf128be7aa140d6a53736b605956935 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 10:05:00.581333 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142cec29-c90e-416b-a305-e1de639881c1 None None] REQUEST_METHOD: `POST` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:05:00.581877 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142cec29-c90e-416b-a305-e1de639881c1 None None] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:05:00.581999 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142cec29-c90e-416b-a305-e1de639881c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:05:00.601931 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.auth.core [None req-142cec29-c90e-416b-a305-e1de639881c1 None None] MFA Rules not processed for user `fd836cf590a145df9acf19cf3ae2ac56`. Rule list: `[]` (Enabled: `True`). {{(pid=88113) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 10:05:00.603127 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.fernet_utils [None req-142cec29-c90e-416b-a305-e1de639881c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:05:00.662220 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 1393/2789] 10.222.0.13 () {58 vars in 958 bytes} [Wed Aug 31 10:05:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 10:05:00.672285 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cfde193-3311-448e-9a84-b15a8279c42e admin admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:05:00.673791 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.fernet_utils [None req-9cfde193-3311-448e-9a84-b15a8279c42e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88114) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:05:00.732310 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cfde193-3311-448e-9a84-b15a8279c42e admin admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:05:00.736843 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db5b2b22-9f3e-4d7f-9679-4f57a95be733 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9bdaacd597442c2bcde2c555f2cc140', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '49548eb44d62415582af820ab66decee', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:05:00.737469 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5b2b22-9f3e-4d7f-9679-4f57a95be733 service neutron] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:05:00.737798 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5b2b22-9f3e-4d7f-9679-4f57a95be733 service neutron] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:05:00.738017 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5b2b22-9f3e-4d7f-9679-4f57a95be733 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:05:00.747153 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db5b2b22-9f3e-4d7f-9679-4f57a95be733 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:05:00.748791 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db5b2b22-9f3e-4d7f-9679-4f57a95be733 service neutron] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:05:00.758579 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 1397/2790] 10.222.0.13 () {60 vars in 1365 bytes} [Wed Aug 31 10:05:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 10:05:01.039544 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-142cec29-c90e-416b-a305-e1de639881c1 None None] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:05:01.040977 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.fernet_utils [None req-142cec29-c90e-416b-a305-e1de639881c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:05:01.055764 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-142cec29-c90e-416b-a305-e1de639881c1 None None] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:05:01.061452 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-213f027c-ec98-4368-92f7-df1c5eb180c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:05:01.063538 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213f027c-ec98-4368-92f7-df1c5eb180c0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:05:01.064062 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213f027c-ec98-4368-92f7-df1c5eb180c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:05:01.064659 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213f027c-ec98-4368-92f7-df1c5eb180c0 admin admin] PATH_INFO: `/v3/projects/afa99c4bd4f848d2b54c1a20dd82ebea` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:05:01.066141 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-213f027c-ec98-4368-92f7-df1c5eb180c0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=afa99c4bd4f848d2b54c1a20dd82ebea)` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:05:01.068245 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-213f027c-ec98-4368-92f7-df1c5eb180c0 admin admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:05:01.119147 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.notifications [None req-213f027c-ec98-4368-92f7-df1c5eb180c0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'afa99c4bd4f848d2b54c1a20dd82ebea', 'request_id': 'req-213f027c-ec98-4368-92f7-df1c5eb180c0'} {{(pid=88113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 10:05:01.124604 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.notifications [None req-213f027c-ec98-4368-92f7-df1c5eb180c0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'afa99c4bd4f848d2b54c1a20dd82ebea', 'request_id': 'req-213f027c-ec98-4368-92f7-df1c5eb180c0'} {{(pid=88113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 10:05:01.132287 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.notifications [None req-213f027c-ec98-4368-92f7-df1c5eb180c0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'afa99c4bd4f848d2b54c1a20dd82ebea', 'request_id': 'req-213f027c-ec98-4368-92f7-df1c5eb180c0'} {{(pid=88113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 10:05:01.144011 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.notifications [None req-213f027c-ec98-4368-92f7-df1c5eb180c0 admin admin] The token cache is being invalidate because project afa99c4bd4f848d2b54c1a20dd82ebea was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88113) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 10:05:01.144284 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.notifications [None req-213f027c-ec98-4368-92f7-df1c5eb180c0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 10:05:01.144921 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.notifications [None req-213f027c-ec98-4368-92f7-df1c5eb180c0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 10:05:01.146474 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 1394/2791] 10.222.0.13 () {58 vars in 1233 bytes} [Wed Aug 31 10:05:01 2022] DELETE /identity/v3/projects/afa99c4bd4f848d2b54c1a20dd82ebea => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 10:05:01.150453 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db5b2b22-9f3e-4d7f-9679-4f57a95be733 service neutron] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:05:01.160717 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db5b2b22-9f3e-4d7f-9679-4f57a95be733 service neutron] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:05:01.179420 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:05:01.218903 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa932a57-b56c-4d46-a0aa-96c20acea3e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:05:01.219537 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa932a57-b56c-4d46-a0aa-96c20acea3e0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:05:01.219840 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa932a57-b56c-4d46-a0aa-96c20acea3e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:05:01.220105 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa932a57-b56c-4d46-a0aa-96c20acea3e0 admin admin] PATH_INFO: `/v3/users/ac1c4af3baee4a5895ee1e8080f4081c` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:05:01.221264 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa932a57-b56c-4d46-a0aa-96c20acea3e0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ac1c4af3baee4a5895ee1e8080f4081c)` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:05:01.223253 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa932a57-b56c-4d46-a0aa-96c20acea3e0 admin admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:05:01.269786 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.notifications [None req-aa932a57-b56c-4d46-a0aa-96c20acea3e0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ac1c4af3baee4a5895ee1e8080f4081c'} {{(pid=88114) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 10:05:01.276644 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.notifications [None req-aa932a57-b56c-4d46-a0aa-96c20acea3e0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ac1c4af3baee4a5895ee1e8080f4081c'} {{(pid=88114) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 10:05:01.283715 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.notifications [None req-aa932a57-b56c-4d46-a0aa-96c20acea3e0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ac1c4af3baee4a5895ee1e8080f4081c'} {{(pid=88114) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 10:05:01.284289 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.notifications [None req-aa932a57-b56c-4d46-a0aa-96c20acea3e0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ac1c4af3baee4a5895ee1e8080f4081c'} {{(pid=88114) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 10:05:01.297798 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.notifications [None req-aa932a57-b56c-4d46-a0aa-96c20acea3e0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ac1c4af3baee4a5895ee1e8080f4081c'} {{(pid=88114) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 10:05:01.303924 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 1398/2792] 10.222.0.13 () {58 vars in 1224 bytes} [Wed Aug 31 10:05:01 2022] DELETE /identity/v3/users/ac1c4af3baee4a5895ee1e8080f4081c => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 10:05:01.573598 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-213f027c-ec98-4368-92f7-df1c5eb180c0 admin admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:05:01.575054 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.fernet_utils [None req-213f027c-ec98-4368-92f7-df1c5eb180c0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:05:01.591971 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-213f027c-ec98-4368-92f7-df1c5eb180c0 admin admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:05:01.651630 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc076c6-9607-43f7-9262-6fe334180fdd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:05:01.652259 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc076c6-9607-43f7-9262-6fe334180fdd admin admin] REQUEST_METHOD: `DELETE` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:05:01.652535 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc076c6-9607-43f7-9262-6fe334180fdd admin admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:05:01.652805 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc076c6-9607-43f7-9262-6fe334180fdd admin admin] PATH_INFO: `/v3/projects/3671d2bceeea4f1abf3a5b112bae3bd0` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:05:01.654062 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbc076c6-9607-43f7-9262-6fe334180fdd admin admin] RBAC: Authorizing `identity:delete_project(project_id=3671d2bceeea4f1abf3a5b112bae3bd0)` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:05:01.656884 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbc076c6-9607-43f7-9262-6fe334180fdd admin admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:05:01.687704 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:05:01.706353 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.notifications [None req-cbc076c6-9607-43f7-9262-6fe334180fdd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3671d2bceeea4f1abf3a5b112bae3bd0', 'request_id': 'req-cbc076c6-9607-43f7-9262-6fe334180fdd'} {{(pid=88113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 10:05:01.711857 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.notifications [None req-cbc076c6-9607-43f7-9262-6fe334180fdd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3671d2bceeea4f1abf3a5b112bae3bd0', 'request_id': 'req-cbc076c6-9607-43f7-9262-6fe334180fdd'} {{(pid=88113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 10:05:01.719281 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.notifications [None req-cbc076c6-9607-43f7-9262-6fe334180fdd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3671d2bceeea4f1abf3a5b112bae3bd0', 'request_id': 'req-cbc076c6-9607-43f7-9262-6fe334180fdd'} {{(pid=88113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 10:05:01.731120 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.notifications [None req-cbc076c6-9607-43f7-9262-6fe334180fdd admin admin] The token cache is being invalidate because project 3671d2bceeea4f1abf3a5b112bae3bd0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88113) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 10:05:01.731405 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.notifications [None req-cbc076c6-9607-43f7-9262-6fe334180fdd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 10:05:01.732036 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.notifications [None req-cbc076c6-9607-43f7-9262-6fe334180fdd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 10:05:01.733644 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 1395/2793] 10.222.0.13 () {58 vars in 1233 bytes} [Wed Aug 31 10:05:01 2022] DELETE /identity/v3/projects/3671d2bceeea4f1abf3a5b112bae3bd0 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 10:05:01.738741 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad0d958-2f68-4184-8df8-34957baa3b5a None None] REQUEST_METHOD: `POST` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:05:01.739226 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad0d958-2f68-4184-8df8-34957baa3b5a None None] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:05:01.739343 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad0d958-2f68-4184-8df8-34957baa3b5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:05:01.758859 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.auth.core [None req-3ad0d958-2f68-4184-8df8-34957baa3b5a None None] MFA Rules not processed for user `fd836cf590a145df9acf19cf3ae2ac56`. Rule list: `[]` (Enabled: `True`). {{(pid=88114) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 10:05:01.760043 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.fernet_utils [None req-3ad0d958-2f68-4184-8df8-34957baa3b5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88114) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:05:01.830924 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 1399/2794] 10.222.0.13 () {58 vars in 958 bytes} [Wed Aug 31 10:05:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 10:05:01.838709 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32156b4c-1811-45db-8835-554d8d9bee43 None None] REQUEST_METHOD: `POST` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:05:01.838994 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32156b4c-1811-45db-8835-554d8d9bee43 None None] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:05:01.839294 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32156b4c-1811-45db-8835-554d8d9bee43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:05:01.860849 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.auth.core [None req-32156b4c-1811-45db-8835-554d8d9bee43 None None] MFA Rules not processed for user `fd836cf590a145df9acf19cf3ae2ac56`. Rule list: `[]` (Enabled: `True`). {{(pid=88113) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 10:05:01.862338 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.fernet_utils [None req-32156b4c-1811-45db-8835-554d8d9bee43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:05:01.870209 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 1396/2795] 10.222.0.13 () {58 vars in 958 bytes} [Wed Aug 31 10:05:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 10:05:01.874941 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad0d958-2f68-4184-8df8-34957baa3b5a None None] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:05:01.884948 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad0d958-2f68-4184-8df8-34957baa3b5a None None] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:05:01.889655 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a9fb750-8cd4-405e-9181-2a1d320f7ccd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:05:01.890615 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9fb750-8cd4-405e-9181-2a1d320f7ccd admin admin] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:05:01.891009 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9fb750-8cd4-405e-9181-2a1d320f7ccd admin admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:05:01.891284 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9fb750-8cd4-405e-9181-2a1d320f7ccd admin admin] PATH_INFO: `/v3/domains` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:05:01.891959 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a9fb750-8cd4-405e-9181-2a1d320f7ccd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88114) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:05:01.892261 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a9fb750-8cd4-405e-9181-2a1d320f7ccd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:05:01.893894 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a9fb750-8cd4-405e-9181-2a1d320f7ccd admin admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:05:01.906997 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 1400/2796] 10.222.0.13 () {58 vars in 1153 bytes} [Wed Aug 31 10:05:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 10:05:01.911530 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32156b4c-1811-45db-8835-554d8d9bee43 None None] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:05:01.921820 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32156b4c-1811-45db-8835-554d8d9bee43 None None] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:05:01.926343 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53389167-91c2-4639-8f75-38bde6fe9b0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:05:01.926929 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53389167-91c2-4639-8f75-38bde6fe9b0e admin admin] REQUEST_METHOD: `POST` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:05:01.927202 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53389167-91c2-4639-8f75-38bde6fe9b0e admin admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:05:01.927464 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53389167-91c2-4639-8f75-38bde6fe9b0e admin admin] PATH_INFO: `/v3/projects` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:05:01.928080 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53389167-91c2-4639-8f75-38bde6fe9b0e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:05:01.930007 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53389167-91c2-4639-8f75-38bde6fe9b0e admin admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:05:01.957317 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 1397/2797] 10.222.0.13 () {60 vars in 1153 bytes} [Wed Aug 31 10:05:01 2022] POST /identity/v3/projects => generated 375 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 10:05:01.961706 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a9fb750-8cd4-405e-9181-2a1d320f7ccd admin admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:05:01.972387 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a9fb750-8cd4-405e-9181-2a1d320f7ccd admin admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:05:02.000797 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:05:02.033503 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b86ca2-da71-4478-bd10-4114b6d920f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:05:02.034104 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b86ca2-da71-4478-bd10-4114b6d920f3 admin admin] REQUEST_METHOD: `POST` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:05:02.034381 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b86ca2-da71-4478-bd10-4114b6d920f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:05:02.034651 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b86ca2-da71-4478-bd10-4114b6d920f3 admin admin] PATH_INFO: `/v3/users` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:05:02.035286 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89b86ca2-da71-4478-bd10-4114b6d920f3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:05:02.037184 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89b86ca2-da71-4478-bd10-4114b6d920f3 admin admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:05:02.059684 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 1401/2798] 10.222.0.13 () {60 vars in 1144 bytes} [Wed Aug 31 10:05:01 2022] POST /identity/v3/users => generated 288 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 10:05:02.065020 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53389167-91c2-4639-8f75-38bde6fe9b0e admin admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:05:02.075474 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53389167-91c2-4639-8f75-38bde6fe9b0e admin admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:05:02.080842 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f288d5-6bbd-49fd-a07f-69eb02a765ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:05:02.081526 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f288d5-6bbd-49fd-a07f-69eb02a765ab admin admin] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:05:02.081746 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f288d5-6bbd-49fd-a07f-69eb02a765ab admin admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:05:02.082014 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f288d5-6bbd-49fd-a07f-69eb02a765ab admin admin] PATH_INFO: `/v3/roles` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:05:02.082659 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1f288d5-6bbd-49fd-a07f-69eb02a765ab admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:05:02.084249 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1f288d5-6bbd-49fd-a07f-69eb02a765ab admin admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:05:02.095491 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 1398/2799] 10.222.0.13 () {58 vars in 1122 bytes} [Wed Aug 31 10:05:02 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 10:05:02.099863 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b86ca2-da71-4478-bd10-4114b6d920f3 admin admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:05:02.109968 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b86ca2-da71-4478-bd10-4114b6d920f3 admin admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:05:02.114950 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb54ee5-fe41-4a1e-96f9-5bf82735645a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:05:02.115636 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb54ee5-fe41-4a1e-96f9-5bf82735645a admin admin] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:05:02.115913 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb54ee5-fe41-4a1e-96f9-5bf82735645a admin admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:05:02.116181 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb54ee5-fe41-4a1e-96f9-5bf82735645a admin admin] PATH_INFO: `/v3/roles` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:05:02.116813 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddb54ee5-fe41-4a1e-96f9-5bf82735645a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:05:02.118425 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddb54ee5-fe41-4a1e-96f9-5bf82735645a admin admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:05:02.130105 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 1402/2800] 10.222.0.13 () {58 vars in 1122 bytes} [Wed Aug 31 10:05:02 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 10:05:02.134462 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f288d5-6bbd-49fd-a07f-69eb02a765ab admin admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:05:02.145090 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f288d5-6bbd-49fd-a07f-69eb02a765ab admin admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:05:02.149770 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cfdc54d-45cf-4188-abff-03a668756167 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:05:02.150413 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cfdc54d-45cf-4188-abff-03a668756167 admin admin] REQUEST_METHOD: `PUT` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:05:02.150693 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cfdc54d-45cf-4188-abff-03a668756167 admin admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:05:02.150976 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cfdc54d-45cf-4188-abff-03a668756167 admin admin] PATH_INFO: `/v3/projects/a2325611e0524838b621b6b1cb629674/users/6a4b56d389b0439cb993a455501db32f/roles/6c99f5d9f1f243eaa134d8540f21095d` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:05:02.166369 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cfdc54d-45cf-4188-abff-03a668756167 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a2325611e0524838b621b6b1cb629674, user_id=6a4b56d389b0439cb993a455501db32f, role_id=6c99f5d9f1f243eaa134d8540f21095d)` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:05:02.169034 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cfdc54d-45cf-4188-abff-03a668756167 admin admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:05:02.183342 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 1399/2801] 10.222.0.13 () {60 vars in 1483 bytes} [Wed Aug 31 10:05:02 2022] PUT /identity/v3/projects/a2325611e0524838b621b6b1cb629674/users/6a4b56d389b0439cb993a455501db32f/roles/6c99f5d9f1f243eaa134d8540f21095d => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 10:05:02.195445 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480b6b66-5c6d-4015-ae5c-96d807642be3 None None] REQUEST_METHOD: `POST` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:05:02.195780 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480b6b66-5c6d-4015-ae5c-96d807642be3 None None] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:05:02.196061 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480b6b66-5c6d-4015-ae5c-96d807642be3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:05:02.223259 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.auth.core [None req-480b6b66-5c6d-4015-ae5c-96d807642be3 None None] MFA Rules not processed for user `6a4b56d389b0439cb993a455501db32f`. Rule list: `[]` (Enabled: `True`). {{(pid=88114) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 10:05:02.224467 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.fernet_utils [None req-480b6b66-5c6d-4015-ae5c-96d807642be3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88114) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:05:02.289868 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 1403/2802] 10.222.0.13 () {58 vars in 958 bytes} [Wed Aug 31 10:05:02 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 10:05:02.294287 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cfdc54d-45cf-4188-abff-03a668756167 admin admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:05:02.358862 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cfdc54d-45cf-4188-abff-03a668756167 admin admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:05:02.363764 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc06bd4-f9a8-4ad3-b1e1-80accee48395 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:05:02.364367 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc06bd4-f9a8-4ad3-b1e1-80accee48395 admin admin] REQUEST_METHOD: `POST` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:05:02.364646 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc06bd4-f9a8-4ad3-b1e1-80accee48395 admin admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:05:02.364920 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc06bd4-f9a8-4ad3-b1e1-80accee48395 admin admin] PATH_INFO: `/v3/projects` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:05:02.365581 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dc06bd4-f9a8-4ad3-b1e1-80accee48395 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:05:02.367488 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dc06bd4-f9a8-4ad3-b1e1-80accee48395 admin admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:05:02.396463 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 1400/2803] 10.222.0.13 () {60 vars in 1153 bytes} [Wed Aug 31 10:05:02 2022] POST /identity/v3/projects => generated 375 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 10:05:02.400712 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-480b6b66-5c6d-4015-ae5c-96d807642be3 None None] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:05:02.413555 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-480b6b66-5c6d-4015-ae5c-96d807642be3 None None] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:05:02.470762 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf27e7dd-3eac-4798-abc5-9e429a001b17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:05:02.471487 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf27e7dd-3eac-4798-abc5-9e429a001b17 admin admin] REQUEST_METHOD: `POST` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:05:02.471766 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf27e7dd-3eac-4798-abc5-9e429a001b17 admin admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:05:02.472035 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf27e7dd-3eac-4798-abc5-9e429a001b17 admin admin] PATH_INFO: `/v3/users` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:05:02.472667 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf27e7dd-3eac-4798-abc5-9e429a001b17 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:05:02.474607 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf27e7dd-3eac-4798-abc5-9e429a001b17 admin admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:05:02.497913 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 1404/2804] 10.222.0.13 () {60 vars in 1144 bytes} [Wed Aug 31 10:05:02 2022] POST /identity/v3/users => generated 294 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 10:05:02.503091 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc06bd4-f9a8-4ad3-b1e1-80accee48395 admin admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:05:02.513472 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc06bd4-f9a8-4ad3-b1e1-80accee48395 admin admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:05:02.519305 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8fc84e0-0bb3-42a5-92fe-b49c69ae984c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:05:02.519917 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8fc84e0-0bb3-42a5-92fe-b49c69ae984c admin admin] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:05:02.520175 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8fc84e0-0bb3-42a5-92fe-b49c69ae984c admin admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:05:02.520441 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8fc84e0-0bb3-42a5-92fe-b49c69ae984c admin admin] PATH_INFO: `/v3/roles` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:05:02.521081 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8fc84e0-0bb3-42a5-92fe-b49c69ae984c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:05:02.522701 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8fc84e0-0bb3-42a5-92fe-b49c69ae984c admin admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:05:02.533728 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 1401/2805] 10.222.0.13 () {58 vars in 1122 bytes} [Wed Aug 31 10:05:02 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 10:05:02.538032 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf27e7dd-3eac-4798-abc5-9e429a001b17 admin admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:05:02.548220 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf27e7dd-3eac-4798-abc5-9e429a001b17 admin admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:05:02.553577 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f09de347-b091-416e-a483-58bf3a306df8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:05:02.554250 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09de347-b091-416e-a483-58bf3a306df8 admin admin] REQUEST_METHOD: `PUT` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:05:02.554529 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09de347-b091-416e-a483-58bf3a306df8 admin admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:05:02.554937 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09de347-b091-416e-a483-58bf3a306df8 admin admin] PATH_INFO: `/v3/projects/94dbeaf803924d2cbe1cd7b8850cbdae/users/09b4fe70a15b4e03910751ce0c00a7ec/roles/5d4ee63a3add47fc9a6b8b16d39e0a1f` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:05:02.573141 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f09de347-b091-416e-a483-58bf3a306df8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=94dbeaf803924d2cbe1cd7b8850cbdae, user_id=09b4fe70a15b4e03910751ce0c00a7ec, role_id=5d4ee63a3add47fc9a6b8b16d39e0a1f)` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:05:02.575812 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f09de347-b091-416e-a483-58bf3a306df8 admin admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:05:02.590012 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 1405/2806] 10.222.0.13 () {60 vars in 1483 bytes} [Wed Aug 31 10:05:02 2022] PUT /identity/v3/projects/94dbeaf803924d2cbe1cd7b8850cbdae/users/09b4fe70a15b4e03910751ce0c00a7ec/roles/5d4ee63a3add47fc9a6b8b16d39e0a1f => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 10:05:02.594226 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8fc84e0-0bb3-42a5-92fe-b49c69ae984c admin admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:05:02.604422 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8fc84e0-0bb3-42a5-92fe-b49c69ae984c admin admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:05:02.626059 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:05:02.665318 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7dff776-b8a6-43ff-a74e-73686269e799 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:05:02.665946 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7dff776-b8a6-43ff-a74e-73686269e799 admin admin] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:05:02.666220 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7dff776-b8a6-43ff-a74e-73686269e799 admin admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:05:02.666486 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7dff776-b8a6-43ff-a74e-73686269e799 admin admin] PATH_INFO: `/v3/roles` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:05:02.667117 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7dff776-b8a6-43ff-a74e-73686269e799 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:05:02.668691 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7dff776-b8a6-43ff-a74e-73686269e799 admin admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:05:02.679822 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 1402/2807] 10.222.0.13 () {58 vars in 1122 bytes} [Wed Aug 31 10:05:02 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 10:05:02.684264 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f09de347-b091-416e-a483-58bf3a306df8 admin admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:05:02.689939 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:05:02.696126 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f09de347-b091-416e-a483-58bf3a306df8 admin admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:05:02.701230 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a47c35e-9b91-408c-9d27-086ea5fedb5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:05:02.701978 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a47c35e-9b91-408c-9d27-086ea5fedb5d admin admin] REQUEST_METHOD: `PUT` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:05:02.702277 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a47c35e-9b91-408c-9d27-086ea5fedb5d admin admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:05:02.702550 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a47c35e-9b91-408c-9d27-086ea5fedb5d admin admin] PATH_INFO: `/v3/projects/94dbeaf803924d2cbe1cd7b8850cbdae/users/09b4fe70a15b4e03910751ce0c00a7ec/roles/5d4ee63a3add47fc9a6b8b16d39e0a1f` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:05:02.704422 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a47c35e-9b91-408c-9d27-086ea5fedb5d admin admin] RBAC: Authorizing `identity:create_grant(project_id=94dbeaf803924d2cbe1cd7b8850cbdae, user_id=09b4fe70a15b4e03910751ce0c00a7ec, role_id=5d4ee63a3add47fc9a6b8b16d39e0a1f)` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:05:02.707050 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a47c35e-9b91-408c-9d27-086ea5fedb5d admin admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:05:02.717220 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 1406/2808] 10.222.0.13 () {60 vars in 1483 bytes} [Wed Aug 31 10:05:02 2022] PUT /identity/v3/projects/94dbeaf803924d2cbe1cd7b8850cbdae/users/09b4fe70a15b4e03910751ce0c00a7ec/roles/5d4ee63a3add47fc9a6b8b16d39e0a1f => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 10:05:02.783525 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ccc3918-8ce1-4ef9-84b2-7ded303b1196 None None] REQUEST_METHOD: `POST` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:05:02.783804 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ccc3918-8ce1-4ef9-84b2-7ded303b1196 None None] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:05:02.784076 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ccc3918-8ce1-4ef9-84b2-7ded303b1196 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:05:02.813354 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.auth.core [None req-3ccc3918-8ce1-4ef9-84b2-7ded303b1196 None None] MFA Rules not processed for user `09b4fe70a15b4e03910751ce0c00a7ec`. Rule list: `[]` (Enabled: `True`). {{(pid=88113) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 10:05:02.814606 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.fernet_utils [None req-3ccc3918-8ce1-4ef9-84b2-7ded303b1196 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:05:02.885304 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 1403/2809] 10.222.0.13 () {58 vars in 958 bytes} [Wed Aug 31 10:05:02 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 10:05:02.896773 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a47c35e-9b91-408c-9d27-086ea5fedb5d admin admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:05:02.898536 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.fernet_utils [None req-0a47c35e-9b91-408c-9d27-086ea5fedb5d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88114) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:05:02.956048 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a47c35e-9b91-408c-9d27-086ea5fedb5d admin admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:05:02.959743 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-070118df-5d85-4d06-8017-b27536d004a2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d52b69c8e94441dacfcff944326f169', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '49548eb44d62415582af820ab66decee', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:05:02.960335 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070118df-5d85-4d06-8017-b27536d004a2 service cinder] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:05:02.960611 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070118df-5d85-4d06-8017-b27536d004a2 service cinder] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:05:02.960879 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070118df-5d85-4d06-8017-b27536d004a2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:05:03.018175 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-070118df-5d85-4d06-8017-b27536d004a2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:05:03.019602 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-070118df-5d85-4d06-8017-b27536d004a2 service cinder] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:05:03.028297 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 1407/2810] 10.222.0.13 () {60 vars in 1365 bytes} [Wed Aug 31 10:05:02 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 10:05:08.679110 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ccc3918-8ce1-4ef9-84b2-7ded303b1196 None None] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:05:08.680488 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.fernet_utils [None req-3ccc3918-8ce1-4ef9-84b2-7ded303b1196 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:05:08.693139 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ccc3918-8ce1-4ef9-84b2-7ded303b1196 None None] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:05:08.696656 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ebe76d-5eb2-419c-b46f-9ab941978463 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d52b69c8e94441dacfcff944326f169', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '49548eb44d62415582af820ab66decee', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:05:08.697251 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ebe76d-5eb2-419c-b46f-9ab941978463 service cinder] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:05:08.697555 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ebe76d-5eb2-419c-b46f-9ab941978463 service cinder] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:05:08.697824 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ebe76d-5eb2-419c-b46f-9ab941978463 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:05:08.707524 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4ebe76d-5eb2-419c-b46f-9ab941978463 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:05:08.708933 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4ebe76d-5eb2-419c-b46f-9ab941978463 service cinder] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:05:08.718264 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 1404/2811] 10.222.0.13 () {60 vars in 1365 bytes} [Wed Aug 31 10:05:08 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 10:05:13.024489 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=52,keystone:INSERT=1 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:05:18.714007 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=58 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:05:37.724749 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-070118df-5d85-4d06-8017-b27536d004a2 service cinder] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:05:37.794769 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-070118df-5d85-4d06-8017-b27536d004a2 service cinder] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:05:37.800585 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ee8c06-851b-4d7c-93e4-78745e9ce418 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:05:37.801215 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ee8c06-851b-4d7c-93e4-78745e9ce418 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:05:37.801567 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ee8c06-851b-4d7c-93e4-78745e9ce418 admin admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:05:37.801802 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ee8c06-851b-4d7c-93e4-78745e9ce418 admin admin] PATH_INFO: `/v3/users/6a4b56d389b0439cb993a455501db32f` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:05:37.802957 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26ee8c06-851b-4d7c-93e4-78745e9ce418 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6a4b56d389b0439cb993a455501db32f)` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:05:37.804869 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26ee8c06-851b-4d7c-93e4-78745e9ce418 admin admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:05:37.857487 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.notifications [None req-26ee8c06-851b-4d7c-93e4-78745e9ce418 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6a4b56d389b0439cb993a455501db32f'} {{(pid=88114) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 10:05:37.867414 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.notifications [None req-26ee8c06-851b-4d7c-93e4-78745e9ce418 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6a4b56d389b0439cb993a455501db32f'} {{(pid=88114) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 10:05:37.876444 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.notifications [None req-26ee8c06-851b-4d7c-93e4-78745e9ce418 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6a4b56d389b0439cb993a455501db32f'} {{(pid=88114) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 10:05:37.877071 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.notifications [None req-26ee8c06-851b-4d7c-93e4-78745e9ce418 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6a4b56d389b0439cb993a455501db32f'} {{(pid=88114) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 10:05:37.892342 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.notifications [None req-26ee8c06-851b-4d7c-93e4-78745e9ce418 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6a4b56d389b0439cb993a455501db32f'} {{(pid=88114) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 10:05:37.898757 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 1408/2812] 10.222.0.13 () {58 vars in 1224 bytes} [Wed Aug 31 10:05:37 2022] DELETE /identity/v3/users/6a4b56d389b0439cb993a455501db32f => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 10:05:37.903297 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349a0faf-c323-4895-ac7d-4915e81503b1 None None] REQUEST_METHOD: `POST` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:05:37.903963 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349a0faf-c323-4895-ac7d-4915e81503b1 None None] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:05:37.903963 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349a0faf-c323-4895-ac7d-4915e81503b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:05:37.924853 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.auth.core [None req-349a0faf-c323-4895-ac7d-4915e81503b1 None None] MFA Rules not processed for user `fd836cf590a145df9acf19cf3ae2ac56`. Rule list: `[]` (Enabled: `True`). {{(pid=88113) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 10:05:37.926832 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.fernet_utils [None req-349a0faf-c323-4895-ac7d-4915e81503b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:05:37.992286 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 1405/2813] 10.222.0.13 () {58 vars in 958 bytes} [Wed Aug 31 10:05:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 10:05:38.002657 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ee8c06-851b-4d7c-93e4-78745e9ce418 admin admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:05:38.004048 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.fernet_utils [None req-26ee8c06-851b-4d7c-93e4-78745e9ce418 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88114) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:05:38.082791 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ee8c06-851b-4d7c-93e4-78745e9ce418 admin admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:05:38.086893 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b26beca2-88a5-43c3-b094-6bec6c5851d9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9bdaacd597442c2bcde2c555f2cc140', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '49548eb44d62415582af820ab66decee', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:05:38.087509 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26beca2-88a5-43c3-b094-6bec6c5851d9 service neutron] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:05:38.087788 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26beca2-88a5-43c3-b094-6bec6c5851d9 service neutron] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:05:38.088059 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26beca2-88a5-43c3-b094-6bec6c5851d9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:05:38.098852 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b26beca2-88a5-43c3-b094-6bec6c5851d9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:05:38.100300 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b26beca2-88a5-43c3-b094-6bec6c5851d9 service neutron] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:05:38.111269 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 1409/2814] 10.222.0.13 () {60 vars in 1365 bytes} [Wed Aug 31 10:05:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 10:05:38.397122 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-349a0faf-c323-4895-ac7d-4915e81503b1 None None] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:05:38.398582 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.fernet_utils [None req-349a0faf-c323-4895-ac7d-4915e81503b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:05:38.410993 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-349a0faf-c323-4895-ac7d-4915e81503b1 None None] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:05:38.415531 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c1facca-6360-4c79-947b-b4fe4fe902e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:05:38.416167 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1facca-6360-4c79-947b-b4fe4fe902e7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:05:38.416437 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1facca-6360-4c79-947b-b4fe4fe902e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:05:38.416710 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1facca-6360-4c79-947b-b4fe4fe902e7 admin admin] PATH_INFO: `/v3/projects/a2325611e0524838b621b6b1cb629674` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:05:38.417854 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c1facca-6360-4c79-947b-b4fe4fe902e7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a2325611e0524838b621b6b1cb629674)` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:05:38.419750 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c1facca-6360-4c79-947b-b4fe4fe902e7 admin admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:05:38.469035 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.notifications [None req-6c1facca-6360-4c79-947b-b4fe4fe902e7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a2325611e0524838b621b6b1cb629674', 'request_id': 'req-6c1facca-6360-4c79-947b-b4fe4fe902e7'} {{(pid=88113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 10:05:38.474537 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.notifications [None req-6c1facca-6360-4c79-947b-b4fe4fe902e7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a2325611e0524838b621b6b1cb629674', 'request_id': 'req-6c1facca-6360-4c79-947b-b4fe4fe902e7'} {{(pid=88113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 10:05:38.481801 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.notifications [None req-6c1facca-6360-4c79-947b-b4fe4fe902e7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a2325611e0524838b621b6b1cb629674', 'request_id': 'req-6c1facca-6360-4c79-947b-b4fe4fe902e7'} {{(pid=88113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 10:05:38.493083 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.notifications [None req-6c1facca-6360-4c79-947b-b4fe4fe902e7 admin admin] The token cache is being invalidate because project a2325611e0524838b621b6b1cb629674 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88113) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 10:05:38.493367 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.notifications [None req-6c1facca-6360-4c79-947b-b4fe4fe902e7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 10:05:38.494016 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.notifications [None req-6c1facca-6360-4c79-947b-b4fe4fe902e7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 10:05:38.495578 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 1406/2815] 10.222.0.13 () {58 vars in 1233 bytes} [Wed Aug 31 10:05:38 2022] DELETE /identity/v3/projects/a2325611e0524838b621b6b1cb629674 => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 10:05:38.499720 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b26beca2-88a5-43c3-b094-6bec6c5851d9 service neutron] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:05:38.509491 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b26beca2-88a5-43c3-b094-6bec6c5851d9 service neutron] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:05:38.531350 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:05:38.570010 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df3bf5b-ca42-4605-8965-99d82689beb3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:05:38.570635 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df3bf5b-ca42-4605-8965-99d82689beb3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:05:38.570908 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df3bf5b-ca42-4605-8965-99d82689beb3 admin admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:05:38.571174 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df3bf5b-ca42-4605-8965-99d82689beb3 admin admin] PATH_INFO: `/v3/users/09b4fe70a15b4e03910751ce0c00a7ec` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:05:38.572288 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8df3bf5b-ca42-4605-8965-99d82689beb3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=09b4fe70a15b4e03910751ce0c00a7ec)` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:05:38.574212 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8df3bf5b-ca42-4605-8965-99d82689beb3 admin admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:05:38.620568 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.notifications [None req-8df3bf5b-ca42-4605-8965-99d82689beb3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '09b4fe70a15b4e03910751ce0c00a7ec'} {{(pid=88114) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 10:05:38.627705 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.notifications [None req-8df3bf5b-ca42-4605-8965-99d82689beb3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '09b4fe70a15b4e03910751ce0c00a7ec'} {{(pid=88114) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 10:05:38.635010 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.notifications [None req-8df3bf5b-ca42-4605-8965-99d82689beb3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '09b4fe70a15b4e03910751ce0c00a7ec'} {{(pid=88114) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 10:05:38.635632 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.notifications [None req-8df3bf5b-ca42-4605-8965-99d82689beb3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '09b4fe70a15b4e03910751ce0c00a7ec'} {{(pid=88114) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 10:05:38.648356 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.notifications [None req-8df3bf5b-ca42-4605-8965-99d82689beb3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '09b4fe70a15b4e03910751ce0c00a7ec'} {{(pid=88114) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 10:05:38.654244 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 1410/2816] 10.222.0.13 () {58 vars in 1224 bytes} [Wed Aug 31 10:05:38 2022] DELETE /identity/v3/users/09b4fe70a15b4e03910751ce0c00a7ec => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 10:05:38.924426 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c1facca-6360-4c79-947b-b4fe4fe902e7 admin admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:05:38.925991 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.fernet_utils [None req-6c1facca-6360-4c79-947b-b4fe4fe902e7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:05:38.939769 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c1facca-6360-4c79-947b-b4fe4fe902e7 admin admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:05:39.000122 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ef3043e-ccb5-4923-94ff-70cec7e34917 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:05:39.000756 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef3043e-ccb5-4923-94ff-70cec7e34917 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:05:39.001031 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef3043e-ccb5-4923-94ff-70cec7e34917 admin admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:05:39.001300 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef3043e-ccb5-4923-94ff-70cec7e34917 admin admin] PATH_INFO: `/v3/projects/94dbeaf803924d2cbe1cd7b8850cbdae` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:05:39.002450 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ef3043e-ccb5-4923-94ff-70cec7e34917 admin admin] RBAC: Authorizing `identity:delete_project(project_id=94dbeaf803924d2cbe1cd7b8850cbdae)` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:05:39.004348 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ef3043e-ccb5-4923-94ff-70cec7e34917 admin admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:05:39.035021 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:05:39.053285 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.notifications [None req-6ef3043e-ccb5-4923-94ff-70cec7e34917 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '94dbeaf803924d2cbe1cd7b8850cbdae', 'request_id': 'req-6ef3043e-ccb5-4923-94ff-70cec7e34917'} {{(pid=88113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 10:05:39.058768 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.notifications [None req-6ef3043e-ccb5-4923-94ff-70cec7e34917 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '94dbeaf803924d2cbe1cd7b8850cbdae', 'request_id': 'req-6ef3043e-ccb5-4923-94ff-70cec7e34917'} {{(pid=88113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 10:05:39.068111 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.notifications [None req-6ef3043e-ccb5-4923-94ff-70cec7e34917 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '94dbeaf803924d2cbe1cd7b8850cbdae', 'request_id': 'req-6ef3043e-ccb5-4923-94ff-70cec7e34917'} {{(pid=88113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 10:05:39.083482 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.notifications [None req-6ef3043e-ccb5-4923-94ff-70cec7e34917 admin admin] The token cache is being invalidate because project 94dbeaf803924d2cbe1cd7b8850cbdae was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88113) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 10:05:39.083814 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.notifications [None req-6ef3043e-ccb5-4923-94ff-70cec7e34917 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 10:05:39.084516 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.notifications [None req-6ef3043e-ccb5-4923-94ff-70cec7e34917 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 10:05:39.086108 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 1407/2817] 10.222.0.13 () {58 vars in 1233 bytes} [Wed Aug 31 10:05:38 2022] DELETE /identity/v3/projects/94dbeaf803924d2cbe1cd7b8850cbdae => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 10:05:39.091480 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5dc260-0f7d-410f-8cdc-182d18be59fc None None] REQUEST_METHOD: `POST` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:05:39.091992 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5dc260-0f7d-410f-8cdc-182d18be59fc None None] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:05:39.092149 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5dc260-0f7d-410f-8cdc-182d18be59fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:05:39.114429 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.auth.core [None req-2c5dc260-0f7d-410f-8cdc-182d18be59fc None None] MFA Rules not processed for user `fd836cf590a145df9acf19cf3ae2ac56`. Rule list: `[]` (Enabled: `True`). {{(pid=88114) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 10:05:39.115653 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.fernet_utils [None req-2c5dc260-0f7d-410f-8cdc-182d18be59fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88114) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:05:39.175653 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 1411/2818] 10.222.0.13 () {58 vars in 958 bytes} [Wed Aug 31 10:05:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 10:05:39.184264 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17af970-8ae3-490b-b932-ae8f99e86377 None None] REQUEST_METHOD: `POST` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:05:39.184553 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17af970-8ae3-490b-b932-ae8f99e86377 None None] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:05:39.184832 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17af970-8ae3-490b-b932-ae8f99e86377 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:05:39.206157 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.auth.core [None req-d17af970-8ae3-490b-b932-ae8f99e86377 None None] MFA Rules not processed for user `fd836cf590a145df9acf19cf3ae2ac56`. Rule list: `[]` (Enabled: `True`). {{(pid=88113) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 10:05:39.207425 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.fernet_utils [None req-d17af970-8ae3-490b-b932-ae8f99e86377 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:05:39.215359 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 1408/2819] 10.222.0.13 () {58 vars in 958 bytes} [Wed Aug 31 10:05:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 10:05:39.220089 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c5dc260-0f7d-410f-8cdc-182d18be59fc None None] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:05:39.229625 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c5dc260-0f7d-410f-8cdc-182d18be59fc None None] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:05:39.234557 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2afef565-0dec-44cd-abd0-82859f870abe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:05:39.235147 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2afef565-0dec-44cd-abd0-82859f870abe admin admin] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:05:39.235681 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2afef565-0dec-44cd-abd0-82859f870abe admin admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:05:39.236300 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2afef565-0dec-44cd-abd0-82859f870abe admin admin] PATH_INFO: `/v3/domains` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:05:39.236943 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2afef565-0dec-44cd-abd0-82859f870abe admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88114) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:05:39.237241 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2afef565-0dec-44cd-abd0-82859f870abe admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:05:39.238847 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2afef565-0dec-44cd-abd0-82859f870abe admin admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:05:39.250184 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 1412/2820] 10.222.0.13 () {58 vars in 1153 bytes} [Wed Aug 31 10:05:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:05:39.254429 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d17af970-8ae3-490b-b932-ae8f99e86377 None None] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:05:39.263746 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d17af970-8ae3-490b-b932-ae8f99e86377 None None] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:05:39.269433 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6811813-3d80-43bc-88e5-3f17c139224f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:05:39.270042 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6811813-3d80-43bc-88e5-3f17c139224f admin admin] REQUEST_METHOD: `POST` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:05:39.270316 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6811813-3d80-43bc-88e5-3f17c139224f admin admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:05:39.270582 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6811813-3d80-43bc-88e5-3f17c139224f admin admin] PATH_INFO: `/v3/projects` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:05:39.271216 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6811813-3d80-43bc-88e5-3f17c139224f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:05:39.273136 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6811813-3d80-43bc-88e5-3f17c139224f admin admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:05:39.303074 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 1409/2821] 10.222.0.13 () {60 vars in 1153 bytes} [Wed Aug 31 10:05:39 2022] POST /identity/v3/projects => generated 387 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 10:05:39.307321 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2afef565-0dec-44cd-abd0-82859f870abe admin admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:05:39.317402 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2afef565-0dec-44cd-abd0-82859f870abe admin admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:05:39.343889 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:05:39.375681 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c5a532e-672f-4e5f-a9dd-b4dc3be88ebc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:05:39.376266 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5a532e-672f-4e5f-a9dd-b4dc3be88ebc admin admin] REQUEST_METHOD: `POST` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:05:39.376539 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5a532e-672f-4e5f-a9dd-b4dc3be88ebc admin admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:05:39.376807 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5a532e-672f-4e5f-a9dd-b4dc3be88ebc admin admin] PATH_INFO: `/v3/users` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:05:39.377466 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c5a532e-672f-4e5f-a9dd-b4dc3be88ebc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:05:39.379383 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c5a532e-672f-4e5f-a9dd-b4dc3be88ebc admin admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:05:39.400526 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 1413/2822] 10.222.0.13 () {60 vars in 1144 bytes} [Wed Aug 31 10:05:39 2022] POST /identity/v3/users => generated 294 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 10:05:39.404666 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6811813-3d80-43bc-88e5-3f17c139224f admin admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:05:39.416088 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6811813-3d80-43bc-88e5-3f17c139224f admin admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:05:39.420911 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed453c72-9e60-4afb-9ebc-e16193e57f4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:05:39.421691 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed453c72-9e60-4afb-9ebc-e16193e57f4a admin admin] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:05:39.421967 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed453c72-9e60-4afb-9ebc-e16193e57f4a admin admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:05:39.422266 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed453c72-9e60-4afb-9ebc-e16193e57f4a admin admin] PATH_INFO: `/v3/roles` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:05:39.422906 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed453c72-9e60-4afb-9ebc-e16193e57f4a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:05:39.424528 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed453c72-9e60-4afb-9ebc-e16193e57f4a admin admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:05:39.435070 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 1410/2823] 10.222.0.13 () {58 vars in 1122 bytes} [Wed Aug 31 10:05:39 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 10:05:39.439965 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c5a532e-672f-4e5f-a9dd-b4dc3be88ebc admin admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:05:39.450273 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c5a532e-672f-4e5f-a9dd-b4dc3be88ebc admin admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:05:39.455693 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-811fd730-06e9-4454-96ca-8374438a127d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:05:39.456276 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811fd730-06e9-4454-96ca-8374438a127d admin admin] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:05:39.456708 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811fd730-06e9-4454-96ca-8374438a127d admin admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:05:39.456877 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811fd730-06e9-4454-96ca-8374438a127d admin admin] PATH_INFO: `/v3/roles` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:05:39.457648 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-811fd730-06e9-4454-96ca-8374438a127d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:05:39.459369 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-811fd730-06e9-4454-96ca-8374438a127d admin admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:05:39.472101 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 1414/2824] 10.222.0.13 () {58 vars in 1122 bytes} [Wed Aug 31 10:05:39 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 10:05:39.476331 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed453c72-9e60-4afb-9ebc-e16193e57f4a admin admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:05:39.486603 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed453c72-9e60-4afb-9ebc-e16193e57f4a admin admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:05:39.491206 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3616bf3e-b45f-4b3b-b6b6-fe10114f5010 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:05:39.491884 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3616bf3e-b45f-4b3b-b6b6-fe10114f5010 admin admin] REQUEST_METHOD: `PUT` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:05:39.492162 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3616bf3e-b45f-4b3b-b6b6-fe10114f5010 admin admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:05:39.492436 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3616bf3e-b45f-4b3b-b6b6-fe10114f5010 admin admin] PATH_INFO: `/v3/projects/923392ae137e48d19db50edd88af8ab0/users/49c8a6c350b54eca98d4e58b35dbaba5/roles/6c99f5d9f1f243eaa134d8540f21095d` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:05:39.508041 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3616bf3e-b45f-4b3b-b6b6-fe10114f5010 admin admin] RBAC: Authorizing `identity:create_grant(project_id=923392ae137e48d19db50edd88af8ab0, user_id=49c8a6c350b54eca98d4e58b35dbaba5, role_id=6c99f5d9f1f243eaa134d8540f21095d)` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:05:39.510714 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3616bf3e-b45f-4b3b-b6b6-fe10114f5010 admin admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:05:39.524454 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 1411/2825] 10.222.0.13 () {60 vars in 1483 bytes} [Wed Aug 31 10:05:39 2022] PUT /identity/v3/projects/923392ae137e48d19db50edd88af8ab0/users/49c8a6c350b54eca98d4e58b35dbaba5/roles/6c99f5d9f1f243eaa134d8540f21095d => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 10:05:39.536161 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26957ff3-f3dd-4091-9f1f-a9169d2ad4cd None None] REQUEST_METHOD: `POST` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:05:39.536441 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26957ff3-f3dd-4091-9f1f-a9169d2ad4cd None None] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:05:39.536712 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26957ff3-f3dd-4091-9f1f-a9169d2ad4cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:05:39.566223 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.auth.core [None req-26957ff3-f3dd-4091-9f1f-a9169d2ad4cd None None] MFA Rules not processed for user `49c8a6c350b54eca98d4e58b35dbaba5`. Rule list: `[]` (Enabled: `True`). {{(pid=88114) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 10:05:39.567362 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.fernet_utils [None req-26957ff3-f3dd-4091-9f1f-a9169d2ad4cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88114) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:05:39.631606 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 1415/2826] 10.222.0.13 () {58 vars in 958 bytes} [Wed Aug 31 10:05:39 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 10:05:39.636150 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3616bf3e-b45f-4b3b-b6b6-fe10114f5010 admin admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:05:39.698579 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3616bf3e-b45f-4b3b-b6b6-fe10114f5010 admin admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:05:39.703881 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4490d42f-e569-4d07-ada4-311eeff3d3aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:05:39.704478 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4490d42f-e569-4d07-ada4-311eeff3d3aa admin admin] REQUEST_METHOD: `POST` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:05:39.704754 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4490d42f-e569-4d07-ada4-311eeff3d3aa admin admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:05:39.705024 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4490d42f-e569-4d07-ada4-311eeff3d3aa admin admin] PATH_INFO: `/v3/projects` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:05:39.705729 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4490d42f-e569-4d07-ada4-311eeff3d3aa admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:05:39.707636 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4490d42f-e569-4d07-ada4-311eeff3d3aa admin admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:05:39.734457 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 1412/2827] 10.222.0.13 () {60 vars in 1153 bytes} [Wed Aug 31 10:05:39 2022] POST /identity/v3/projects => generated 385 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 10:05:39.738861 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26957ff3-f3dd-4091-9f1f-a9169d2ad4cd None None] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:05:39.750530 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26957ff3-f3dd-4091-9f1f-a9169d2ad4cd None None] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:05:39.806063 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a43bc3b-59a6-4bde-a024-6fb9fd1de177 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:05:39.806667 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a43bc3b-59a6-4bde-a024-6fb9fd1de177 admin admin] REQUEST_METHOD: `POST` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:05:39.806939 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a43bc3b-59a6-4bde-a024-6fb9fd1de177 admin admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:05:39.807216 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a43bc3b-59a6-4bde-a024-6fb9fd1de177 admin admin] PATH_INFO: `/v3/users` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:05:39.807829 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a43bc3b-59a6-4bde-a024-6fb9fd1de177 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:05:39.809759 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a43bc3b-59a6-4bde-a024-6fb9fd1de177 admin admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:05:39.831674 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 1416/2828] 10.222.0.13 () {60 vars in 1144 bytes} [Wed Aug 31 10:05:39 2022] POST /identity/v3/users => generated 299 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 10:05:39.835893 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4490d42f-e569-4d07-ada4-311eeff3d3aa admin admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:05:39.846422 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4490d42f-e569-4d07-ada4-311eeff3d3aa admin admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:05:39.850997 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a34bb2b7-8ec0-4f23-9469-f2f6b4cb49df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:05:39.851589 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34bb2b7-8ec0-4f23-9469-f2f6b4cb49df admin admin] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:05:39.851862 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34bb2b7-8ec0-4f23-9469-f2f6b4cb49df admin admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:05:39.852125 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34bb2b7-8ec0-4f23-9469-f2f6b4cb49df admin admin] PATH_INFO: `/v3/roles` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:05:39.852761 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a34bb2b7-8ec0-4f23-9469-f2f6b4cb49df admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:05:39.854364 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a34bb2b7-8ec0-4f23-9469-f2f6b4cb49df admin admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:05:39.865404 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 1413/2829] 10.222.0.13 () {58 vars in 1122 bytes} [Wed Aug 31 10:05:39 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 10:05:39.869626 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a43bc3b-59a6-4bde-a024-6fb9fd1de177 admin admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:05:39.880821 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a43bc3b-59a6-4bde-a024-6fb9fd1de177 admin admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:05:39.885335 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64716c0b-f3f0-473b-9c73-f7fc8a81fe1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:05:39.886004 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64716c0b-f3f0-473b-9c73-f7fc8a81fe1c admin admin] REQUEST_METHOD: `PUT` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:05:39.886283 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64716c0b-f3f0-473b-9c73-f7fc8a81fe1c admin admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:05:39.886558 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64716c0b-f3f0-473b-9c73-f7fc8a81fe1c admin admin] PATH_INFO: `/v3/projects/d72cc69702734955a8dbe4a48a5296d8/users/c08b37dcc0c34926a3a5875c0e7fc1d1/roles/5d4ee63a3add47fc9a6b8b16d39e0a1f` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:05:39.904994 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64716c0b-f3f0-473b-9c73-f7fc8a81fe1c admin admin] RBAC: Authorizing `identity:create_grant(project_id=d72cc69702734955a8dbe4a48a5296d8, user_id=c08b37dcc0c34926a3a5875c0e7fc1d1, role_id=5d4ee63a3add47fc9a6b8b16d39e0a1f)` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:05:39.907650 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64716c0b-f3f0-473b-9c73-f7fc8a81fe1c admin admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:05:39.920045 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 1417/2830] 10.222.0.13 () {60 vars in 1483 bytes} [Wed Aug 31 10:05:39 2022] PUT /identity/v3/projects/d72cc69702734955a8dbe4a48a5296d8/users/c08b37dcc0c34926a3a5875c0e7fc1d1/roles/5d4ee63a3add47fc9a6b8b16d39e0a1f => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 10:05:39.924599 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a34bb2b7-8ec0-4f23-9469-f2f6b4cb49df admin admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:05:39.934587 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a34bb2b7-8ec0-4f23-9469-f2f6b4cb49df admin admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:05:39.955470 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:05:39.995639 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5706c7c-918d-41c9-b254-04133510cd0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:05:39.996242 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5706c7c-918d-41c9-b254-04133510cd0c admin admin] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:05:39.996519 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5706c7c-918d-41c9-b254-04133510cd0c admin admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:05:39.996787 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5706c7c-918d-41c9-b254-04133510cd0c admin admin] PATH_INFO: `/v3/roles` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:05:39.997446 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5706c7c-918d-41c9-b254-04133510cd0c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:05:40.000245 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5706c7c-918d-41c9-b254-04133510cd0c admin admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:05:40.010010 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 1414/2831] 10.222.0.13 () {58 vars in 1122 bytes} [Wed Aug 31 10:05:39 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 10:05:40.014253 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64716c0b-f3f0-473b-9c73-f7fc8a81fe1c admin admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:05:40.020285 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:05:40.026421 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64716c0b-f3f0-473b-9c73-f7fc8a81fe1c admin admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:05:40.031960 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56940caf-03c5-41e2-9927-fc517c5e16a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:05:40.032606 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56940caf-03c5-41e2-9927-fc517c5e16a7 admin admin] REQUEST_METHOD: `PUT` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:05:40.032883 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56940caf-03c5-41e2-9927-fc517c5e16a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:05:40.033156 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56940caf-03c5-41e2-9927-fc517c5e16a7 admin admin] PATH_INFO: `/v3/projects/d72cc69702734955a8dbe4a48a5296d8/users/c08b37dcc0c34926a3a5875c0e7fc1d1/roles/5d4ee63a3add47fc9a6b8b16d39e0a1f` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:05:40.035108 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56940caf-03c5-41e2-9927-fc517c5e16a7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d72cc69702734955a8dbe4a48a5296d8, user_id=c08b37dcc0c34926a3a5875c0e7fc1d1, role_id=5d4ee63a3add47fc9a6b8b16d39e0a1f)` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:05:40.037768 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56940caf-03c5-41e2-9927-fc517c5e16a7 admin admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:05:40.050200 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 1418/2832] 10.222.0.13 () {60 vars in 1483 bytes} [Wed Aug 31 10:05:40 2022] PUT /identity/v3/projects/d72cc69702734955a8dbe4a48a5296d8/users/c08b37dcc0c34926a3a5875c0e7fc1d1/roles/5d4ee63a3add47fc9a6b8b16d39e0a1f => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 10:05:40.060377 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea7690d-c734-421b-8932-1944dc5b26a0 None None] REQUEST_METHOD: `POST` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:05:40.060678 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea7690d-c734-421b-8932-1944dc5b26a0 None None] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:05:40.060960 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea7690d-c734-421b-8932-1944dc5b26a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:05:40.091272 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.auth.core [None req-fea7690d-c734-421b-8932-1944dc5b26a0 None None] MFA Rules not processed for user `c08b37dcc0c34926a3a5875c0e7fc1d1`. Rule list: `[]` (Enabled: `True`). {{(pid=88113) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 10:05:40.092396 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.fernet_utils [None req-fea7690d-c734-421b-8932-1944dc5b26a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:05:40.162675 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 1415/2833] 10.222.0.13 () {58 vars in 958 bytes} [Wed Aug 31 10:05:40 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 10:05:40.174858 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56940caf-03c5-41e2-9927-fc517c5e16a7 admin admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:05:40.176591 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.fernet_utils [None req-56940caf-03c5-41e2-9927-fc517c5e16a7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88114) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:05:40.232664 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56940caf-03c5-41e2-9927-fc517c5e16a7 admin admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:05:40.237047 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a8bcde-d521-490c-8abe-1f3458089acc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d52b69c8e94441dacfcff944326f169', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '49548eb44d62415582af820ab66decee', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:05:40.237671 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a8bcde-d521-490c-8abe-1f3458089acc service cinder] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:05:40.237940 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a8bcde-d521-490c-8abe-1f3458089acc service cinder] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:05:40.238203 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a8bcde-d521-490c-8abe-1f3458089acc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:05:40.246952 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40a8bcde-d521-490c-8abe-1f3458089acc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:05:40.248373 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40a8bcde-d521-490c-8abe-1f3458089acc service cinder] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:05:40.258202 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 1419/2834] 10.222.0.13 () {60 vars in 1365 bytes} [Wed Aug 31 10:05:40 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 10:05:40.287404 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea7690d-c734-421b-8932-1944dc5b26a0 None None] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:05:40.348999 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea7690d-c734-421b-8932-1944dc5b26a0 None None] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:05:40.353672 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d4d4c5-c2de-41ac-a04a-1eb654db9ad0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:05:40.354296 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d4d4c5-c2de-41ac-a04a-1eb654db9ad0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:05:40.354568 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d4d4c5-c2de-41ac-a04a-1eb654db9ad0 admin admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:05:40.354833 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d4d4c5-c2de-41ac-a04a-1eb654db9ad0 admin admin] PATH_INFO: `/v3/users/49c8a6c350b54eca98d4e58b35dbaba5` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:05:40.355953 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1d4d4c5-c2de-41ac-a04a-1eb654db9ad0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=49c8a6c350b54eca98d4e58b35dbaba5)` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:05:40.357882 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1d4d4c5-c2de-41ac-a04a-1eb654db9ad0 admin admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:05:40.402566 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.notifications [None req-b1d4d4c5-c2de-41ac-a04a-1eb654db9ad0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '49c8a6c350b54eca98d4e58b35dbaba5'} {{(pid=88113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 10:05:40.403749 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:05:40.411105 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.notifications [None req-b1d4d4c5-c2de-41ac-a04a-1eb654db9ad0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '49c8a6c350b54eca98d4e58b35dbaba5'} {{(pid=88113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 10:05:40.424020 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.notifications [None req-b1d4d4c5-c2de-41ac-a04a-1eb654db9ad0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '49c8a6c350b54eca98d4e58b35dbaba5'} {{(pid=88113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 10:05:40.424754 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.notifications [None req-b1d4d4c5-c2de-41ac-a04a-1eb654db9ad0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '49c8a6c350b54eca98d4e58b35dbaba5'} {{(pid=88113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 10:05:40.432095 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.notifications [None req-b1d4d4c5-c2de-41ac-a04a-1eb654db9ad0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '49c8a6c350b54eca98d4e58b35dbaba5'} {{(pid=88113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 10:05:40.438308 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 1416/2835] 10.222.0.13 () {58 vars in 1224 bytes} [Wed Aug 31 10:05:40 2022] DELETE /identity/v3/users/49c8a6c350b54eca98d4e58b35dbaba5 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 10:05:40.442883 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f88d5d4-c2f7-415c-b83c-0fcaae00f6e2 None None] REQUEST_METHOD: `POST` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:05:40.443164 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f88d5d4-c2f7-415c-b83c-0fcaae00f6e2 None None] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:05:40.443434 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f88d5d4-c2f7-415c-b83c-0fcaae00f6e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:05:40.461972 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.auth.core [None req-5f88d5d4-c2f7-415c-b83c-0fcaae00f6e2 None None] MFA Rules not processed for user `fd836cf590a145df9acf19cf3ae2ac56`. Rule list: `[]` (Enabled: `True`). {{(pid=88114) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 10:05:40.463153 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.fernet_utils [None req-5f88d5d4-c2f7-415c-b83c-0fcaae00f6e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88114) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:05:40.521542 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 1420/2836] 10.222.0.13 () {58 vars in 958 bytes} [Wed Aug 31 10:05:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 10:05:40.531795 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d4d4c5-c2de-41ac-a04a-1eb654db9ad0 admin admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:05:40.533242 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.fernet_utils [None req-b1d4d4c5-c2de-41ac-a04a-1eb654db9ad0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:05:40.597584 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d4d4c5-c2de-41ac-a04a-1eb654db9ad0 admin admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:05:40.601276 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8146e5f-f303-488f-a996-4324a41719b7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9bdaacd597442c2bcde2c555f2cc140', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '49548eb44d62415582af820ab66decee', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:05:40.601846 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8146e5f-f303-488f-a996-4324a41719b7 service neutron] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:05:40.602121 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8146e5f-f303-488f-a996-4324a41719b7 service neutron] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:05:40.602414 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8146e5f-f303-488f-a996-4324a41719b7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:05:40.613293 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8146e5f-f303-488f-a996-4324a41719b7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:05:40.614749 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8146e5f-f303-488f-a996-4324a41719b7 service neutron] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:05:40.624705 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 1417/2837] 10.222.0.13 () {60 vars in 1365 bytes} [Wed Aug 31 10:05:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 10:05:40.891135 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f88d5d4-c2f7-415c-b83c-0fcaae00f6e2 None None] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:05:40.892647 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.fernet_utils [None req-5f88d5d4-c2f7-415c-b83c-0fcaae00f6e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88114) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:05:40.904843 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f88d5d4-c2f7-415c-b83c-0fcaae00f6e2 None None] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:05:40.910221 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b833b1f-9cda-4c0f-9113-adeca917b909 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:05:40.910858 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b833b1f-9cda-4c0f-9113-adeca917b909 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:05:40.911133 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b833b1f-9cda-4c0f-9113-adeca917b909 admin admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:05:40.911402 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b833b1f-9cda-4c0f-9113-adeca917b909 admin admin] PATH_INFO: `/v3/projects/923392ae137e48d19db50edd88af8ab0` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:05:40.912548 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b833b1f-9cda-4c0f-9113-adeca917b909 admin admin] RBAC: Authorizing `identity:delete_project(project_id=923392ae137e48d19db50edd88af8ab0)` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:05:40.914547 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b833b1f-9cda-4c0f-9113-adeca917b909 admin admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:05:40.960165 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.notifications [None req-0b833b1f-9cda-4c0f-9113-adeca917b909 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '923392ae137e48d19db50edd88af8ab0', 'request_id': 'req-0b833b1f-9cda-4c0f-9113-adeca917b909'} {{(pid=88114) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 10:05:40.966320 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.notifications [None req-0b833b1f-9cda-4c0f-9113-adeca917b909 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '923392ae137e48d19db50edd88af8ab0', 'request_id': 'req-0b833b1f-9cda-4c0f-9113-adeca917b909'} {{(pid=88114) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 10:05:40.972159 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.notifications [None req-0b833b1f-9cda-4c0f-9113-adeca917b909 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '923392ae137e48d19db50edd88af8ab0', 'request_id': 'req-0b833b1f-9cda-4c0f-9113-adeca917b909'} {{(pid=88114) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 10:05:40.985951 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.notifications [None req-0b833b1f-9cda-4c0f-9113-adeca917b909 admin admin] The token cache is being invalidate because project 923392ae137e48d19db50edd88af8ab0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88114) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 10:05:40.986240 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.notifications [None req-0b833b1f-9cda-4c0f-9113-adeca917b909 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88114) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 10:05:40.986866 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.notifications [None req-0b833b1f-9cda-4c0f-9113-adeca917b909 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88114) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 10:05:40.988351 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 1421/2838] 10.222.0.13 () {58 vars in 1233 bytes} [Wed Aug 31 10:05:40 2022] DELETE /identity/v3/projects/923392ae137e48d19db50edd88af8ab0 => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 10:05:40.992342 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8146e5f-f303-488f-a996-4324a41719b7 service neutron] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:05:41.003153 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8146e5f-f303-488f-a996-4324a41719b7 service neutron] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:05:41.060283 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef791ef-a850-4142-bf9c-e42cce0b0198 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:05:41.060906 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef791ef-a850-4142-bf9c-e42cce0b0198 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:05:41.061180 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef791ef-a850-4142-bf9c-e42cce0b0198 admin admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:05:41.062248 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef791ef-a850-4142-bf9c-e42cce0b0198 admin admin] PATH_INFO: `/v3/users/c08b37dcc0c34926a3a5875c0e7fc1d1` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:05:41.063379 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ef791ef-a850-4142-bf9c-e42cce0b0198 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c08b37dcc0c34926a3a5875c0e7fc1d1)` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:05:41.065299 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ef791ef-a850-4142-bf9c-e42cce0b0198 admin admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:05:41.113689 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.notifications [None req-0ef791ef-a850-4142-bf9c-e42cce0b0198 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c08b37dcc0c34926a3a5875c0e7fc1d1'} {{(pid=88113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 10:05:41.120496 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.notifications [None req-0ef791ef-a850-4142-bf9c-e42cce0b0198 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c08b37dcc0c34926a3a5875c0e7fc1d1'} {{(pid=88113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 10:05:41.130991 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:05:41.133635 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.notifications [None req-0ef791ef-a850-4142-bf9c-e42cce0b0198 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c08b37dcc0c34926a3a5875c0e7fc1d1'} {{(pid=88113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 10:05:41.134566 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.notifications [None req-0ef791ef-a850-4142-bf9c-e42cce0b0198 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c08b37dcc0c34926a3a5875c0e7fc1d1'} {{(pid=88113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 10:05:41.143493 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.notifications [None req-0ef791ef-a850-4142-bf9c-e42cce0b0198 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c08b37dcc0c34926a3a5875c0e7fc1d1'} {{(pid=88113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 10:05:41.150156 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 1418/2839] 10.222.0.13 () {58 vars in 1224 bytes} [Wed Aug 31 10:05:40 2022] DELETE /identity/v3/users/c08b37dcc0c34926a3a5875c0e7fc1d1 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 10:05:41.422231 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b833b1f-9cda-4c0f-9113-adeca917b909 admin admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:05:41.423756 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.fernet_utils [None req-0b833b1f-9cda-4c0f-9113-adeca917b909 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88114) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:05:41.438476 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b833b1f-9cda-4c0f-9113-adeca917b909 admin admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:05:41.451142 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:05:41.500918 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8625b6f-2bc3-442f-8f26-1273b09f37e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:05:41.501587 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8625b6f-2bc3-442f-8f26-1273b09f37e7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:05:41.501866 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8625b6f-2bc3-442f-8f26-1273b09f37e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:05:41.502137 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8625b6f-2bc3-442f-8f26-1273b09f37e7 admin admin] PATH_INFO: `/v3/projects/d72cc69702734955a8dbe4a48a5296d8` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:05:41.503250 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8625b6f-2bc3-442f-8f26-1273b09f37e7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d72cc69702734955a8dbe4a48a5296d8)` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:05:41.505146 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8625b6f-2bc3-442f-8f26-1273b09f37e7 admin admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:05:41.554687 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.notifications [None req-a8625b6f-2bc3-442f-8f26-1273b09f37e7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd72cc69702734955a8dbe4a48a5296d8', 'request_id': 'req-a8625b6f-2bc3-442f-8f26-1273b09f37e7'} {{(pid=88114) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 10:05:41.560783 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.notifications [None req-a8625b6f-2bc3-442f-8f26-1273b09f37e7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd72cc69702734955a8dbe4a48a5296d8', 'request_id': 'req-a8625b6f-2bc3-442f-8f26-1273b09f37e7'} {{(pid=88114) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 10:05:41.567194 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.notifications [None req-a8625b6f-2bc3-442f-8f26-1273b09f37e7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd72cc69702734955a8dbe4a48a5296d8', 'request_id': 'req-a8625b6f-2bc3-442f-8f26-1273b09f37e7'} {{(pid=88114) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 10:05:41.580935 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.notifications [None req-a8625b6f-2bc3-442f-8f26-1273b09f37e7 admin admin] The token cache is being invalidate because project d72cc69702734955a8dbe4a48a5296d8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88114) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 10:05:41.581218 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.notifications [None req-a8625b6f-2bc3-442f-8f26-1273b09f37e7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88114) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 10:05:41.581890 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.notifications [None req-a8625b6f-2bc3-442f-8f26-1273b09f37e7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88114) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 10:05:41.583371 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 1422/2840] 10.222.0.13 () {58 vars in 1233 bytes} [Wed Aug 31 10:05:41 2022] DELETE /identity/v3/projects/d72cc69702734955a8dbe4a48a5296d8 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 10:05:41.588342 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7e8690-35d5-4d53-98f4-152174a42103 None None] REQUEST_METHOD: `POST` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:05:41.588630 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7e8690-35d5-4d53-98f4-152174a42103 None None] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:05:41.588907 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7e8690-35d5-4d53-98f4-152174a42103 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:05:41.610112 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.auth.core [None req-fe7e8690-35d5-4d53-98f4-152174a42103 None None] MFA Rules not processed for user `fd836cf590a145df9acf19cf3ae2ac56`. Rule list: `[]` (Enabled: `True`). {{(pid=88113) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 10:05:41.611583 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.fernet_utils [None req-fe7e8690-35d5-4d53-98f4-152174a42103 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:05:41.672488 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 1419/2841] 10.222.0.13 () {58 vars in 958 bytes} [Wed Aug 31 10:05:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 10:05:41.681345 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a8b37b-e212-45c3-9f4c-dba36a0a55d1 None None] REQUEST_METHOD: `POST` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:05:41.681680 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a8b37b-e212-45c3-9f4c-dba36a0a55d1 None None] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:05:41.681952 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a8b37b-e212-45c3-9f4c-dba36a0a55d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:05:41.704482 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.auth.core [None req-a6a8b37b-e212-45c3-9f4c-dba36a0a55d1 None None] MFA Rules not processed for user `fd836cf590a145df9acf19cf3ae2ac56`. Rule list: `[]` (Enabled: `True`). {{(pid=88114) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 10:05:41.705741 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.fernet_utils [None req-a6a8b37b-e212-45c3-9f4c-dba36a0a55d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88114) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:05:41.713911 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 1423/2842] 10.222.0.13 () {58 vars in 958 bytes} [Wed Aug 31 10:05:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 10:05:41.719548 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe7e8690-35d5-4d53-98f4-152174a42103 None None] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:05:41.729398 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe7e8690-35d5-4d53-98f4-152174a42103 None None] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:05:41.735017 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91024f9c-fedb-46b6-aad6-f6eb418af030 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:05:41.735928 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91024f9c-fedb-46b6-aad6-f6eb418af030 admin admin] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:05:41.736208 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91024f9c-fedb-46b6-aad6-f6eb418af030 admin admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:05:41.736479 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91024f9c-fedb-46b6-aad6-f6eb418af030 admin admin] PATH_INFO: `/v3/domains` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:05:41.737130 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91024f9c-fedb-46b6-aad6-f6eb418af030 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88113) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:05:41.737464 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91024f9c-fedb-46b6-aad6-f6eb418af030 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:05:41.739091 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91024f9c-fedb-46b6-aad6-f6eb418af030 admin admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:05:41.751828 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 1420/2843] 10.222.0.13 () {58 vars in 1153 bytes} [Wed Aug 31 10:05:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:05:41.756225 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a8b37b-e212-45c3-9f4c-dba36a0a55d1 None None] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:05:41.765580 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a8b37b-e212-45c3-9f4c-dba36a0a55d1 None None] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:05:41.770363 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b46947-f42d-4cb7-bcd5-c24715f9bee1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:05:41.770990 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b46947-f42d-4cb7-bcd5-c24715f9bee1 admin admin] REQUEST_METHOD: `POST` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:05:41.771264 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b46947-f42d-4cb7-bcd5-c24715f9bee1 admin admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:05:41.771524 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b46947-f42d-4cb7-bcd5-c24715f9bee1 admin admin] PATH_INFO: `/v3/projects` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:05:41.772146 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2b46947-f42d-4cb7-bcd5-c24715f9bee1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:05:41.774057 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2b46947-f42d-4cb7-bcd5-c24715f9bee1 admin admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:05:41.802225 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 1424/2844] 10.222.0.13 () {60 vars in 1153 bytes} [Wed Aug 31 10:05:41 2022] POST /identity/v3/projects => generated 381 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 10:05:41.808078 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91024f9c-fedb-46b6-aad6-f6eb418af030 admin admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:05:41.818415 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91024f9c-fedb-46b6-aad6-f6eb418af030 admin admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:05:41.877866 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1625b95-9572-4916-980f-1f5cc267bb28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:05:41.878471 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1625b95-9572-4916-980f-1f5cc267bb28 admin admin] REQUEST_METHOD: `POST` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:05:41.878751 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1625b95-9572-4916-980f-1f5cc267bb28 admin admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:05:41.879020 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1625b95-9572-4916-980f-1f5cc267bb28 admin admin] PATH_INFO: `/v3/users` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:05:41.879649 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1625b95-9572-4916-980f-1f5cc267bb28 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:05:41.881588 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1625b95-9572-4916-980f-1f5cc267bb28 admin admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:05:41.907178 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 1421/2845] 10.222.0.13 () {60 vars in 1144 bytes} [Wed Aug 31 10:05:41 2022] POST /identity/v3/users => generated 291 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 10:05:41.911535 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b46947-f42d-4cb7-bcd5-c24715f9bee1 admin admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:05:41.922392 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b46947-f42d-4cb7-bcd5-c24715f9bee1 admin admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:05:41.927124 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e7d9115-32be-4af4-b5b0-46d4ec35cf75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:05:41.927646 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7d9115-32be-4af4-b5b0-46d4ec35cf75 admin admin] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:05:41.927918 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7d9115-32be-4af4-b5b0-46d4ec35cf75 admin admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:05:41.928181 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7d9115-32be-4af4-b5b0-46d4ec35cf75 admin admin] PATH_INFO: `/v3/roles` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:05:41.929017 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e7d9115-32be-4af4-b5b0-46d4ec35cf75 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:05:41.930630 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e7d9115-32be-4af4-b5b0-46d4ec35cf75 admin admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:05:41.941298 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 1425/2846] 10.222.0.13 () {58 vars in 1122 bytes} [Wed Aug 31 10:05:41 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 10:05:41.945241 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1625b95-9572-4916-980f-1f5cc267bb28 admin admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:05:41.955643 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1625b95-9572-4916-980f-1f5cc267bb28 admin admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:05:41.960401 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0fa5c61-ad0c-4439-9cb5-0f4442073465 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:05:41.961002 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0fa5c61-ad0c-4439-9cb5-0f4442073465 admin admin] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:05:41.961290 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0fa5c61-ad0c-4439-9cb5-0f4442073465 admin admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:05:41.961588 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0fa5c61-ad0c-4439-9cb5-0f4442073465 admin admin] PATH_INFO: `/v3/roles` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:05:41.962229 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0fa5c61-ad0c-4439-9cb5-0f4442073465 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:05:41.963821 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0fa5c61-ad0c-4439-9cb5-0f4442073465 admin admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:05:41.978019 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 1422/2847] 10.222.0.13 () {58 vars in 1122 bytes} [Wed Aug 31 10:05:41 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 10:05:41.982731 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e7d9115-32be-4af4-b5b0-46d4ec35cf75 admin admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:05:41.994074 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e7d9115-32be-4af4-b5b0-46d4ec35cf75 admin admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:05:41.998717 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e97f4507-7473-4336-bbc0-f695de57ba30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:05:41.999612 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97f4507-7473-4336-bbc0-f695de57ba30 admin admin] REQUEST_METHOD: `PUT` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:05:41.999883 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97f4507-7473-4336-bbc0-f695de57ba30 admin admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:05:42.000150 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97f4507-7473-4336-bbc0-f695de57ba30 admin admin] PATH_INFO: `/v3/projects/fc4454c4633643309377783648391b0a/users/6ffcaad06a9b46eaba695d6e7d887df2/roles/6c99f5d9f1f243eaa134d8540f21095d` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:05:42.014166 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e97f4507-7473-4336-bbc0-f695de57ba30 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fc4454c4633643309377783648391b0a, user_id=6ffcaad06a9b46eaba695d6e7d887df2, role_id=6c99f5d9f1f243eaa134d8540f21095d)` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:05:42.016899 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e97f4507-7473-4336-bbc0-f695de57ba30 admin admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:05:42.031035 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 1426/2848] 10.222.0.13 () {60 vars in 1483 bytes} [Wed Aug 31 10:05:41 2022] PUT /identity/v3/projects/fc4454c4633643309377783648391b0a/users/6ffcaad06a9b46eaba695d6e7d887df2/roles/6c99f5d9f1f243eaa134d8540f21095d => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 10:05:42.041542 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316703d4-0bf7-4221-ba81-6f995e1e99b4 None None] REQUEST_METHOD: `POST` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:05:42.042050 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316703d4-0bf7-4221-ba81-6f995e1e99b4 None None] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:05:42.042176 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316703d4-0bf7-4221-ba81-6f995e1e99b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:05:42.079151 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.auth.core [None req-316703d4-0bf7-4221-ba81-6f995e1e99b4 None None] MFA Rules not processed for user `6ffcaad06a9b46eaba695d6e7d887df2`. Rule list: `[]` (Enabled: `True`). {{(pid=88113) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 10:05:42.080369 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.fernet_utils [None req-316703d4-0bf7-4221-ba81-6f995e1e99b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:05:42.122600 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:DELETE=2,keystone:SELECT=94,keystone:INSERT=4 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:05:42.147137 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 1423/2849] 10.222.0.13 () {58 vars in 958 bytes} [Wed Aug 31 10:05:42 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 10:05:42.151766 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e97f4507-7473-4336-bbc0-f695de57ba30 admin admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:05:42.197260 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:05:42.216369 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e97f4507-7473-4336-bbc0-f695de57ba30 admin admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:05:42.221852 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-713af903-3ae5-4e9e-832d-0aa3f379ece9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:05:42.222448 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713af903-3ae5-4e9e-832d-0aa3f379ece9 admin admin] REQUEST_METHOD: `POST` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:05:42.222729 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713af903-3ae5-4e9e-832d-0aa3f379ece9 admin admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:05:42.222990 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713af903-3ae5-4e9e-832d-0aa3f379ece9 admin admin] PATH_INFO: `/v3/projects` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:05:42.223605 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-713af903-3ae5-4e9e-832d-0aa3f379ece9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:05:42.225513 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-713af903-3ae5-4e9e-832d-0aa3f379ece9 admin admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:05:42.252012 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 1427/2850] 10.222.0.13 () {60 vars in 1153 bytes} [Wed Aug 31 10:05:42 2022] POST /identity/v3/projects => generated 379 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 10:05:42.256076 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-316703d4-0bf7-4221-ba81-6f995e1e99b4 None None] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:05:42.267917 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-316703d4-0bf7-4221-ba81-6f995e1e99b4 None None] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:05:42.324000 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ecbf6a-0541-450e-a909-4e11703a6906 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:05:42.324612 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ecbf6a-0541-450e-a909-4e11703a6906 admin admin] REQUEST_METHOD: `POST` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:05:42.324888 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ecbf6a-0541-450e-a909-4e11703a6906 admin admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:05:42.325156 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ecbf6a-0541-450e-a909-4e11703a6906 admin admin] PATH_INFO: `/v3/users` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:05:42.325821 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9ecbf6a-0541-450e-a909-4e11703a6906 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:05:42.328433 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9ecbf6a-0541-450e-a909-4e11703a6906 admin admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:05:42.349229 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 1424/2851] 10.222.0.13 () {60 vars in 1144 bytes} [Wed Aug 31 10:05:42 2022] POST /identity/v3/users => generated 296 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 10:05:42.353321 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-713af903-3ae5-4e9e-832d-0aa3f379ece9 admin admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:05:42.364770 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-713af903-3ae5-4e9e-832d-0aa3f379ece9 admin admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:05:42.369581 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ed2a73-0bc2-4604-a6a5-4d665e17c265 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:05:42.370177 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ed2a73-0bc2-4604-a6a5-4d665e17c265 admin admin] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:05:42.370461 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ed2a73-0bc2-4604-a6a5-4d665e17c265 admin admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:05:42.370731 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ed2a73-0bc2-4604-a6a5-4d665e17c265 admin admin] PATH_INFO: `/v3/roles` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:05:42.371371 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22ed2a73-0bc2-4604-a6a5-4d665e17c265 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:05:42.372946 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22ed2a73-0bc2-4604-a6a5-4d665e17c265 admin admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:05:42.383743 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 1428/2852] 10.222.0.13 () {58 vars in 1122 bytes} [Wed Aug 31 10:05:42 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 10:05:42.387951 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ecbf6a-0541-450e-a909-4e11703a6906 admin admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:05:42.399470 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ecbf6a-0541-450e-a909-4e11703a6906 admin admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:05:42.404174 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eac5069-e53f-4a5c-a807-7a08c0a5e16d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:05:42.404829 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eac5069-e53f-4a5c-a807-7a08c0a5e16d admin admin] REQUEST_METHOD: `PUT` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:05:42.405788 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eac5069-e53f-4a5c-a807-7a08c0a5e16d admin admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:05:42.406226 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eac5069-e53f-4a5c-a807-7a08c0a5e16d admin admin] PATH_INFO: `/v3/projects/a29993fff0be42e0b120cc8d3016f4e6/users/4a9453be1fd045598f7223b41cc7b32d/roles/5d4ee63a3add47fc9a6b8b16d39e0a1f` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:05:42.421661 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eac5069-e53f-4a5c-a807-7a08c0a5e16d admin admin] RBAC: Authorizing `identity:create_grant(project_id=a29993fff0be42e0b120cc8d3016f4e6, user_id=4a9453be1fd045598f7223b41cc7b32d, role_id=5d4ee63a3add47fc9a6b8b16d39e0a1f)` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:05:42.424380 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eac5069-e53f-4a5c-a807-7a08c0a5e16d admin admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:05:42.437914 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 1425/2853] 10.222.0.13 () {60 vars in 1483 bytes} [Wed Aug 31 10:05:42 2022] PUT /identity/v3/projects/a29993fff0be42e0b120cc8d3016f4e6/users/4a9453be1fd045598f7223b41cc7b32d/roles/5d4ee63a3add47fc9a6b8b16d39e0a1f => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 10:05:42.442085 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ed2a73-0bc2-4604-a6a5-4d665e17c265 admin admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:05:42.452843 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ed2a73-0bc2-4604-a6a5-4d665e17c265 admin admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:05:42.510418 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7eeb65d-4490-4825-b1d5-ef18b5eb2010 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:05:42.511021 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7eeb65d-4490-4825-b1d5-ef18b5eb2010 admin admin] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:05:42.511299 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7eeb65d-4490-4825-b1d5-ef18b5eb2010 admin admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:05:42.511621 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7eeb65d-4490-4825-b1d5-ef18b5eb2010 admin admin] PATH_INFO: `/v3/roles` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:05:42.512366 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7eeb65d-4490-4825-b1d5-ef18b5eb2010 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:05:42.514023 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7eeb65d-4490-4825-b1d5-ef18b5eb2010 admin admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:05:42.524367 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 1429/2854] 10.222.0.13 () {58 vars in 1122 bytes} [Wed Aug 31 10:05:42 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 10:05:42.528559 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eac5069-e53f-4a5c-a807-7a08c0a5e16d admin admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:05:42.540065 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eac5069-e53f-4a5c-a807-7a08c0a5e16d admin admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:05:42.544682 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc8e645e-fa7c-472e-b739-52523d3a5496 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:05:42.545354 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8e645e-fa7c-472e-b739-52523d3a5496 admin admin] REQUEST_METHOD: `PUT` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:05:42.545704 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8e645e-fa7c-472e-b739-52523d3a5496 admin admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:05:42.545978 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8e645e-fa7c-472e-b739-52523d3a5496 admin admin] PATH_INFO: `/v3/projects/a29993fff0be42e0b120cc8d3016f4e6/users/4a9453be1fd045598f7223b41cc7b32d/roles/5d4ee63a3add47fc9a6b8b16d39e0a1f` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:05:42.548061 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc8e645e-fa7c-472e-b739-52523d3a5496 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a29993fff0be42e0b120cc8d3016f4e6, user_id=4a9453be1fd045598f7223b41cc7b32d, role_id=5d4ee63a3add47fc9a6b8b16d39e0a1f)` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:05:42.550706 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc8e645e-fa7c-472e-b739-52523d3a5496 admin admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:05:42.565863 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 1426/2855] 10.222.0.13 () {60 vars in 1483 bytes} [Wed Aug 31 10:05:42 2022] PUT /identity/v3/projects/a29993fff0be42e0b120cc8d3016f4e6/users/4a9453be1fd045598f7223b41cc7b32d/roles/5d4ee63a3add47fc9a6b8b16d39e0a1f => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 10:05:42.576193 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e2312e-87ee-48dc-ad3a-5c3ca52bd4c1 None None] REQUEST_METHOD: `POST` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:05:42.576485 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e2312e-87ee-48dc-ad3a-5c3ca52bd4c1 None None] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:05:42.576764 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e2312e-87ee-48dc-ad3a-5c3ca52bd4c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:05:42.605302 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.auth.core [None req-c5e2312e-87ee-48dc-ad3a-5c3ca52bd4c1 None None] MFA Rules not processed for user `4a9453be1fd045598f7223b41cc7b32d`. Rule list: `[]` (Enabled: `True`). {{(pid=88114) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 10:05:42.606951 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.fernet_utils [None req-c5e2312e-87ee-48dc-ad3a-5c3ca52bd4c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88114) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:05:42.676291 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 1430/2856] 10.222.0.13 () {58 vars in 958 bytes} [Wed Aug 31 10:05:42 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 10:05:42.687658 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc8e645e-fa7c-472e-b739-52523d3a5496 admin admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:05:42.689531 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.fernet_utils [None req-cc8e645e-fa7c-472e-b739-52523d3a5496 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:05:42.746690 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc8e645e-fa7c-472e-b739-52523d3a5496 admin admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:05:42.751099 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36830d3b-f125-4552-85b6-b4ab25f74443 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d52b69c8e94441dacfcff944326f169', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '49548eb44d62415582af820ab66decee', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:05:42.751704 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36830d3b-f125-4552-85b6-b4ab25f74443 service cinder] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:05:42.751973 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36830d3b-f125-4552-85b6-b4ab25f74443 service cinder] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:05:42.752238 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36830d3b-f125-4552-85b6-b4ab25f74443 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:05:42.761108 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36830d3b-f125-4552-85b6-b4ab25f74443 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:05:42.762552 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36830d3b-f125-4552-85b6-b4ab25f74443 service cinder] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:05:42.772527 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 1427/2857] 10.222.0.13 () {60 vars in 1365 bytes} [Wed Aug 31 10:05:42 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 10:05:43.148011 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e2312e-87ee-48dc-ad3a-5c3ca52bd4c1 None None] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:05:43.188727 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:05:43.210570 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e2312e-87ee-48dc-ad3a-5c3ca52bd4c1 None None] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:05:43.215235 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e66f40-9f71-4db8-8581-75073f1f5f78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:05:43.215889 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e66f40-9f71-4db8-8581-75073f1f5f78 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:05:43.216173 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e66f40-9f71-4db8-8581-75073f1f5f78 admin admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:05:43.216470 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e66f40-9f71-4db8-8581-75073f1f5f78 admin admin] PATH_INFO: `/v3/users/6ffcaad06a9b46eaba695d6e7d887df2` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:05:43.218898 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4e66f40-9f71-4db8-8581-75073f1f5f78 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6ffcaad06a9b46eaba695d6e7d887df2)` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:05:43.219848 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4e66f40-9f71-4db8-8581-75073f1f5f78 admin admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:05:43.266942 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.notifications [None req-a4e66f40-9f71-4db8-8581-75073f1f5f78 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6ffcaad06a9b46eaba695d6e7d887df2'} {{(pid=88114) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 10:05:43.273580 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.notifications [None req-a4e66f40-9f71-4db8-8581-75073f1f5f78 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6ffcaad06a9b46eaba695d6e7d887df2'} {{(pid=88114) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 10:05:43.281368 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.notifications [None req-a4e66f40-9f71-4db8-8581-75073f1f5f78 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6ffcaad06a9b46eaba695d6e7d887df2'} {{(pid=88114) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 10:05:43.281993 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.notifications [None req-a4e66f40-9f71-4db8-8581-75073f1f5f78 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6ffcaad06a9b46eaba695d6e7d887df2'} {{(pid=88114) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 10:05:43.294035 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.notifications [None req-a4e66f40-9f71-4db8-8581-75073f1f5f78 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6ffcaad06a9b46eaba695d6e7d887df2'} {{(pid=88114) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 10:05:43.300297 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 1431/2858] 10.222.0.13 () {58 vars in 1224 bytes} [Wed Aug 31 10:05:43 2022] DELETE /identity/v3/users/6ffcaad06a9b46eaba695d6e7d887df2 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 10:05:43.305313 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3074e8fa-56b6-4c91-a9ac-eec885bde4ce None None] REQUEST_METHOD: `POST` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:05:43.305860 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3074e8fa-56b6-4c91-a9ac-eec885bde4ce None None] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:05:43.305978 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3074e8fa-56b6-4c91-a9ac-eec885bde4ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:05:43.324989 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.auth.core [None req-3074e8fa-56b6-4c91-a9ac-eec885bde4ce None None] MFA Rules not processed for user `fd836cf590a145df9acf19cf3ae2ac56`. Rule list: `[]` (Enabled: `True`). {{(pid=88113) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 10:05:43.326211 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.fernet_utils [None req-3074e8fa-56b6-4c91-a9ac-eec885bde4ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:05:43.358540 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:05:43.387244 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 1428/2859] 10.222.0.13 () {58 vars in 958 bytes} [Wed Aug 31 10:05:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 10:05:43.398498 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e66f40-9f71-4db8-8581-75073f1f5f78 admin admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:05:43.399917 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.fernet_utils [None req-a4e66f40-9f71-4db8-8581-75073f1f5f78 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88114) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:05:43.458557 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e66f40-9f71-4db8-8581-75073f1f5f78 admin admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:05:43.462302 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb8a933-15ae-4fe4-82d1-018cc1838706 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9bdaacd597442c2bcde2c555f2cc140', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '49548eb44d62415582af820ab66decee', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:05:43.462941 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb8a933-15ae-4fe4-82d1-018cc1838706 service neutron] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:05:43.463221 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb8a933-15ae-4fe4-82d1-018cc1838706 service neutron] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:05:43.463512 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb8a933-15ae-4fe4-82d1-018cc1838706 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:05:43.473017 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eb8a933-15ae-4fe4-82d1-018cc1838706 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:05:43.474505 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eb8a933-15ae-4fe4-82d1-018cc1838706 service neutron] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:05:43.484050 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 1432/2860] 10.222.0.13 () {60 vars in 1365 bytes} [Wed Aug 31 10:05:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 10:05:43.754493 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3074e8fa-56b6-4c91-a9ac-eec885bde4ce None None] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:05:43.755922 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.fernet_utils [None req-3074e8fa-56b6-4c91-a9ac-eec885bde4ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:05:43.768826 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3074e8fa-56b6-4c91-a9ac-eec885bde4ce None None] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:05:43.773676 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f4e1f56-8a39-4278-a350-0aca206ae8ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:05:43.774328 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4e1f56-8a39-4278-a350-0aca206ae8ea admin admin] REQUEST_METHOD: `DELETE` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:05:43.774599 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4e1f56-8a39-4278-a350-0aca206ae8ea admin admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:05:43.774872 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4e1f56-8a39-4278-a350-0aca206ae8ea admin admin] PATH_INFO: `/v3/projects/fc4454c4633643309377783648391b0a` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:05:43.776696 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f4e1f56-8a39-4278-a350-0aca206ae8ea admin admin] RBAC: Authorizing `identity:delete_project(project_id=fc4454c4633643309377783648391b0a)` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:05:43.778662 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f4e1f56-8a39-4278-a350-0aca206ae8ea admin admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:05:43.826512 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.notifications [None req-9f4e1f56-8a39-4278-a350-0aca206ae8ea admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fc4454c4633643309377783648391b0a', 'request_id': 'req-9f4e1f56-8a39-4278-a350-0aca206ae8ea'} {{(pid=88113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 10:05:43.832868 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.notifications [None req-9f4e1f56-8a39-4278-a350-0aca206ae8ea admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fc4454c4633643309377783648391b0a', 'request_id': 'req-9f4e1f56-8a39-4278-a350-0aca206ae8ea'} {{(pid=88113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 10:05:43.845971 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.notifications [None req-9f4e1f56-8a39-4278-a350-0aca206ae8ea admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fc4454c4633643309377783648391b0a', 'request_id': 'req-9f4e1f56-8a39-4278-a350-0aca206ae8ea'} {{(pid=88113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 10:05:43.857338 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.notifications [None req-9f4e1f56-8a39-4278-a350-0aca206ae8ea admin admin] The token cache is being invalidate because project fc4454c4633643309377783648391b0a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88113) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 10:05:43.857660 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.notifications [None req-9f4e1f56-8a39-4278-a350-0aca206ae8ea admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 10:05:43.858307 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.notifications [None req-9f4e1f56-8a39-4278-a350-0aca206ae8ea admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 10:05:43.859825 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 1429/2861] 10.222.0.13 () {58 vars in 1233 bytes} [Wed Aug 31 10:05:43 2022] DELETE /identity/v3/projects/fc4454c4633643309377783648391b0a => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 10:05:43.864071 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb8a933-15ae-4fe4-82d1-018cc1838706 service neutron] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:05:43.873971 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb8a933-15ae-4fe4-82d1-018cc1838706 service neutron] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:05:43.931513 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b16f22-b099-4377-969e-1e306def70b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:05:43.932134 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b16f22-b099-4377-969e-1e306def70b4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:05:43.932404 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b16f22-b099-4377-969e-1e306def70b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:05:43.932669 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b16f22-b099-4377-969e-1e306def70b4 admin admin] PATH_INFO: `/v3/users/4a9453be1fd045598f7223b41cc7b32d` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:05:43.933803 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42b16f22-b099-4377-969e-1e306def70b4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4a9453be1fd045598f7223b41cc7b32d)` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:05:43.935688 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42b16f22-b099-4377-969e-1e306def70b4 admin admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:05:43.937230 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:05:43.987887 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.notifications [None req-42b16f22-b099-4377-969e-1e306def70b4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4a9453be1fd045598f7223b41cc7b32d'} {{(pid=88114) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 10:05:43.994688 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.notifications [None req-42b16f22-b099-4377-969e-1e306def70b4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4a9453be1fd045598f7223b41cc7b32d'} {{(pid=88114) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 10:05:44.002341 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.notifications [None req-42b16f22-b099-4377-969e-1e306def70b4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4a9453be1fd045598f7223b41cc7b32d'} {{(pid=88114) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 10:05:44.003220 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.notifications [None req-42b16f22-b099-4377-969e-1e306def70b4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4a9453be1fd045598f7223b41cc7b32d'} {{(pid=88114) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 10:05:44.016064 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.notifications [None req-42b16f22-b099-4377-969e-1e306def70b4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4a9453be1fd045598f7223b41cc7b32d'} {{(pid=88114) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 10:05:44.021896 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 1433/2862] 10.222.0.13 () {58 vars in 1224 bytes} [Wed Aug 31 10:05:43 2022] DELETE /identity/v3/users/4a9453be1fd045598f7223b41cc7b32d => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 10:05:44.290408 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f4e1f56-8a39-4278-a350-0aca206ae8ea admin admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:05:44.292014 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.fernet_utils [None req-9f4e1f56-8a39-4278-a350-0aca206ae8ea admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:05:44.305720 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f4e1f56-8a39-4278-a350-0aca206ae8ea admin admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:05:44.365042 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ee89c7-fe0d-426e-96dc-6ee455470dd7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:05:44.365703 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ee89c7-fe0d-426e-96dc-6ee455470dd7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:05:44.365979 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ee89c7-fe0d-426e-96dc-6ee455470dd7 admin admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:05:44.366245 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ee89c7-fe0d-426e-96dc-6ee455470dd7 admin admin] PATH_INFO: `/v3/projects/a29993fff0be42e0b120cc8d3016f4e6` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:05:44.367353 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45ee89c7-fe0d-426e-96dc-6ee455470dd7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a29993fff0be42e0b120cc8d3016f4e6)` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:05:44.369365 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45ee89c7-fe0d-426e-96dc-6ee455470dd7 admin admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:05:44.415308 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.notifications [None req-45ee89c7-fe0d-426e-96dc-6ee455470dd7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a29993fff0be42e0b120cc8d3016f4e6', 'request_id': 'req-45ee89c7-fe0d-426e-96dc-6ee455470dd7'} {{(pid=88113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 10:05:44.420707 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.notifications [None req-45ee89c7-fe0d-426e-96dc-6ee455470dd7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a29993fff0be42e0b120cc8d3016f4e6', 'request_id': 'req-45ee89c7-fe0d-426e-96dc-6ee455470dd7'} {{(pid=88113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 10:05:44.424855 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:05:44.430062 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.notifications [None req-45ee89c7-fe0d-426e-96dc-6ee455470dd7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a29993fff0be42e0b120cc8d3016f4e6', 'request_id': 'req-45ee89c7-fe0d-426e-96dc-6ee455470dd7'} {{(pid=88113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 10:05:44.444042 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.notifications [None req-45ee89c7-fe0d-426e-96dc-6ee455470dd7 admin admin] The token cache is being invalidate because project a29993fff0be42e0b120cc8d3016f4e6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88113) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 10:05:44.444327 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.notifications [None req-45ee89c7-fe0d-426e-96dc-6ee455470dd7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 10:05:44.444984 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.notifications [None req-45ee89c7-fe0d-426e-96dc-6ee455470dd7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 10:05:44.447342 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 1430/2863] 10.222.0.13 () {58 vars in 1233 bytes} [Wed Aug 31 10:05:44 2022] DELETE /identity/v3/projects/a29993fff0be42e0b120cc8d3016f4e6 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 10:05:44.453182 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e2439c-fb9f-47fa-aad4-9aa6051372d9 None None] REQUEST_METHOD: `POST` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:05:44.453530 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e2439c-fb9f-47fa-aad4-9aa6051372d9 None None] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:05:44.453808 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e2439c-fb9f-47fa-aad4-9aa6051372d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:05:44.474589 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.auth.core [None req-52e2439c-fb9f-47fa-aad4-9aa6051372d9 None None] MFA Rules not processed for user `fd836cf590a145df9acf19cf3ae2ac56`. Rule list: `[]` (Enabled: `True`). {{(pid=88114) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 10:05:44.476424 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.fernet_utils [None req-52e2439c-fb9f-47fa-aad4-9aa6051372d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88114) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:05:44.536746 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 1434/2864] 10.222.0.13 () {58 vars in 958 bytes} [Wed Aug 31 10:05:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 10:05:44.544564 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf76e4c-7511-417e-a8d9-8e27310dd4be None None] REQUEST_METHOD: `POST` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:05:44.544844 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf76e4c-7511-417e-a8d9-8e27310dd4be None None] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:05:44.545114 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf76e4c-7511-417e-a8d9-8e27310dd4be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:05:44.564546 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.auth.core [None req-cdf76e4c-7511-417e-a8d9-8e27310dd4be None None] MFA Rules not processed for user `fd836cf590a145df9acf19cf3ae2ac56`. Rule list: `[]` (Enabled: `True`). {{(pid=88113) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 10:05:44.565769 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.fernet_utils [None req-cdf76e4c-7511-417e-a8d9-8e27310dd4be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:05:44.573583 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 1431/2865] 10.222.0.13 () {58 vars in 958 bytes} [Wed Aug 31 10:05:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 10:05:44.578804 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e2439c-fb9f-47fa-aad4-9aa6051372d9 None None] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:05:44.588460 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e2439c-fb9f-47fa-aad4-9aa6051372d9 None None] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:05:44.593848 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d55edd3-5a7a-43f2-83c6-f6761006801f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:05:44.594445 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d55edd3-5a7a-43f2-83c6-f6761006801f admin admin] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:05:44.594718 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d55edd3-5a7a-43f2-83c6-f6761006801f admin admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:05:44.594986 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d55edd3-5a7a-43f2-83c6-f6761006801f admin admin] PATH_INFO: `/v3/domains` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:05:44.595661 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d55edd3-5a7a-43f2-83c6-f6761006801f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88114) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:05:44.595956 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d55edd3-5a7a-43f2-83c6-f6761006801f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:05:44.597573 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d55edd3-5a7a-43f2-83c6-f6761006801f admin admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:05:44.609132 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 1435/2866] 10.222.0.13 () {58 vars in 1153 bytes} [Wed Aug 31 10:05:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 10:05:44.613859 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf76e4c-7511-417e-a8d9-8e27310dd4be None None] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:05:44.623082 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf76e4c-7511-417e-a8d9-8e27310dd4be None None] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:05:44.627913 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d89a466-f92e-4735-9261-26178d880a18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:05:44.628616 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d89a466-f92e-4735-9261-26178d880a18 admin admin] REQUEST_METHOD: `POST` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:05:44.628816 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d89a466-f92e-4735-9261-26178d880a18 admin admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:05:44.629083 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d89a466-f92e-4735-9261-26178d880a18 admin admin] PATH_INFO: `/v3/projects` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:05:44.629744 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d89a466-f92e-4735-9261-26178d880a18 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:05:44.631642 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d89a466-f92e-4735-9261-26178d880a18 admin admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:05:44.659685 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 1432/2867] 10.222.0.13 () {60 vars in 1153 bytes} [Wed Aug 31 10:05:44 2022] POST /identity/v3/projects => generated 371 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 10:05:44.663776 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d55edd3-5a7a-43f2-83c6-f6761006801f admin admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:05:44.673919 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d55edd3-5a7a-43f2-83c6-f6761006801f admin admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:05:44.730629 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32abbf89-2f1f-4511-8fc3-1ee4fb47972c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:05:44.731235 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32abbf89-2f1f-4511-8fc3-1ee4fb47972c admin admin] REQUEST_METHOD: `POST` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:05:44.731517 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32abbf89-2f1f-4511-8fc3-1ee4fb47972c admin admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:05:44.731789 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32abbf89-2f1f-4511-8fc3-1ee4fb47972c admin admin] PATH_INFO: `/v3/users` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:05:44.732431 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32abbf89-2f1f-4511-8fc3-1ee4fb47972c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:05:44.734589 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32abbf89-2f1f-4511-8fc3-1ee4fb47972c admin admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:05:44.746062 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:05:44.758125 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 1436/2868] 10.222.0.13 () {60 vars in 1144 bytes} [Wed Aug 31 10:05:44 2022] POST /identity/v3/users => generated 286 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 10:05:44.762946 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d89a466-f92e-4735-9261-26178d880a18 admin admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:05:44.773952 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d89a466-f92e-4735-9261-26178d880a18 admin admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:05:44.779093 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9cdc658-faef-4a0e-a947-49e7e43923ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:05:44.779767 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9cdc658-faef-4a0e-a947-49e7e43923ca admin admin] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:05:44.780782 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9cdc658-faef-4a0e-a947-49e7e43923ca admin admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:05:44.781057 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9cdc658-faef-4a0e-a947-49e7e43923ca admin admin] PATH_INFO: `/v3/roles` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:05:44.781733 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9cdc658-faef-4a0e-a947-49e7e43923ca admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:05:44.783360 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9cdc658-faef-4a0e-a947-49e7e43923ca admin admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:05:44.793799 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 1433/2869] 10.222.0.13 () {58 vars in 1122 bytes} [Wed Aug 31 10:05:44 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 10:05:44.798246 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32abbf89-2f1f-4511-8fc3-1ee4fb47972c admin admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:05:44.808165 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32abbf89-2f1f-4511-8fc3-1ee4fb47972c admin admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:05:44.813670 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4243fe0c-1a13-428a-89a0-e066fa133399 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:05:44.814280 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4243fe0c-1a13-428a-89a0-e066fa133399 admin admin] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:05:44.814575 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4243fe0c-1a13-428a-89a0-e066fa133399 admin admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:05:44.814845 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4243fe0c-1a13-428a-89a0-e066fa133399 admin admin] PATH_INFO: `/v3/roles` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:05:44.815486 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4243fe0c-1a13-428a-89a0-e066fa133399 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:05:44.817058 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4243fe0c-1a13-428a-89a0-e066fa133399 admin admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:05:44.828538 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 1437/2870] 10.222.0.13 () {58 vars in 1122 bytes} [Wed Aug 31 10:05:44 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 10:05:44.832779 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9cdc658-faef-4a0e-a947-49e7e43923ca admin admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:05:44.842877 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9cdc658-faef-4a0e-a947-49e7e43923ca admin admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:05:44.847651 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc4012c7-20b6-4b93-8efd-9a4083f70eb3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:05:44.848302 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4012c7-20b6-4b93-8efd-9a4083f70eb3 admin admin] REQUEST_METHOD: `PUT` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:05:44.848576 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4012c7-20b6-4b93-8efd-9a4083f70eb3 admin admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:05:44.848845 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4012c7-20b6-4b93-8efd-9a4083f70eb3 admin admin] PATH_INFO: `/v3/projects/cd96a40f9f7b4b169de03caade7ea07b/users/9cbe904973b54726bd7dad7ec3cc31bb/roles/6c99f5d9f1f243eaa134d8540f21095d` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:05:44.863959 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc4012c7-20b6-4b93-8efd-9a4083f70eb3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cd96a40f9f7b4b169de03caade7ea07b, user_id=9cbe904973b54726bd7dad7ec3cc31bb, role_id=6c99f5d9f1f243eaa134d8540f21095d)` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:05:44.866670 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc4012c7-20b6-4b93-8efd-9a4083f70eb3 admin admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:05:44.880560 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 1434/2871] 10.222.0.13 () {60 vars in 1483 bytes} [Wed Aug 31 10:05:44 2022] PUT /identity/v3/projects/cd96a40f9f7b4b169de03caade7ea07b/users/9cbe904973b54726bd7dad7ec3cc31bb/roles/6c99f5d9f1f243eaa134d8540f21095d => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 10:05:44.885623 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e884922e-8955-44cc-b81f-88b215dd340d None None] REQUEST_METHOD: `POST` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:05:44.885909 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e884922e-8955-44cc-b81f-88b215dd340d None None] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:05:44.886181 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e884922e-8955-44cc-b81f-88b215dd340d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:05:44.905567 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.auth.core [None req-e884922e-8955-44cc-b81f-88b215dd340d None None] MFA Rules not processed for user `fd836cf590a145df9acf19cf3ae2ac56`. Rule list: `[]` (Enabled: `True`). {{(pid=88114) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 10:05:44.906759 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.fernet_utils [None req-e884922e-8955-44cc-b81f-88b215dd340d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88114) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:05:44.964743 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 1438/2872] 10.222.0.13 () {58 vars in 958 bytes} [Wed Aug 31 10:05:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 10:05:44.974990 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc4012c7-20b6-4b93-8efd-9a4083f70eb3 admin admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:05:44.976726 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.fernet_utils [None req-cc4012c7-20b6-4b93-8efd-9a4083f70eb3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:05:45.034741 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc4012c7-20b6-4b93-8efd-9a4083f70eb3 admin admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:05:45.038514 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f1e679-c737-4abf-8419-3a1b3e545651 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9bdaacd597442c2bcde2c555f2cc140', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '49548eb44d62415582af820ab66decee', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:05:45.039121 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f1e679-c737-4abf-8419-3a1b3e545651 service neutron] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:05:45.039397 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f1e679-c737-4abf-8419-3a1b3e545651 service neutron] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:05:45.039664 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f1e679-c737-4abf-8419-3a1b3e545651 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:05:45.059363 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98f1e679-c737-4abf-8419-3a1b3e545651 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:05:45.060829 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98f1e679-c737-4abf-8419-3a1b3e545651 service neutron] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:05:45.071868 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 1435/2873] 10.222.0.13 () {60 vars in 1365 bytes} [Wed Aug 31 10:05:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 10:05:45.709181 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313f3353-5c4b-4a5c-b03c-5e0fb76ffda7 None None] REQUEST_METHOD: `POST` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:05:45.709775 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313f3353-5c4b-4a5c-b03c-5e0fb76ffda7 None None] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:05:45.709775 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313f3353-5c4b-4a5c-b03c-5e0fb76ffda7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:05:45.739598 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.auth.core [None req-313f3353-5c4b-4a5c-b03c-5e0fb76ffda7 None None] MFA Rules not processed for user `9cbe904973b54726bd7dad7ec3cc31bb`. Rule list: `[]` (Enabled: `True`). {{(pid=88114) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 10:05:45.740986 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.fernet_utils [None req-313f3353-5c4b-4a5c-b03c-5e0fb76ffda7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88114) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:05:45.809006 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 1439/2874] 10.222.0.13 () {58 vars in 958 bytes} [Wed Aug 31 10:05:45 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 10:05:45.813612 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f1e679-c737-4abf-8419-3a1b3e545651 service neutron] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:05:45.823192 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f1e679-c737-4abf-8419-3a1b3e545651 service neutron] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:05:45.827740 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-685bfecd-74ee-492a-abeb-d008394eb2bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:05:45.828332 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685bfecd-74ee-492a-abeb-d008394eb2bd admin admin] REQUEST_METHOD: `POST` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:05:45.828604 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685bfecd-74ee-492a-abeb-d008394eb2bd admin admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:05:45.828924 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685bfecd-74ee-492a-abeb-d008394eb2bd admin admin] PATH_INFO: `/v3/projects` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:05:45.829530 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-685bfecd-74ee-492a-abeb-d008394eb2bd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:05:45.831433 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-685bfecd-74ee-492a-abeb-d008394eb2bd admin admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:05:45.859765 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 1436/2875] 10.222.0.13 () {60 vars in 1153 bytes} [Wed Aug 31 10:05:45 2022] POST /identity/v3/projects => generated 373 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 10:05:45.864184 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-313f3353-5c4b-4a5c-b03c-5e0fb76ffda7 None None] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:05:45.875138 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-313f3353-5c4b-4a5c-b03c-5e0fb76ffda7 None None] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:05:45.932529 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffce195b-2601-40ce-aef7-858c0a864809 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:05:45.933139 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffce195b-2601-40ce-aef7-858c0a864809 admin admin] REQUEST_METHOD: `POST` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:05:45.933451 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffce195b-2601-40ce-aef7-858c0a864809 admin admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:05:45.933728 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffce195b-2601-40ce-aef7-858c0a864809 admin admin] PATH_INFO: `/v3/users` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:05:45.934355 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffce195b-2601-40ce-aef7-858c0a864809 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:05:45.936322 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffce195b-2601-40ce-aef7-858c0a864809 admin admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:05:45.957666 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 1440/2876] 10.222.0.13 () {60 vars in 1144 bytes} [Wed Aug 31 10:05:45 2022] POST /identity/v3/users => generated 293 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 10:05:45.970398 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-685bfecd-74ee-492a-abeb-d008394eb2bd admin admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:05:45.980692 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-685bfecd-74ee-492a-abeb-d008394eb2bd admin admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:05:45.985781 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f21b6a-fb48-4214-ab0f-b0be39f9054c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:05:45.986379 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f21b6a-fb48-4214-ab0f-b0be39f9054c admin admin] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:05:45.986654 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f21b6a-fb48-4214-ab0f-b0be39f9054c admin admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:05:45.986922 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f21b6a-fb48-4214-ab0f-b0be39f9054c admin admin] PATH_INFO: `/v3/roles` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:05:45.987559 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22f21b6a-fb48-4214-ab0f-b0be39f9054c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:05:45.989147 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22f21b6a-fb48-4214-ab0f-b0be39f9054c admin admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:05:46.000327 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 1437/2877] 10.222.0.13 () {58 vars in 1122 bytes} [Wed Aug 31 10:05:45 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 10:05:46.004561 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffce195b-2601-40ce-aef7-858c0a864809 admin admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:05:46.010882 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:05:46.015357 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffce195b-2601-40ce-aef7-858c0a864809 admin admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:05:46.021613 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73259120-9fa7-4f7f-8d32-0ab57e6a5790 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:05:46.022438 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73259120-9fa7-4f7f-8d32-0ab57e6a5790 admin admin] REQUEST_METHOD: `PUT` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:05:46.022807 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73259120-9fa7-4f7f-8d32-0ab57e6a5790 admin admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:05:46.023168 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73259120-9fa7-4f7f-8d32-0ab57e6a5790 admin admin] PATH_INFO: `/v3/projects/13c825a025994cb2aeb2454f4eaf526e/users/fb01ac2f8a984bcea1ff460abc39ff0a/roles/5d4ee63a3add47fc9a6b8b16d39e0a1f` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:05:46.038892 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73259120-9fa7-4f7f-8d32-0ab57e6a5790 admin admin] RBAC: Authorizing `identity:create_grant(project_id=13c825a025994cb2aeb2454f4eaf526e, user_id=fb01ac2f8a984bcea1ff460abc39ff0a, role_id=5d4ee63a3add47fc9a6b8b16d39e0a1f)` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:05:46.041537 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73259120-9fa7-4f7f-8d32-0ab57e6a5790 admin admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:05:46.057549 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 1441/2878] 10.222.0.13 () {60 vars in 1483 bytes} [Wed Aug 31 10:05:46 2022] PUT /identity/v3/projects/13c825a025994cb2aeb2454f4eaf526e/users/fb01ac2f8a984bcea1ff460abc39ff0a/roles/5d4ee63a3add47fc9a6b8b16d39e0a1f => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 10:05:46.061682 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f21b6a-fb48-4214-ab0f-b0be39f9054c admin admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:05:46.071991 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f21b6a-fb48-4214-ab0f-b0be39f9054c admin admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:05:46.123636 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:05:46.130020 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e852c1-d13f-48fc-9c0d-c21455b2dd77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:05:46.130871 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e852c1-d13f-48fc-9c0d-c21455b2dd77 admin admin] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:05:46.131158 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e852c1-d13f-48fc-9c0d-c21455b2dd77 admin admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:05:46.131492 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e852c1-d13f-48fc-9c0d-c21455b2dd77 admin admin] PATH_INFO: `/v3/roles` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:05:46.132365 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64e852c1-d13f-48fc-9c0d-c21455b2dd77 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:05:46.134214 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64e852c1-d13f-48fc-9c0d-c21455b2dd77 admin admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:05:46.145080 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 1438/2879] 10.222.0.13 () {58 vars in 1122 bytes} [Wed Aug 31 10:05:46 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 10:05:46.149308 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73259120-9fa7-4f7f-8d32-0ab57e6a5790 admin admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:05:46.160598 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73259120-9fa7-4f7f-8d32-0ab57e6a5790 admin admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:05:46.165298 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb110580-18dc-4d02-b6b7-22aea15772ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:05:46.165982 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb110580-18dc-4d02-b6b7-22aea15772ea admin admin] REQUEST_METHOD: `PUT` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:05:46.166283 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb110580-18dc-4d02-b6b7-22aea15772ea admin admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:05:46.166554 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb110580-18dc-4d02-b6b7-22aea15772ea admin admin] PATH_INFO: `/v3/projects/13c825a025994cb2aeb2454f4eaf526e/users/fb01ac2f8a984bcea1ff460abc39ff0a/roles/5d4ee63a3add47fc9a6b8b16d39e0a1f` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:05:46.168481 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb110580-18dc-4d02-b6b7-22aea15772ea admin admin] RBAC: Authorizing `identity:create_grant(project_id=13c825a025994cb2aeb2454f4eaf526e, user_id=fb01ac2f8a984bcea1ff460abc39ff0a, role_id=5d4ee63a3add47fc9a6b8b16d39e0a1f)` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:05:46.171402 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb110580-18dc-4d02-b6b7-22aea15772ea admin admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:05:46.181511 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 1442/2880] 10.222.0.13 () {60 vars in 1483 bytes} [Wed Aug 31 10:05:46 2022] PUT /identity/v3/projects/13c825a025994cb2aeb2454f4eaf526e/users/fb01ac2f8a984bcea1ff460abc39ff0a/roles/5d4ee63a3add47fc9a6b8b16d39e0a1f => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 10:05:46.822148 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85fd0809-ccec-42ba-abbe-3fed9e3d080c None None] REQUEST_METHOD: `POST` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:05:46.822623 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85fd0809-ccec-42ba-abbe-3fed9e3d080c None None] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:05:46.822761 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85fd0809-ccec-42ba-abbe-3fed9e3d080c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:05:46.854637 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.auth.core [None req-85fd0809-ccec-42ba-abbe-3fed9e3d080c None None] MFA Rules not processed for user `fb01ac2f8a984bcea1ff460abc39ff0a`. Rule list: `[]` (Enabled: `True`). {{(pid=88113) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 10:05:46.855762 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.fernet_utils [None req-85fd0809-ccec-42ba-abbe-3fed9e3d080c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:05:46.927658 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 1439/2881] 10.222.0.13 () {58 vars in 958 bytes} [Wed Aug 31 10:05:46 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 10:05:46.939481 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb110580-18dc-4d02-b6b7-22aea15772ea admin admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:05:46.941243 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.fernet_utils [None req-fb110580-18dc-4d02-b6b7-22aea15772ea admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88114) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:05:47.008177 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb110580-18dc-4d02-b6b7-22aea15772ea admin admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:05:47.011749 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a5c7fb8-ff08-4bca-b7c9-2d2cc106bafd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d52b69c8e94441dacfcff944326f169', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '49548eb44d62415582af820ab66decee', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:05:47.012348 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a5c7fb8-ff08-4bca-b7c9-2d2cc106bafd service cinder] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:05:47.012617 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a5c7fb8-ff08-4bca-b7c9-2d2cc106bafd service cinder] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:05:47.012879 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a5c7fb8-ff08-4bca-b7c9-2d2cc106bafd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:05:47.075005 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a5c7fb8-ff08-4bca-b7c9-2d2cc106bafd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:05:47.076721 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a5c7fb8-ff08-4bca-b7c9-2d2cc106bafd service cinder] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:05:47.086293 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 1443/2882] 10.222.0.13 () {60 vars in 1365 bytes} [Wed Aug 31 10:05:46 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 10:05:48.443531 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85fd0809-ccec-42ba-abbe-3fed9e3d080c None None] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:05:48.444924 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.fernet_utils [None req-85fd0809-ccec-42ba-abbe-3fed9e3d080c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:05:48.457959 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85fd0809-ccec-42ba-abbe-3fed9e3d080c None None] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:05:48.461505 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-732c8daf-b5bf-4385-a5aa-4f603491a47c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d52b69c8e94441dacfcff944326f169', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '49548eb44d62415582af820ab66decee', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:05:48.462104 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732c8daf-b5bf-4385-a5aa-4f603491a47c service cinder] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:05:48.462383 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732c8daf-b5bf-4385-a5aa-4f603491a47c service cinder] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:05:48.462654 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732c8daf-b5bf-4385-a5aa-4f603491a47c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:05:48.472860 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-732c8daf-b5bf-4385-a5aa-4f603491a47c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:05:48.474400 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-732c8daf-b5bf-4385-a5aa-4f603491a47c service cinder] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:05:48.484215 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 1440/2883] 10.222.0.13 () {60 vars in 1365 bytes} [Wed Aug 31 10:05:48 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 10:05:57.082401 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=61,keystone:INSERT=2 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:05:58.479590 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=43 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:06:01.428924 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a5c7fb8-ff08-4bca-b7c9-2d2cc106bafd service cinder] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:06:01.430410 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.fernet_utils [None req-1a5c7fb8-ff08-4bca-b7c9-2d2cc106bafd service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88114) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:06:01.482345 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-732c8daf-b5bf-4385-a5aa-4f603491a47c service cinder] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:06:01.571282 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a5c7fb8-ff08-4bca-b7c9-2d2cc106bafd service cinder] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:06:01.577006 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de81bf5d-3943-4c94-888f-410f58bd4f1c service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ff3aff5ef444910be6dbb989d0e3194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '49548eb44d62415582af820ab66decee', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:06:01.577810 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de81bf5d-3943-4c94-888f-410f58bd4f1c service placement] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:06:01.578088 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de81bf5d-3943-4c94-888f-410f58bd4f1c service placement] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:06:01.578366 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de81bf5d-3943-4c94-888f-410f58bd4f1c service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:06:01.580219 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.fernet_utils [None req-de81bf5d-3943-4c94-888f-410f58bd4f1c service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88114) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:06:01.591867 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de81bf5d-3943-4c94-888f-410f58bd4f1c service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:06:01.594243 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de81bf5d-3943-4c94-888f-410f58bd4f1c service placement] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:06:01.598653 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-732c8daf-b5bf-4385-a5aa-4f603491a47c service cinder] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:06:01.604134 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c60b04b3-d334-42c0-b2a1-917dd47a001a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:06:01.604678 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 1444/2884] 10.222.0.13 () {60 vars in 1365 bytes} [Wed Aug 31 10:06:01 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 176 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 10:06:01.604805 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60b04b3-d334-42c0-b2a1-917dd47a001a admin admin] REQUEST_METHOD: `DELETE` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:06:01.605060 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60b04b3-d334-42c0-b2a1-917dd47a001a admin admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:06:01.605337 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60b04b3-d334-42c0-b2a1-917dd47a001a admin admin] PATH_INFO: `/v3/users/9cbe904973b54726bd7dad7ec3cc31bb` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:06:01.606598 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c60b04b3-d334-42c0-b2a1-917dd47a001a admin admin] RBAC: Authorizing `identity:delete_user(user_id=9cbe904973b54726bd7dad7ec3cc31bb)` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:06:01.608557 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c60b04b3-d334-42c0-b2a1-917dd47a001a admin admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:06:01.660675 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.notifications [None req-c60b04b3-d334-42c0-b2a1-917dd47a001a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9cbe904973b54726bd7dad7ec3cc31bb'} {{(pid=88113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 10:06:01.667675 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.notifications [None req-c60b04b3-d334-42c0-b2a1-917dd47a001a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9cbe904973b54726bd7dad7ec3cc31bb'} {{(pid=88113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 10:06:01.679631 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.notifications [None req-c60b04b3-d334-42c0-b2a1-917dd47a001a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9cbe904973b54726bd7dad7ec3cc31bb'} {{(pid=88113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 10:06:01.680245 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.notifications [None req-c60b04b3-d334-42c0-b2a1-917dd47a001a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9cbe904973b54726bd7dad7ec3cc31bb'} {{(pid=88113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 10:06:01.688915 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.notifications [None req-c60b04b3-d334-42c0-b2a1-917dd47a001a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9cbe904973b54726bd7dad7ec3cc31bb'} {{(pid=88113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 10:06:01.695139 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 1441/2885] 10.222.0.13 () {58 vars in 1224 bytes} [Wed Aug 31 10:06:01 2022] DELETE /identity/v3/users/9cbe904973b54726bd7dad7ec3cc31bb => generated 0 bytes in 213 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 10:06:01.874981 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de81bf5d-3943-4c94-888f-410f58bd4f1c service placement] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:06:01.886466 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de81bf5d-3943-4c94-888f-410f58bd4f1c service placement] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:06:01.949499 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e67916-a6d1-4115-8a74-f8d5d4f257e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:06:01.950135 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e67916-a6d1-4115-8a74-f8d5d4f257e8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:06:01.950415 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e67916-a6d1-4115-8a74-f8d5d4f257e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:06:01.950689 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e67916-a6d1-4115-8a74-f8d5d4f257e8 admin admin] PATH_INFO: `/v3/projects/cd96a40f9f7b4b169de03caade7ea07b` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:06:01.951950 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95e67916-a6d1-4115-8a74-f8d5d4f257e8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cd96a40f9f7b4b169de03caade7ea07b)` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:06:01.953886 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95e67916-a6d1-4115-8a74-f8d5d4f257e8 admin admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:06:01.998890 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:06:02.003145 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.notifications [None req-95e67916-a6d1-4115-8a74-f8d5d4f257e8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cd96a40f9f7b4b169de03caade7ea07b', 'request_id': 'req-95e67916-a6d1-4115-8a74-f8d5d4f257e8'} {{(pid=88114) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 10:06:02.009726 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.notifications [None req-95e67916-a6d1-4115-8a74-f8d5d4f257e8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cd96a40f9f7b4b169de03caade7ea07b', 'request_id': 'req-95e67916-a6d1-4115-8a74-f8d5d4f257e8'} {{(pid=88114) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 10:06:02.016213 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.notifications [None req-95e67916-a6d1-4115-8a74-f8d5d4f257e8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cd96a40f9f7b4b169de03caade7ea07b', 'request_id': 'req-95e67916-a6d1-4115-8a74-f8d5d4f257e8'} {{(pid=88114) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 10:06:02.041278 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.notifications [None req-95e67916-a6d1-4115-8a74-f8d5d4f257e8 admin admin] The token cache is being invalidate because project cd96a40f9f7b4b169de03caade7ea07b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88114) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 10:06:02.041589 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.notifications [None req-95e67916-a6d1-4115-8a74-f8d5d4f257e8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88114) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 10:06:02.042211 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.notifications [None req-95e67916-a6d1-4115-8a74-f8d5d4f257e8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88114) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 10:06:02.044262 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 1445/2886] 10.222.0.13 () {58 vars in 1233 bytes} [Wed Aug 31 10:06:01 2022] DELETE /identity/v3/projects/cd96a40f9f7b4b169de03caade7ea07b => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 10:06:02.049669 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c60b04b3-d334-42c0-b2a1-917dd47a001a admin admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:06:02.051421 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.fernet_utils [None req-c60b04b3-d334-42c0-b2a1-917dd47a001a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:06:02.064819 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c60b04b3-d334-42c0-b2a1-917dd47a001a admin admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:06:02.100296 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:06:02.125048 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b02960-bdfe-48bf-8565-51eeb83354a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:06:02.125704 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b02960-bdfe-48bf-8565-51eeb83354a8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:06:02.125989 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b02960-bdfe-48bf-8565-51eeb83354a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:06:02.126264 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b02960-bdfe-48bf-8565-51eeb83354a8 admin admin] PATH_INFO: `/v3/users/fb01ac2f8a984bcea1ff460abc39ff0a` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:06:02.127393 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85b02960-bdfe-48bf-8565-51eeb83354a8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fb01ac2f8a984bcea1ff460abc39ff0a)` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:06:02.129307 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85b02960-bdfe-48bf-8565-51eeb83354a8 admin admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:06:02.175566 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.notifications [None req-85b02960-bdfe-48bf-8565-51eeb83354a8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fb01ac2f8a984bcea1ff460abc39ff0a'} {{(pid=88113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 10:06:02.182971 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.notifications [None req-85b02960-bdfe-48bf-8565-51eeb83354a8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fb01ac2f8a984bcea1ff460abc39ff0a'} {{(pid=88113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 10:06:02.195346 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.notifications [None req-85b02960-bdfe-48bf-8565-51eeb83354a8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fb01ac2f8a984bcea1ff460abc39ff0a'} {{(pid=88113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 10:06:02.196056 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.notifications [None req-85b02960-bdfe-48bf-8565-51eeb83354a8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fb01ac2f8a984bcea1ff460abc39ff0a'} {{(pid=88113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 10:06:02.204137 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.notifications [None req-85b02960-bdfe-48bf-8565-51eeb83354a8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fb01ac2f8a984bcea1ff460abc39ff0a'} {{(pid=88113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 10:06:02.210137 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 1442/2887] 10.222.0.13 () {58 vars in 1224 bytes} [Wed Aug 31 10:06:02 2022] DELETE /identity/v3/users/fb01ac2f8a984bcea1ff460abc39ff0a => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 10:06:02.383815 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e67916-a6d1-4115-8a74-f8d5d4f257e8 admin admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:06:02.385238 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.fernet_utils [None req-95e67916-a6d1-4115-8a74-f8d5d4f257e8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88114) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:06:02.402151 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e67916-a6d1-4115-8a74-f8d5d4f257e8 admin admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:06:02.468584 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f343ac-0352-491a-acb3-3ac30282ac4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:06:02.470034 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f343ac-0352-491a-acb3-3ac30282ac4a admin admin] REQUEST_METHOD: `DELETE` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:06:02.470392 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f343ac-0352-491a-acb3-3ac30282ac4a admin admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:06:02.470676 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f343ac-0352-491a-acb3-3ac30282ac4a admin admin] PATH_INFO: `/v3/projects/13c825a025994cb2aeb2454f4eaf526e` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:06:02.472031 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72f343ac-0352-491a-acb3-3ac30282ac4a admin admin] RBAC: Authorizing `identity:delete_project(project_id=13c825a025994cb2aeb2454f4eaf526e)` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:06:02.473963 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72f343ac-0352-491a-acb3-3ac30282ac4a admin admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:06:02.523309 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.notifications [None req-72f343ac-0352-491a-acb3-3ac30282ac4a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '13c825a025994cb2aeb2454f4eaf526e', 'request_id': 'req-72f343ac-0352-491a-acb3-3ac30282ac4a'} {{(pid=88114) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 10:06:02.529345 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.notifications [None req-72f343ac-0352-491a-acb3-3ac30282ac4a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '13c825a025994cb2aeb2454f4eaf526e', 'request_id': 'req-72f343ac-0352-491a-acb3-3ac30282ac4a'} {{(pid=88114) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 10:06:02.535391 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.notifications [None req-72f343ac-0352-491a-acb3-3ac30282ac4a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '13c825a025994cb2aeb2454f4eaf526e', 'request_id': 'req-72f343ac-0352-491a-acb3-3ac30282ac4a'} {{(pid=88114) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 10:06:02.547920 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.notifications [None req-72f343ac-0352-491a-acb3-3ac30282ac4a admin admin] The token cache is being invalidate because project 13c825a025994cb2aeb2454f4eaf526e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88114) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 10:06:02.548235 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.notifications [None req-72f343ac-0352-491a-acb3-3ac30282ac4a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88114) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 10:06:02.548847 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.notifications [None req-72f343ac-0352-491a-acb3-3ac30282ac4a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88114) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 10:06:02.550373 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 1446/2888] 10.222.0.13 () {58 vars in 1233 bytes} [Wed Aug 31 10:06:02 2022] DELETE /identity/v3/projects/13c825a025994cb2aeb2454f4eaf526e => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 10:06:02.555308 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f30e97-034b-4dc0-b3da-db511af6585d None None] REQUEST_METHOD: `POST` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:06:02.555867 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f30e97-034b-4dc0-b3da-db511af6585d None None] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:06:02.555867 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f30e97-034b-4dc0-b3da-db511af6585d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:06:02.578375 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.auth.core [None req-95f30e97-034b-4dc0-b3da-db511af6585d None None] MFA Rules not processed for user `fd836cf590a145df9acf19cf3ae2ac56`. Rule list: `[]` (Enabled: `True`). {{(pid=88113) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 10:06:02.579593 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.fernet_utils [None req-95f30e97-034b-4dc0-b3da-db511af6585d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:06:02.641803 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 1443/2889] 10.222.0.13 () {58 vars in 958 bytes} [Wed Aug 31 10:06:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 10:06:02.650130 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3fcbd4-c2a4-4066-8cfb-bdcf413bb676 None None] REQUEST_METHOD: `POST` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:06:02.650411 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3fcbd4-c2a4-4066-8cfb-bdcf413bb676 None None] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:06:02.650682 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3fcbd4-c2a4-4066-8cfb-bdcf413bb676 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:06:02.671389 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.auth.core [None req-0b3fcbd4-c2a4-4066-8cfb-bdcf413bb676 None None] MFA Rules not processed for user `fd836cf590a145df9acf19cf3ae2ac56`. Rule list: `[]` (Enabled: `True`). {{(pid=88114) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 10:06:02.672587 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.fernet_utils [None req-0b3fcbd4-c2a4-4066-8cfb-bdcf413bb676 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88114) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:06:02.680328 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 1447/2890] 10.222.0.13 () {58 vars in 958 bytes} [Wed Aug 31 10:06:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 10:06:02.684608 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f30e97-034b-4dc0-b3da-db511af6585d None None] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:06:02.695216 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f30e97-034b-4dc0-b3da-db511af6585d None None] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:06:02.699795 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf071db-96db-4810-b10e-c40b236f5ca7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:06:02.700398 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf071db-96db-4810-b10e-c40b236f5ca7 admin admin] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:06:02.700671 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf071db-96db-4810-b10e-c40b236f5ca7 admin admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:06:02.700951 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf071db-96db-4810-b10e-c40b236f5ca7 admin admin] PATH_INFO: `/v3/domains` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:06:02.702234 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbf071db-96db-4810-b10e-c40b236f5ca7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88113) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:06:02.702619 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbf071db-96db-4810-b10e-c40b236f5ca7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:06:02.704228 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbf071db-96db-4810-b10e-c40b236f5ca7 admin admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:06:02.715808 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 1444/2891] 10.222.0.13 () {58 vars in 1153 bytes} [Wed Aug 31 10:06:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:06:02.720235 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b3fcbd4-c2a4-4066-8cfb-bdcf413bb676 None None] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:06:02.729779 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b3fcbd4-c2a4-4066-8cfb-bdcf413bb676 None None] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:06:02.734521 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de04a6d1-87ad-4e50-aba2-67c4b2a4af37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:06:02.735371 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de04a6d1-87ad-4e50-aba2-67c4b2a4af37 admin admin] REQUEST_METHOD: `POST` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:06:02.735648 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de04a6d1-87ad-4e50-aba2-67c4b2a4af37 admin admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:06:02.735914 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de04a6d1-87ad-4e50-aba2-67c4b2a4af37 admin admin] PATH_INFO: `/v3/projects` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:06:02.736576 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de04a6d1-87ad-4e50-aba2-67c4b2a4af37 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:06:02.738632 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de04a6d1-87ad-4e50-aba2-67c4b2a4af37 admin admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:06:02.768602 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 1448/2892] 10.222.0.13 () {60 vars in 1153 bytes} [Wed Aug 31 10:06:02 2022] POST /identity/v3/projects => generated 391 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 10:06:02.772797 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf071db-96db-4810-b10e-c40b236f5ca7 admin admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:06:02.783680 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf071db-96db-4810-b10e-c40b236f5ca7 admin admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:06:02.825516 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:06:02.841507 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bcd559e-a190-42ce-9f2e-af168395fdd8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:06:02.842254 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bcd559e-a190-42ce-9f2e-af168395fdd8 admin admin] REQUEST_METHOD: `POST` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:06:02.842708 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bcd559e-a190-42ce-9f2e-af168395fdd8 admin admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:06:02.842975 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bcd559e-a190-42ce-9f2e-af168395fdd8 admin admin] PATH_INFO: `/v3/users` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:06:02.843619 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bcd559e-a190-42ce-9f2e-af168395fdd8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:06:02.845864 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bcd559e-a190-42ce-9f2e-af168395fdd8 admin admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:06:02.867729 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 1445/2893] 10.222.0.13 () {60 vars in 1144 bytes} [Wed Aug 31 10:06:02 2022] POST /identity/v3/users => generated 296 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 10:06:02.871866 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de04a6d1-87ad-4e50-aba2-67c4b2a4af37 admin admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:06:02.882603 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de04a6d1-87ad-4e50-aba2-67c4b2a4af37 admin admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:06:02.887563 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68267b67-efd4-4ef9-af66-8a2a6f773817 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:06:02.888152 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68267b67-efd4-4ef9-af66-8a2a6f773817 admin admin] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:06:02.888428 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68267b67-efd4-4ef9-af66-8a2a6f773817 admin admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:06:02.888692 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68267b67-efd4-4ef9-af66-8a2a6f773817 admin admin] PATH_INFO: `/v3/roles` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:06:02.889400 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68267b67-efd4-4ef9-af66-8a2a6f773817 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:06:02.891280 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68267b67-efd4-4ef9-af66-8a2a6f773817 admin admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:06:02.901252 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 1449/2894] 10.222.0.13 () {58 vars in 1122 bytes} [Wed Aug 31 10:06:02 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 10:06:02.905228 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bcd559e-a190-42ce-9f2e-af168395fdd8 admin admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:06:02.915187 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bcd559e-a190-42ce-9f2e-af168395fdd8 admin admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:06:02.919817 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e4ec16-4181-4b51-bbd8-78df7803cdda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:06:02.920946 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e4ec16-4181-4b51-bbd8-78df7803cdda admin admin] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:06:02.921216 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e4ec16-4181-4b51-bbd8-78df7803cdda admin admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:06:02.921518 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e4ec16-4181-4b51-bbd8-78df7803cdda admin admin] PATH_INFO: `/v3/roles` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:06:02.922153 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0e4ec16-4181-4b51-bbd8-78df7803cdda admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:06:02.923750 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0e4ec16-4181-4b51-bbd8-78df7803cdda admin admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:06:02.934337 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 1446/2895] 10.222.0.13 () {58 vars in 1122 bytes} [Wed Aug 31 10:06:02 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 10:06:02.939767 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68267b67-efd4-4ef9-af66-8a2a6f773817 admin admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:06:02.949618 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68267b67-efd4-4ef9-af66-8a2a6f773817 admin admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:06:02.955060 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b500cd53-19d2-430d-bd1b-021ac85d70a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:06:02.955699 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b500cd53-19d2-430d-bd1b-021ac85d70a5 admin admin] REQUEST_METHOD: `PUT` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:06:02.955975 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b500cd53-19d2-430d-bd1b-021ac85d70a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:06:02.956243 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b500cd53-19d2-430d-bd1b-021ac85d70a5 admin admin] PATH_INFO: `/v3/projects/80a6f0d1b93941a09843ef5125517996/users/1f364d51e15f451fb49beaf618d10131/roles/6c99f5d9f1f243eaa134d8540f21095d` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:06:02.966004 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=86,keystone:DELETE=11,keystone:INSERT=3 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:06:02.974885 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b500cd53-19d2-430d-bd1b-021ac85d70a5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=80a6f0d1b93941a09843ef5125517996, user_id=1f364d51e15f451fb49beaf618d10131, role_id=6c99f5d9f1f243eaa134d8540f21095d)` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:06:02.977825 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b500cd53-19d2-430d-bd1b-021ac85d70a5 admin admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:06:02.991906 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 1450/2896] 10.222.0.13 () {60 vars in 1483 bytes} [Wed Aug 31 10:06:02 2022] PUT /identity/v3/projects/80a6f0d1b93941a09843ef5125517996/users/1f364d51e15f451fb49beaf618d10131/roles/6c99f5d9f1f243eaa134d8540f21095d => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 10:06:03.004686 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f56de08-fba8-47c3-bd29-079dd451502e None None] REQUEST_METHOD: `POST` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:06:03.004973 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f56de08-fba8-47c3-bd29-079dd451502e None None] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:06:03.005248 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f56de08-fba8-47c3-bd29-079dd451502e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:06:03.032586 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.auth.core [None req-0f56de08-fba8-47c3-bd29-079dd451502e None None] MFA Rules not processed for user `1f364d51e15f451fb49beaf618d10131`. Rule list: `[]` (Enabled: `True`). {{(pid=88113) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 10:06:03.033827 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.fernet_utils [None req-0f56de08-fba8-47c3-bd29-079dd451502e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:06:03.100864 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 1447/2897] 10.222.0.13 () {58 vars in 958 bytes} [Wed Aug 31 10:06:03 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 10:06:03.113167 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b500cd53-19d2-430d-bd1b-021ac85d70a5 admin admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:06:03.114799 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.fernet_utils [None req-b500cd53-19d2-430d-bd1b-021ac85d70a5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88114) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:06:03.169857 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b500cd53-19d2-430d-bd1b-021ac85d70a5 admin admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:06:03.174418 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd2f7327-90d4-4ee7-9e10-9be935cb77e6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d52b69c8e94441dacfcff944326f169', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '49548eb44d62415582af820ab66decee', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:06:03.175014 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2f7327-90d4-4ee7-9e10-9be935cb77e6 service cinder] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:06:03.175287 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2f7327-90d4-4ee7-9e10-9be935cb77e6 service cinder] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:06:03.175548 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2f7327-90d4-4ee7-9e10-9be935cb77e6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:06:03.184147 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd2f7327-90d4-4ee7-9e10-9be935cb77e6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:06:03.185586 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd2f7327-90d4-4ee7-9e10-9be935cb77e6 service cinder] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:06:03.195063 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 1451/2898] 10.222.0.13 () {60 vars in 1365 bytes} [Wed Aug 31 10:06:03 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 10:06:13.098196 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=46,keystone:INSERT=3 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:06:13.190854 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=31,keystone:INSERT=1 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:06:24.926186 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f56de08-fba8-47c3-bd29-079dd451502e None None] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:06:24.990839 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f56de08-fba8-47c3-bd29-079dd451502e None None] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:06:24.995373 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c6e323-9d33-4aa9-a3a8-cf616fc31974 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:06:24.996005 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c6e323-9d33-4aa9-a3a8-cf616fc31974 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:06:24.996287 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c6e323-9d33-4aa9-a3a8-cf616fc31974 admin admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:06:24.996560 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c6e323-9d33-4aa9-a3a8-cf616fc31974 admin admin] PATH_INFO: `/v3/users/1f364d51e15f451fb49beaf618d10131` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:06:24.997718 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c6e323-9d33-4aa9-a3a8-cf616fc31974 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1f364d51e15f451fb49beaf618d10131)` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:06:24.999819 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c6e323-9d33-4aa9-a3a8-cf616fc31974 admin admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:06:25.044955 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.notifications [None req-58c6e323-9d33-4aa9-a3a8-cf616fc31974 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1f364d51e15f451fb49beaf618d10131'} {{(pid=88113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 10:06:25.052186 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.notifications [None req-58c6e323-9d33-4aa9-a3a8-cf616fc31974 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1f364d51e15f451fb49beaf618d10131'} {{(pid=88113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 10:06:25.069613 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.notifications [None req-58c6e323-9d33-4aa9-a3a8-cf616fc31974 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1f364d51e15f451fb49beaf618d10131'} {{(pid=88113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 10:06:25.070377 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.notifications [None req-58c6e323-9d33-4aa9-a3a8-cf616fc31974 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1f364d51e15f451fb49beaf618d10131'} {{(pid=88113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 10:06:25.079543 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.notifications [None req-58c6e323-9d33-4aa9-a3a8-cf616fc31974 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1f364d51e15f451fb49beaf618d10131'} {{(pid=88113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 10:06:25.086018 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 1448/2899] 10.222.0.13 () {58 vars in 1224 bytes} [Wed Aug 31 10:06:24 2022] DELETE /identity/v3/users/1f364d51e15f451fb49beaf618d10131 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 10:06:25.090646 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed8e981-2092-4fe7-8033-d7d12c3a7d29 None None] REQUEST_METHOD: `POST` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:06:25.091130 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed8e981-2092-4fe7-8033-d7d12c3a7d29 None None] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:06:25.091252 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed8e981-2092-4fe7-8033-d7d12c3a7d29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:06:25.120516 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.auth.core [None req-1ed8e981-2092-4fe7-8033-d7d12c3a7d29 None None] MFA Rules not processed for user `fd836cf590a145df9acf19cf3ae2ac56`. Rule list: `[]` (Enabled: `True`). {{(pid=88114) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 10:06:25.121738 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.fernet_utils [None req-1ed8e981-2092-4fe7-8033-d7d12c3a7d29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88114) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:06:25.179627 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 1452/2900] 10.222.0.13 () {58 vars in 958 bytes} [Wed Aug 31 10:06:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 10:06:25.189831 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c6e323-9d33-4aa9-a3a8-cf616fc31974 admin admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:06:25.191267 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.fernet_utils [None req-58c6e323-9d33-4aa9-a3a8-cf616fc31974 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:06:25.271701 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c6e323-9d33-4aa9-a3a8-cf616fc31974 admin admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:06:25.275480 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b259e60-479a-4ba4-91b1-a74f01dbb00e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9bdaacd597442c2bcde2c555f2cc140', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '49548eb44d62415582af820ab66decee', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:06:25.276079 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b259e60-479a-4ba4-91b1-a74f01dbb00e service neutron] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:06:25.276352 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b259e60-479a-4ba4-91b1-a74f01dbb00e service neutron] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:06:25.276617 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b259e60-479a-4ba4-91b1-a74f01dbb00e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:06:25.286136 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b259e60-479a-4ba4-91b1-a74f01dbb00e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:06:25.287572 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b259e60-479a-4ba4-91b1-a74f01dbb00e service neutron] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:06:25.297611 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 1449/2901] 10.222.0.13 () {60 vars in 1365 bytes} [Wed Aug 31 10:06:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 10:06:25.572855 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed8e981-2092-4fe7-8033-d7d12c3a7d29 None None] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:06:25.574336 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.fernet_utils [None req-1ed8e981-2092-4fe7-8033-d7d12c3a7d29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88114) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:06:25.589294 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed8e981-2092-4fe7-8033-d7d12c3a7d29 None None] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:06:25.595265 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da06211-fede-4ebc-bb95-bdc9ba7d545e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:06:25.596001 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da06211-fede-4ebc-bb95-bdc9ba7d545e admin admin] REQUEST_METHOD: `DELETE` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:06:25.596345 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da06211-fede-4ebc-bb95-bdc9ba7d545e admin admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:06:25.596683 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da06211-fede-4ebc-bb95-bdc9ba7d545e admin admin] PATH_INFO: `/v3/projects/80a6f0d1b93941a09843ef5125517996` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:06:25.597928 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6da06211-fede-4ebc-bb95-bdc9ba7d545e admin admin] RBAC: Authorizing `identity:delete_project(project_id=80a6f0d1b93941a09843ef5125517996)` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:06:25.599917 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6da06211-fede-4ebc-bb95-bdc9ba7d545e admin admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:06:25.647653 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.notifications [None req-6da06211-fede-4ebc-bb95-bdc9ba7d545e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '80a6f0d1b93941a09843ef5125517996', 'request_id': 'req-6da06211-fede-4ebc-bb95-bdc9ba7d545e'} {{(pid=88114) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 10:06:25.653850 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.notifications [None req-6da06211-fede-4ebc-bb95-bdc9ba7d545e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '80a6f0d1b93941a09843ef5125517996', 'request_id': 'req-6da06211-fede-4ebc-bb95-bdc9ba7d545e'} {{(pid=88114) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 10:06:25.660456 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.notifications [None req-6da06211-fede-4ebc-bb95-bdc9ba7d545e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '80a6f0d1b93941a09843ef5125517996', 'request_id': 'req-6da06211-fede-4ebc-bb95-bdc9ba7d545e'} {{(pid=88114) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 10:06:25.674481 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.notifications [None req-6da06211-fede-4ebc-bb95-bdc9ba7d545e admin admin] The token cache is being invalidate because project 80a6f0d1b93941a09843ef5125517996 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88114) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 10:06:25.674699 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.notifications [None req-6da06211-fede-4ebc-bb95-bdc9ba7d545e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88114) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 10:06:25.675309 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.notifications [None req-6da06211-fede-4ebc-bb95-bdc9ba7d545e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88114) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 10:06:25.676819 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 1453/2902] 10.222.0.13 () {58 vars in 1233 bytes} [Wed Aug 31 10:06:25 2022] DELETE /identity/v3/projects/80a6f0d1b93941a09843ef5125517996 => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 10:06:25.681805 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0bf6ae-3b44-4837-a654-5f460933c875 None None] REQUEST_METHOD: `POST` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:06:25.682095 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0bf6ae-3b44-4837-a654-5f460933c875 None None] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:06:25.682372 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0bf6ae-3b44-4837-a654-5f460933c875 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:06:25.688046 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:06:25.735998 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.auth.core [None req-bc0bf6ae-3b44-4837-a654-5f460933c875 None None] MFA Rules not processed for user `fd836cf590a145df9acf19cf3ae2ac56`. Rule list: `[]` (Enabled: `True`). {{(pid=88113) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 10:06:25.737107 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.fernet_utils [None req-bc0bf6ae-3b44-4837-a654-5f460933c875 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:06:25.795277 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 1450/2903] 10.222.0.13 () {58 vars in 958 bytes} [Wed Aug 31 10:06:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 10:06:25.803307 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9445386-bcb7-46a1-b352-c371303875de None None] REQUEST_METHOD: `POST` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:06:25.803603 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9445386-bcb7-46a1-b352-c371303875de None None] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:06:25.803853 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9445386-bcb7-46a1-b352-c371303875de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:06:25.824942 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.auth.core [None req-c9445386-bcb7-46a1-b352-c371303875de None None] MFA Rules not processed for user `fd836cf590a145df9acf19cf3ae2ac56`. Rule list: `[]` (Enabled: `True`). {{(pid=88114) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 10:06:25.826266 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.fernet_utils [None req-c9445386-bcb7-46a1-b352-c371303875de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88114) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:06:25.834747 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 1454/2904] 10.222.0.13 () {58 vars in 958 bytes} [Wed Aug 31 10:06:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 10:06:25.839269 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc0bf6ae-3b44-4837-a654-5f460933c875 None None] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:06:25.849645 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc0bf6ae-3b44-4837-a654-5f460933c875 None None] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:06:25.854327 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a70dd53-e12d-441d-af34-77be8cc40515 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:06:25.854933 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a70dd53-e12d-441d-af34-77be8cc40515 admin admin] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:06:25.855213 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a70dd53-e12d-441d-af34-77be8cc40515 admin admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:06:25.855485 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a70dd53-e12d-441d-af34-77be8cc40515 admin admin] PATH_INFO: `/v3/domains` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:06:25.856135 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a70dd53-e12d-441d-af34-77be8cc40515 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88113) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:06:25.856437 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a70dd53-e12d-441d-af34-77be8cc40515 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:06:25.858344 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a70dd53-e12d-441d-af34-77be8cc40515 admin admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:06:25.870420 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 1451/2905] 10.222.0.13 () {58 vars in 1153 bytes} [Wed Aug 31 10:06:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:06:25.875498 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9445386-bcb7-46a1-b352-c371303875de None None] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:06:25.885637 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9445386-bcb7-46a1-b352-c371303875de None None] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:06:25.891027 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63712e7e-dbff-4132-8216-6b28202c51bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:06:25.891515 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63712e7e-dbff-4132-8216-6b28202c51bc admin admin] REQUEST_METHOD: `POST` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:06:25.891793 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63712e7e-dbff-4132-8216-6b28202c51bc admin admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:06:25.892065 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63712e7e-dbff-4132-8216-6b28202c51bc admin admin] PATH_INFO: `/v3/projects` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:06:25.892706 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63712e7e-dbff-4132-8216-6b28202c51bc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:06:25.894634 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63712e7e-dbff-4132-8216-6b28202c51bc admin admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:06:25.922173 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 1455/2906] 10.222.0.13 () {60 vars in 1153 bytes} [Wed Aug 31 10:06:25 2022] POST /identity/v3/projects => generated 383 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 10:06:25.926499 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a70dd53-e12d-441d-af34-77be8cc40515 admin admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:06:25.938593 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a70dd53-e12d-441d-af34-77be8cc40515 admin admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:06:25.995975 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8128067d-795c-4576-a2fa-d424a2159700 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:06:25.996506 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8128067d-795c-4576-a2fa-d424a2159700 admin admin] REQUEST_METHOD: `POST` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:06:25.996783 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8128067d-795c-4576-a2fa-d424a2159700 admin admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:06:25.997046 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8128067d-795c-4576-a2fa-d424a2159700 admin admin] PATH_INFO: `/v3/users` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:06:25.997720 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8128067d-795c-4576-a2fa-d424a2159700 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:06:26.000167 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8128067d-795c-4576-a2fa-d424a2159700 admin admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:06:26.021089 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 1452/2907] 10.222.0.13 () {60 vars in 1144 bytes} [Wed Aug 31 10:06:25 2022] POST /identity/v3/users => generated 292 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 10:06:26.025272 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63712e7e-dbff-4132-8216-6b28202c51bc admin admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:06:26.035812 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63712e7e-dbff-4132-8216-6b28202c51bc admin admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:06:26.040360 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a15f891d-c2a9-44a2-b1db-f7093de61069 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:06:26.040947 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15f891d-c2a9-44a2-b1db-f7093de61069 admin admin] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:06:26.041224 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15f891d-c2a9-44a2-b1db-f7093de61069 admin admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:06:26.041527 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15f891d-c2a9-44a2-b1db-f7093de61069 admin admin] PATH_INFO: `/v3/roles` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:06:26.042166 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a15f891d-c2a9-44a2-b1db-f7093de61069 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:06:26.043739 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a15f891d-c2a9-44a2-b1db-f7093de61069 admin admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:06:26.054914 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 1456/2908] 10.222.0.13 () {58 vars in 1122 bytes} [Wed Aug 31 10:06:26 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 10:06:26.058965 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8128067d-795c-4576-a2fa-d424a2159700 admin admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:06:26.069924 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8128067d-795c-4576-a2fa-d424a2159700 admin admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:06:26.074568 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e81d5c2-092c-4b25-9849-fcae39110860 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:06:26.075161 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e81d5c2-092c-4b25-9849-fcae39110860 admin admin] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:06:26.075440 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e81d5c2-092c-4b25-9849-fcae39110860 admin admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:06:26.075707 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e81d5c2-092c-4b25-9849-fcae39110860 admin admin] PATH_INFO: `/v3/roles` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:06:26.077176 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e81d5c2-092c-4b25-9849-fcae39110860 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:06:26.078881 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e81d5c2-092c-4b25-9849-fcae39110860 admin admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:06:26.089914 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 1453/2909] 10.222.0.13 () {58 vars in 1122 bytes} [Wed Aug 31 10:06:26 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 10:06:26.095091 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a15f891d-c2a9-44a2-b1db-f7093de61069 admin admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:06:26.105256 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a15f891d-c2a9-44a2-b1db-f7093de61069 admin admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:06:26.111656 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e5cd81e-284c-4f9e-997b-3f766b8e63d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:06:26.112321 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5cd81e-284c-4f9e-997b-3f766b8e63d6 admin admin] REQUEST_METHOD: `PUT` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:06:26.112597 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5cd81e-284c-4f9e-997b-3f766b8e63d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:06:26.112865 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5cd81e-284c-4f9e-997b-3f766b8e63d6 admin admin] PATH_INFO: `/v3/projects/12a138e95d884fdaa8bb8653b151a546/users/2e8d67de6305464a982b7ad155a1ecca/roles/6c99f5d9f1f243eaa134d8540f21095d` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:06:26.132050 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e5cd81e-284c-4f9e-997b-3f766b8e63d6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=12a138e95d884fdaa8bb8653b151a546, user_id=2e8d67de6305464a982b7ad155a1ecca, role_id=6c99f5d9f1f243eaa134d8540f21095d)` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:06:26.134714 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e5cd81e-284c-4f9e-997b-3f766b8e63d6 admin admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:06:26.148145 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 1457/2910] 10.222.0.13 () {60 vars in 1483 bytes} [Wed Aug 31 10:06:26 2022] PUT /identity/v3/projects/12a138e95d884fdaa8bb8653b151a546/users/2e8d67de6305464a982b7ad155a1ecca/roles/6c99f5d9f1f243eaa134d8540f21095d => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 10:06:26.153250 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a88923d-5d2b-4bbd-9db4-5302b2a89267 None None] REQUEST_METHOD: `POST` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:06:26.153562 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a88923d-5d2b-4bbd-9db4-5302b2a89267 None None] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:06:26.153835 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a88923d-5d2b-4bbd-9db4-5302b2a89267 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:06:26.173915 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.auth.core [None req-4a88923d-5d2b-4bbd-9db4-5302b2a89267 None None] MFA Rules not processed for user `fd836cf590a145df9acf19cf3ae2ac56`. Rule list: `[]` (Enabled: `True`). {{(pid=88113) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 10:06:26.175116 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.fernet_utils [None req-4a88923d-5d2b-4bbd-9db4-5302b2a89267 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:06:26.216720 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:06:26.237686 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 1454/2911] 10.222.0.13 () {58 vars in 958 bytes} [Wed Aug 31 10:06:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 10:06:26.247370 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e5cd81e-284c-4f9e-997b-3f766b8e63d6 admin admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:06:26.249249 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.fernet_utils [None req-9e5cd81e-284c-4f9e-997b-3f766b8e63d6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88114) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:06:26.263724 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:06:26.317616 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e5cd81e-284c-4f9e-997b-3f766b8e63d6 admin admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:06:26.321267 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-334fdb03-e5be-4ebf-a2f3-7a07d8d786d6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9bdaacd597442c2bcde2c555f2cc140', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '49548eb44d62415582af820ab66decee', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:06:26.321977 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334fdb03-e5be-4ebf-a2f3-7a07d8d786d6 service neutron] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:06:26.322253 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334fdb03-e5be-4ebf-a2f3-7a07d8d786d6 service neutron] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:06:26.322522 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334fdb03-e5be-4ebf-a2f3-7a07d8d786d6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:06:26.332316 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-334fdb03-e5be-4ebf-a2f3-7a07d8d786d6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:06:26.333775 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-334fdb03-e5be-4ebf-a2f3-7a07d8d786d6 service neutron] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:06:26.343795 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 1458/2912] 10.222.0.13 () {60 vars in 1365 bytes} [Wed Aug 31 10:06:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 10:06:30.439629 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5195ea-f175-47ba-89d6-3b858d5bfd03 None None] REQUEST_METHOD: `POST` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:06:30.440183 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5195ea-f175-47ba-89d6-3b858d5bfd03 None None] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:06:30.440183 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5195ea-f175-47ba-89d6-3b858d5bfd03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:06:30.471623 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.auth.core [None req-5e5195ea-f175-47ba-89d6-3b858d5bfd03 None None] MFA Rules not processed for user `2e8d67de6305464a982b7ad155a1ecca`. Rule list: `[]` (Enabled: `True`). {{(pid=88113) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 10:06:30.472861 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.fernet_utils [None req-5e5195ea-f175-47ba-89d6-3b858d5bfd03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:06:30.554857 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 1455/2913] 10.222.0.13 () {58 vars in 958 bytes} [Wed Aug 31 10:06:30 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 10:06:30.559104 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-334fdb03-e5be-4ebf-a2f3-7a07d8d786d6 service neutron] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:06:30.572787 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-334fdb03-e5be-4ebf-a2f3-7a07d8d786d6 service neutron] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:06:30.579643 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4baaea70-243b-4d6a-bf20-a8ff79f140b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:06:30.580333 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4baaea70-243b-4d6a-bf20-a8ff79f140b5 admin admin] REQUEST_METHOD: `POST` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:06:30.581187 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4baaea70-243b-4d6a-bf20-a8ff79f140b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:06:30.581187 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4baaea70-243b-4d6a-bf20-a8ff79f140b5 admin admin] PATH_INFO: `/v3/projects` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:06:30.581548 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4baaea70-243b-4d6a-bf20-a8ff79f140b5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:06:30.583649 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4baaea70-243b-4d6a-bf20-a8ff79f140b5 admin admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:06:30.614511 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 1459/2914] 10.222.0.13 () {60 vars in 1153 bytes} [Wed Aug 31 10:06:30 2022] POST /identity/v3/projects => generated 385 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 10:06:30.618670 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e5195ea-f175-47ba-89d6-3b858d5bfd03 None None] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:06:30.630804 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e5195ea-f175-47ba-89d6-3b858d5bfd03 None None] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:06:30.694856 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d65a485-6078-4ab2-8799-c5e31cc22114 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:06:30.695473 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d65a485-6078-4ab2-8799-c5e31cc22114 admin admin] REQUEST_METHOD: `POST` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:06:30.695751 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d65a485-6078-4ab2-8799-c5e31cc22114 admin admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:06:30.696020 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d65a485-6078-4ab2-8799-c5e31cc22114 admin admin] PATH_INFO: `/v3/users` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:06:30.696896 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d65a485-6078-4ab2-8799-c5e31cc22114 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:06:30.699268 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d65a485-6078-4ab2-8799-c5e31cc22114 admin admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:06:30.721010 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 1456/2915] 10.222.0.13 () {60 vars in 1144 bytes} [Wed Aug 31 10:06:30 2022] POST /identity/v3/users => generated 299 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 10:06:30.725171 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4baaea70-243b-4d6a-bf20-a8ff79f140b5 admin admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:06:30.737527 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4baaea70-243b-4d6a-bf20-a8ff79f140b5 admin admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:06:30.742244 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe2cedf-6d56-4146-93ec-ed7d49ad0724 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:06:30.742864 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe2cedf-6d56-4146-93ec-ed7d49ad0724 admin admin] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:06:30.743141 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe2cedf-6d56-4146-93ec-ed7d49ad0724 admin admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:06:30.743412 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe2cedf-6d56-4146-93ec-ed7d49ad0724 admin admin] PATH_INFO: `/v3/roles` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:06:30.744064 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fe2cedf-6d56-4146-93ec-ed7d49ad0724 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:06:30.745694 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fe2cedf-6d56-4146-93ec-ed7d49ad0724 admin admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:06:30.758035 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 1460/2916] 10.222.0.13 () {58 vars in 1122 bytes} [Wed Aug 31 10:06:30 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 10:06:30.762517 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d65a485-6078-4ab2-8799-c5e31cc22114 admin admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:06:30.773524 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d65a485-6078-4ab2-8799-c5e31cc22114 admin admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:06:30.778246 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0933b3cc-0c93-4b10-954e-d1e506192554 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:06:30.778918 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0933b3cc-0c93-4b10-954e-d1e506192554 admin admin] REQUEST_METHOD: `PUT` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:06:30.779195 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0933b3cc-0c93-4b10-954e-d1e506192554 admin admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:06:30.779494 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0933b3cc-0c93-4b10-954e-d1e506192554 admin admin] PATH_INFO: `/v3/projects/ca314df23321471bab248ce29b9d1427/users/66d1d908cf3c4a46a3350f60be3bc555/roles/5d4ee63a3add47fc9a6b8b16d39e0a1f` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:06:30.795446 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0933b3cc-0c93-4b10-954e-d1e506192554 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ca314df23321471bab248ce29b9d1427, user_id=66d1d908cf3c4a46a3350f60be3bc555, role_id=5d4ee63a3add47fc9a6b8b16d39e0a1f)` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:06:30.798207 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0933b3cc-0c93-4b10-954e-d1e506192554 admin admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:06:30.814333 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 1457/2917] 10.222.0.13 () {60 vars in 1483 bytes} [Wed Aug 31 10:06:30 2022] PUT /identity/v3/projects/ca314df23321471bab248ce29b9d1427/users/66d1d908cf3c4a46a3350f60be3bc555/roles/5d4ee63a3add47fc9a6b8b16d39e0a1f => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 10:06:30.818611 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe2cedf-6d56-4146-93ec-ed7d49ad0724 admin admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:06:30.829479 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe2cedf-6d56-4146-93ec-ed7d49ad0724 admin admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:06:30.891106 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-027e866b-6446-472e-9565-966ce704ca1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:06:30.891706 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027e866b-6446-472e-9565-966ce704ca1a admin admin] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:06:30.891982 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027e866b-6446-472e-9565-966ce704ca1a admin admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:06:30.892252 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027e866b-6446-472e-9565-966ce704ca1a admin admin] PATH_INFO: `/v3/roles` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:06:30.892897 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-027e866b-6446-472e-9565-966ce704ca1a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:06:30.894558 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-027e866b-6446-472e-9565-966ce704ca1a admin admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:06:30.904986 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 1461/2918] 10.222.0.13 () {58 vars in 1122 bytes} [Wed Aug 31 10:06:30 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 10:06:30.909376 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0933b3cc-0c93-4b10-954e-d1e506192554 admin admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:06:30.919934 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0933b3cc-0c93-4b10-954e-d1e506192554 admin admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:06:30.924607 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da3d1b4-0e78-4e40-868d-85bf03248ab5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:06:30.925249 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da3d1b4-0e78-4e40-868d-85bf03248ab5 admin admin] REQUEST_METHOD: `PUT` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:06:30.925557 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da3d1b4-0e78-4e40-868d-85bf03248ab5 admin admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:06:30.925827 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da3d1b4-0e78-4e40-868d-85bf03248ab5 admin admin] PATH_INFO: `/v3/projects/ca314df23321471bab248ce29b9d1427/users/66d1d908cf3c4a46a3350f60be3bc555/roles/5d4ee63a3add47fc9a6b8b16d39e0a1f` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:06:30.927750 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4da3d1b4-0e78-4e40-868d-85bf03248ab5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ca314df23321471bab248ce29b9d1427, user_id=66d1d908cf3c4a46a3350f60be3bc555, role_id=5d4ee63a3add47fc9a6b8b16d39e0a1f)` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:06:30.930441 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4da3d1b4-0e78-4e40-868d-85bf03248ab5 admin admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:06:30.941104 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 1458/2919] 10.222.0.13 () {60 vars in 1483 bytes} [Wed Aug 31 10:06:30 2022] PUT /identity/v3/projects/ca314df23321471bab248ce29b9d1427/users/66d1d908cf3c4a46a3350f60be3bc555/roles/5d4ee63a3add47fc9a6b8b16d39e0a1f => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 10:06:35.350432 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b3bcd2-3919-495b-85a1-20b64294cfc2 None None] REQUEST_METHOD: `POST` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:06:35.351032 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b3bcd2-3919-495b-85a1-20b64294cfc2 None None] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:06:35.351032 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b3bcd2-3919-495b-85a1-20b64294cfc2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:06:35.385629 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.auth.core [None req-f1b3bcd2-3919-495b-85a1-20b64294cfc2 None None] MFA Rules not processed for user `66d1d908cf3c4a46a3350f60be3bc555`. Rule list: `[]` (Enabled: `True`). {{(pid=88114) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 10:06:35.387110 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.fernet_utils [None req-f1b3bcd2-3919-495b-85a1-20b64294cfc2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88114) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:06:35.514361 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 1462/2920] 10.222.0.13 () {58 vars in 958 bytes} [Wed Aug 31 10:06:35 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 10:06:35.523820 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da3d1b4-0e78-4e40-868d-85bf03248ab5 admin admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:06:35.525880 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.fernet_utils [None req-4da3d1b4-0e78-4e40-868d-85bf03248ab5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:06:35.578109 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:06:35.635535 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da3d1b4-0e78-4e40-868d-85bf03248ab5 admin admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:06:35.640450 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c3e2fd-6507-4ada-9f60-4c85654891b4 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be216d156b54452cbbf978c7734343aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '49548eb44d62415582af820ab66decee', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:06:35.641035 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c3e2fd-6507-4ada-9f60-4c85654891b4 service glance] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:06:35.641311 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c3e2fd-6507-4ada-9f60-4c85654891b4 service glance] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:06:35.641620 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c3e2fd-6507-4ada-9f60-4c85654891b4 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:06:35.709712 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22c3e2fd-6507-4ada-9f60-4c85654891b4 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:06:35.711303 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22c3e2fd-6507-4ada-9f60-4c85654891b4 service glance] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:06:35.721786 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 1459/2921] 10.222.0.13 () {60 vars in 1365 bytes} [Wed Aug 31 10:06:35 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 199 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 10:06:35.855671 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0454f7d8-e7ec-436d-8581-0a9db6d4d291 None None] REQUEST_METHOD: `POST` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:06:35.856193 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0454f7d8-e7ec-436d-8581-0a9db6d4d291 None None] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:06:35.856469 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0454f7d8-e7ec-436d-8581-0a9db6d4d291 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:06:35.864140 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:06:35.892155 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.auth.core [None req-0454f7d8-e7ec-436d-8581-0a9db6d4d291 None None] MFA Rules not processed for user `768cb67c1ef24251aaa113c12d81a250`. Rule list: `[]` (Enabled: `True`). {{(pid=88114) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 10:06:35.893273 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.fernet_utils [None req-0454f7d8-e7ec-436d-8581-0a9db6d4d291 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88114) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:06:35.960400 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 1463/2922] 10.222.0.13 () {58 vars in 1009 bytes} [Wed Aug 31 10:06:35 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 10:06:36.011275 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c3e2fd-6507-4ada-9f60-4c85654891b4 service glance] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:06:36.012733 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.fernet_utils [None req-22c3e2fd-6507-4ada-9f60-4c85654891b4 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:06:36.093688 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c3e2fd-6507-4ada-9f60-4c85654891b4 service glance] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:06:36.098809 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baeefef4-965a-4278-b49c-f53a5585588e service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '202bb9db7a8a4977ba788a290ed9a3fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '49548eb44d62415582af820ab66decee', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:06:36.099415 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baeefef4-965a-4278-b49c-f53a5585588e service swift] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:06:36.099690 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baeefef4-965a-4278-b49c-f53a5585588e service swift] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:06:36.099958 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baeefef4-965a-4278-b49c-f53a5585588e service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:06:36.108640 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baeefef4-965a-4278-b49c-f53a5585588e service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:06:36.110546 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baeefef4-965a-4278-b49c-f53a5585588e service swift] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:06:36.118646 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 1460/2923] 10.222.0.13 () {60 vars in 1384 bytes} [Wed Aug 31 10:06:36 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 31 10:06:44.804165 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47579b8d-34bd-489e-b371-9d5e5f29aa3d None None] REQUEST_METHOD: `POST` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:06:44.804692 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47579b8d-34bd-489e-b371-9d5e5f29aa3d None None] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:06:44.804802 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47579b8d-34bd-489e-b371-9d5e5f29aa3d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:06:44.824006 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.auth.core [None req-47579b8d-34bd-489e-b371-9d5e5f29aa3d None None] MFA Rules not processed for user `2e8d67de6305464a982b7ad155a1ecca`. Rule list: `[]` (Enabled: `True`). {{(pid=88114) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 10:06:44.825191 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.fernet_utils [None req-47579b8d-34bd-489e-b371-9d5e5f29aa3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88114) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:06:44.832612 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 1464/2924] 10.222.0.13 () {58 vars in 958 bytes} [Wed Aug 31 10:06:44 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 10:06:45.105703 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baeefef4-965a-4278-b49c-f53a5585588e service swift] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:06:45.107956 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.fernet_utils [None req-baeefef4-965a-4278-b49c-f53a5585588e service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:06:45.167190 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baeefef4-965a-4278-b49c-f53a5585588e service swift] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:06:45.171536 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac10371-dfe7-4360-8da4-4d45d9125e9c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9bdaacd597442c2bcde2c555f2cc140', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '49548eb44d62415582af820ab66decee', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:06:45.172142 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac10371-dfe7-4360-8da4-4d45d9125e9c service neutron] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:06:45.172418 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac10371-dfe7-4360-8da4-4d45d9125e9c service neutron] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:06:45.172686 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac10371-dfe7-4360-8da4-4d45d9125e9c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:06:45.174434 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.fernet_utils [None req-bac10371-dfe7-4360-8da4-4d45d9125e9c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:06:45.180612 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=100 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:06:45.185856 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bac10371-dfe7-4360-8da4-4d45d9125e9c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:06:45.187158 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bac10371-dfe7-4360-8da4-4d45d9125e9c service neutron] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:06:45.195305 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 1461/2925] 10.222.0.13 () {60 vars in 1365 bytes} [Wed Aug 31 10:06:45 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 10:06:45.515135 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47579b8d-34bd-489e-b371-9d5e5f29aa3d None None] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:06:45.516800 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.fernet_utils [None req-47579b8d-34bd-489e-b371-9d5e5f29aa3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88114) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:06:45.613981 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47579b8d-34bd-489e-b371-9d5e5f29aa3d None None] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:06:45.619001 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95be8ea7-386c-45d1-a5d5-8124e46550d0 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ff3aff5ef444910be6dbb989d0e3194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '49548eb44d62415582af820ab66decee', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:06:45.619601 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95be8ea7-386c-45d1-a5d5-8124e46550d0 service placement] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:06:45.619876 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95be8ea7-386c-45d1-a5d5-8124e46550d0 service placement] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:06:45.620144 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95be8ea7-386c-45d1-a5d5-8124e46550d0 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:06:45.621888 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.fernet_utils [None req-95be8ea7-386c-45d1-a5d5-8124e46550d0 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88114) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:06:45.633230 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95be8ea7-386c-45d1-a5d5-8124e46550d0 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:06:45.634818 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95be8ea7-386c-45d1-a5d5-8124e46550d0 service placement] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:06:45.644879 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 1465/2926] 10.222.0.13 () {60 vars in 1365 bytes} [Wed Aug 31 10:06:45 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 10:06:47.012578 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1423d4-87c2-4015-b2b9-81f388236218 None None] REQUEST_METHOD: `POST` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:06:47.013089 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1423d4-87c2-4015-b2b9-81f388236218 None None] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:06:47.013355 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1423d4-87c2-4015-b2b9-81f388236218 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:06:47.033833 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.auth.core [None req-9d1423d4-87c2-4015-b2b9-81f388236218 None None] MFA Rules not processed for user `768cb67c1ef24251aaa113c12d81a250`. Rule list: `[]` (Enabled: `True`). {{(pid=88113) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 10:06:47.035168 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.fernet_utils [None req-9d1423d4-87c2-4015-b2b9-81f388236218 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:06:47.042450 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 1462/2927] 10.222.0.13 () {58 vars in 1009 bytes} [Wed Aug 31 10:06:47 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 10:06:47.055332 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95be8ea7-386c-45d1-a5d5-8124e46550d0 service placement] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:06:47.068896 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95be8ea7-386c-45d1-a5d5-8124e46550d0 service placement] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:06:47.074616 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea380ceb-c43f-4e14-98c3-3e9a67e3bbb9 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '202bb9db7a8a4977ba788a290ed9a3fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '49548eb44d62415582af820ab66decee', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:06:47.075227 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea380ceb-c43f-4e14-98c3-3e9a67e3bbb9 service swift] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:06:47.075502 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea380ceb-c43f-4e14-98c3-3e9a67e3bbb9 service swift] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:06:47.075774 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea380ceb-c43f-4e14-98c3-3e9a67e3bbb9 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:06:47.085202 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea380ceb-c43f-4e14-98c3-3e9a67e3bbb9 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:06:47.086826 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea380ceb-c43f-4e14-98c3-3e9a67e3bbb9 service swift] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:06:47.095897 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 1466/2928] 10.222.0.13 () {60 vars in 1384 bytes} [Wed Aug 31 10:06:47 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 31 10:06:57.028041 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=6 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:06:57.091947 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=80 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:06:57.390777 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d1423d4-87c2-4015-b2b9-81f388236218 None None] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:06:57.400333 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d1423d4-87c2-4015-b2b9-81f388236218 None None] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:06:57.403967 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d5bf552-a549-4ee8-a6b7-1d2087f20b43 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9bdaacd597442c2bcde2c555f2cc140', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '49548eb44d62415582af820ab66decee', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:06:57.404569 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5bf552-a549-4ee8-a6b7-1d2087f20b43 service neutron] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:06:57.405463 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5bf552-a549-4ee8-a6b7-1d2087f20b43 service neutron] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:06:57.405903 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5bf552-a549-4ee8-a6b7-1d2087f20b43 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:06:57.407851 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.fernet_utils [None req-9d5bf552-a549-4ee8-a6b7-1d2087f20b43 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:06:57.422433 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d5bf552-a549-4ee8-a6b7-1d2087f20b43 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:06:57.423885 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d5bf552-a549-4ee8-a6b7-1d2087f20b43 service neutron] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:06:57.432624 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 1463/2929] 10.222.0.13 () {60 vars in 1365 bytes} [Wed Aug 31 10:06:57 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 10:06:58.004296 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea380ceb-c43f-4e14-98c3-3e9a67e3bbb9 service swift] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:06:58.005821 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.fernet_utils [None req-ea380ceb-c43f-4e14-98c3-3e9a67e3bbb9 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88114) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:06:58.113305 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea380ceb-c43f-4e14-98c3-3e9a67e3bbb9 service swift] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:06:58.119145 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa19640-3ba9-4c8a-98eb-8f61a3872847 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc076493239d4ccdbdc37c0369c49529', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '49548eb44d62415582af820ab66decee', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:06:58.119759 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa19640-3ba9-4c8a-98eb-8f61a3872847 service nova] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:06:58.120036 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa19640-3ba9-4c8a-98eb-8f61a3872847 service nova] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:06:58.120306 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa19640-3ba9-4c8a-98eb-8f61a3872847 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:06:58.122056 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.fernet_utils [None req-dfa19640-3ba9-4c8a-98eb-8f61a3872847 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88114) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:06:58.135095 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfa19640-3ba9-4c8a-98eb-8f61a3872847 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:06:58.136671 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfa19640-3ba9-4c8a-98eb-8f61a3872847 service nova] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:06:58.148432 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 1467/2930] 10.222.0.13 () {60 vars in 1365 bytes} [Wed Aug 31 10:06:58 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 10:07:04.788336 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d5bf552-a549-4ee8-a6b7-1d2087f20b43 service neutron] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:07:04.790084 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.fernet_utils [None req-9d5bf552-a549-4ee8-a6b7-1d2087f20b43 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:07:04.808559 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d5bf552-a549-4ee8-a6b7-1d2087f20b43 service neutron] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:07:04.815211 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7481668c-b511-4bf9-b144-6d3b28af276c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc076493239d4ccdbdc37c0369c49529', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '49548eb44d62415582af820ab66decee', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:07:04.815826 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7481668c-b511-4bf9-b144-6d3b28af276c service nova] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:07:04.816108 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7481668c-b511-4bf9-b144-6d3b28af276c service nova] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:07:04.816499 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7481668c-b511-4bf9-b144-6d3b28af276c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:07:04.818294 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.fernet_utils [None req-7481668c-b511-4bf9-b144-6d3b28af276c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:07:04.832022 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7481668c-b511-4bf9-b144-6d3b28af276c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:07:04.833822 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7481668c-b511-4bf9-b144-6d3b28af276c service nova] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:07:04.846854 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 1464/2931] 10.222.0.13 () {60 vars in 1365 bytes} [Wed Aug 31 10:07:04 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 58 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 10:07:08.143793 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=43 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:07:08.901928 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa19640-3ba9-4c8a-98eb-8f61a3872847 service nova] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:07:08.913683 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa19640-3ba9-4c8a-98eb-8f61a3872847 service nova] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:07:08.918713 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbca8a81-aba9-4dc0-bb46-d5fb0dc1ff26 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ff3aff5ef444910be6dbb989d0e3194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '49548eb44d62415582af820ab66decee', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:07:08.919303 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbca8a81-aba9-4dc0-bb46-d5fb0dc1ff26 service placement] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:07:08.919582 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbca8a81-aba9-4dc0-bb46-d5fb0dc1ff26 service placement] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:07:08.919853 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbca8a81-aba9-4dc0-bb46-d5fb0dc1ff26 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:07:08.922240 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.fernet_utils [None req-dbca8a81-aba9-4dc0-bb46-d5fb0dc1ff26 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88114) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:07:08.933878 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbca8a81-aba9-4dc0-bb46-d5fb0dc1ff26 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:07:08.935451 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbca8a81-aba9-4dc0-bb46-d5fb0dc1ff26 service placement] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:07:08.945583 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 1468/2932] 10.222.0.13 () {60 vars in 1365 bytes} [Wed Aug 31 10:07:08 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 10:07:14.841113 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=12 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:07:17.423550 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7481668c-b511-4bf9-b144-6d3b28af276c service nova] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:07:17.434961 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7481668c-b511-4bf9-b144-6d3b28af276c service nova] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:07:17.439203 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84881da5-173d-4562-8f45-276d7cd43627 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9bdaacd597442c2bcde2c555f2cc140', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '49548eb44d62415582af820ab66decee', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:07:17.440155 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84881da5-173d-4562-8f45-276d7cd43627 service neutron] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:07:17.440464 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84881da5-173d-4562-8f45-276d7cd43627 service neutron] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:07:17.440743 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84881da5-173d-4562-8f45-276d7cd43627 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:07:17.450846 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84881da5-173d-4562-8f45-276d7cd43627 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:07:17.452792 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84881da5-173d-4562-8f45-276d7cd43627 service neutron] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:07:17.462348 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 1465/2933] 10.222.0.13 () {60 vars in 1365 bytes} [Wed Aug 31 10:07:17 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 10:07:18.941242 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=6 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:07:19.288373 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbca8a81-aba9-4dc0-bb46-d5fb0dc1ff26 service placement] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:07:19.305158 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbca8a81-aba9-4dc0-bb46-d5fb0dc1ff26 service placement] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:07:19.311010 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af4048cb-06b4-4367-83d1-a5fb2a750109 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc076493239d4ccdbdc37c0369c49529', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '49548eb44d62415582af820ab66decee', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:07:19.312299 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4048cb-06b4-4367-83d1-a5fb2a750109 service nova] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:07:19.312586 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4048cb-06b4-4367-83d1-a5fb2a750109 service nova] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:07:19.313006 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4048cb-06b4-4367-83d1-a5fb2a750109 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:07:19.314843 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.fernet_utils [None req-af4048cb-06b4-4367-83d1-a5fb2a750109 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88114) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:07:19.329002 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af4048cb-06b4-4367-83d1-a5fb2a750109 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:07:19.330886 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af4048cb-06b4-4367-83d1-a5fb2a750109 service nova] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:07:19.343064 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 1469/2934] 10.222.0.13 () {60 vars in 1365 bytes} [Wed Aug 31 10:07:19 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 10:07:27.458159 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=6 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:07:29.337667 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=6 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:08:21.774568 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84881da5-173d-4562-8f45-276d7cd43627 service neutron] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:08:21.784022 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84881da5-173d-4562-8f45-276d7cd43627 service neutron] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:08:21.787723 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f9b4609-0fc3-413e-97a3-c1020338aa6c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9bdaacd597442c2bcde2c555f2cc140', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '49548eb44d62415582af820ab66decee', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:08:21.788315 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9b4609-0fc3-413e-97a3-c1020338aa6c service neutron] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:08:21.788596 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9b4609-0fc3-413e-97a3-c1020338aa6c service neutron] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:08:21.788867 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9b4609-0fc3-413e-97a3-c1020338aa6c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:08:21.790620 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.fernet_utils [None req-3f9b4609-0fc3-413e-97a3-c1020338aa6c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:08:21.806084 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f9b4609-0fc3-413e-97a3-c1020338aa6c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:08:21.807875 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f9b4609-0fc3-413e-97a3-c1020338aa6c service neutron] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:08:21.823913 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 1466/2935] 10.222.0.13 () {60 vars in 1365 bytes} [Wed Aug 31 10:08:21 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 10:08:23.110313 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af4048cb-06b4-4367-83d1-a5fb2a750109 service nova] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:08:23.120318 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af4048cb-06b4-4367-83d1-a5fb2a750109 service nova] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:08:23.124539 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-102326fa-db2c-4214-8084-7f7f522c7d61 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9bdaacd597442c2bcde2c555f2cc140', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '49548eb44d62415582af820ab66decee', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:08:23.125131 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102326fa-db2c-4214-8084-7f7f522c7d61 service neutron] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:08:23.125423 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102326fa-db2c-4214-8084-7f7f522c7d61 service neutron] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:08:23.125695 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102326fa-db2c-4214-8084-7f7f522c7d61 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:08:23.127394 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.fernet_utils [None req-102326fa-db2c-4214-8084-7f7f522c7d61 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88114) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:08:23.140696 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-102326fa-db2c-4214-8084-7f7f522c7d61 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:08:23.142139 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-102326fa-db2c-4214-8084-7f7f522c7d61 service neutron] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:08:23.152806 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 1470/2936] 10.222.0.13 () {60 vars in 1365 bytes} [Wed Aug 31 10:08:23 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 10:08:31.815730 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=6 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:08:33.148017 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=6 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:09:45.625460 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34dd097-64ab-42c2-ab68-f2b81b413d20 None None] REQUEST_METHOD: `POST` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:09:45.625982 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34dd097-64ab-42c2-ab68-f2b81b413d20 None None] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:09:45.625982 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34dd097-64ab-42c2-ab68-f2b81b413d20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:09:45.676355 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.auth.core [None req-e34dd097-64ab-42c2-ab68-f2b81b413d20 None None] MFA Rules not processed for user `768cb67c1ef24251aaa113c12d81a250`. Rule list: `[]` (Enabled: `True`). {{(pid=88113) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 10:09:45.677607 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.fernet_utils [None req-e34dd097-64ab-42c2-ab68-f2b81b413d20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:09:45.694595 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 1467/2937] 10.222.0.13 () {58 vars in 967 bytes} [Wed Aug 31 10:09:45 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 70 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 10:09:45.737087 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-102326fa-db2c-4214-8084-7f7f522c7d61 service neutron] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:09:45.758576 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-102326fa-db2c-4214-8084-7f7f522c7d61 service neutron] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:09:45.763767 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a97e06c-b52b-4e7d-aded-b2bb9ec2e108 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '202bb9db7a8a4977ba788a290ed9a3fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '49548eb44d62415582af820ab66decee', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:09:45.764358 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a97e06c-b52b-4e7d-aded-b2bb9ec2e108 service swift] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:09:45.764631 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a97e06c-b52b-4e7d-aded-b2bb9ec2e108 service swift] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:09:45.764897 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a97e06c-b52b-4e7d-aded-b2bb9ec2e108 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:09:45.773135 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a97e06c-b52b-4e7d-aded-b2bb9ec2e108 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:09:45.775319 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a97e06c-b52b-4e7d-aded-b2bb9ec2e108 service swift] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:09:45.783333 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 1471/2938] 10.222.0.13 () {60 vars in 1384 bytes} [Wed Aug 31 10:09:45 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 31 10:09:52.425304 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e34dd097-64ab-42c2-ab68-f2b81b413d20 None None] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:09:52.439529 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e34dd097-64ab-42c2-ab68-f2b81b413d20 None None] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:09:52.444968 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-393d84fd-9cb5-40a5-8ef9-cd192f681a96 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ff3aff5ef444910be6dbb989d0e3194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '49548eb44d62415582af820ab66decee', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:09:52.448412 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393d84fd-9cb5-40a5-8ef9-cd192f681a96 service placement] REQUEST_METHOD: `GET` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:09:52.448412 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393d84fd-9cb5-40a5-8ef9-cd192f681a96 service placement] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:09:52.448412 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393d84fd-9cb5-40a5-8ef9-cd192f681a96 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:09:52.451697 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.fernet_utils [None req-393d84fd-9cb5-40a5-8ef9-cd192f681a96 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:09:52.464736 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-393d84fd-9cb5-40a5-8ef9-cd192f681a96 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:09:52.466548 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-393d84fd-9cb5-40a5-8ef9-cd192f681a96 service placement] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:09:52.478920 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 1468/2939] 10.222.0.13 () {60 vars in 1365 bytes} [Wed Aug 31 10:09:52 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 10:09:52.484970 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a97e06c-b52b-4e7d-aded-b2bb9ec2e108 service swift] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:09:52.552464 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a97e06c-b52b-4e7d-aded-b2bb9ec2e108 service swift] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:09:52.556956 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-020c2436-8a81-4a4a-9ebc-7bbd1ad6597b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:09:52.557604 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020c2436-8a81-4a4a-9ebc-7bbd1ad6597b admin admin] REQUEST_METHOD: `DELETE` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:09:52.557883 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020c2436-8a81-4a4a-9ebc-7bbd1ad6597b admin admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:09:52.558155 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020c2436-8a81-4a4a-9ebc-7bbd1ad6597b admin admin] PATH_INFO: `/v3/users/2e8d67de6305464a982b7ad155a1ecca` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:09:52.559265 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-020c2436-8a81-4a4a-9ebc-7bbd1ad6597b admin admin] RBAC: Authorizing `identity:delete_user(user_id=2e8d67de6305464a982b7ad155a1ecca)` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:09:52.561167 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-020c2436-8a81-4a4a-9ebc-7bbd1ad6597b admin admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:09:52.607966 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.notifications [None req-020c2436-8a81-4a4a-9ebc-7bbd1ad6597b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2e8d67de6305464a982b7ad155a1ecca'} {{(pid=88114) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 10:09:52.614908 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.notifications [None req-020c2436-8a81-4a4a-9ebc-7bbd1ad6597b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2e8d67de6305464a982b7ad155a1ecca'} {{(pid=88114) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 10:09:52.621878 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.notifications [None req-020c2436-8a81-4a4a-9ebc-7bbd1ad6597b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2e8d67de6305464a982b7ad155a1ecca'} {{(pid=88114) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 10:09:52.622552 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.notifications [None req-020c2436-8a81-4a4a-9ebc-7bbd1ad6597b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2e8d67de6305464a982b7ad155a1ecca'} {{(pid=88114) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 10:09:52.635036 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.notifications [None req-020c2436-8a81-4a4a-9ebc-7bbd1ad6597b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2e8d67de6305464a982b7ad155a1ecca'} {{(pid=88114) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 10:09:52.640870 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 1472/2940] 10.222.0.13 () {58 vars in 1224 bytes} [Wed Aug 31 10:09:52 2022] DELETE /identity/v3/users/2e8d67de6305464a982b7ad155a1ecca => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 10:09:52.811204 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-393d84fd-9cb5-40a5-8ef9-cd192f681a96 service placement] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:09:52.822689 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-393d84fd-9cb5-40a5-8ef9-cd192f681a96 service placement] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:09:52.880954 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-828f4700-97a9-4107-8427-8fc70351ce33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:09:52.881620 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828f4700-97a9-4107-8427-8fc70351ce33 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:09:52.882067 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828f4700-97a9-4107-8427-8fc70351ce33 admin admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:09:52.882339 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828f4700-97a9-4107-8427-8fc70351ce33 admin admin] PATH_INFO: `/v3/projects/12a138e95d884fdaa8bb8653b151a546` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:09:52.883584 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-828f4700-97a9-4107-8427-8fc70351ce33 admin admin] RBAC: Authorizing `identity:delete_project(project_id=12a138e95d884fdaa8bb8653b151a546)` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:09:52.885510 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-828f4700-97a9-4107-8427-8fc70351ce33 admin admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:09:52.930720 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.notifications [None req-828f4700-97a9-4107-8427-8fc70351ce33 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '12a138e95d884fdaa8bb8653b151a546', 'request_id': 'req-828f4700-97a9-4107-8427-8fc70351ce33'} {{(pid=88113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 10:09:52.936217 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.notifications [None req-828f4700-97a9-4107-8427-8fc70351ce33 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '12a138e95d884fdaa8bb8653b151a546', 'request_id': 'req-828f4700-97a9-4107-8427-8fc70351ce33'} {{(pid=88113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 10:09:52.943439 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.notifications [None req-828f4700-97a9-4107-8427-8fc70351ce33 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '12a138e95d884fdaa8bb8653b151a546', 'request_id': 'req-828f4700-97a9-4107-8427-8fc70351ce33'} {{(pid=88113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 10:09:52.954794 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.notifications [None req-828f4700-97a9-4107-8427-8fc70351ce33 admin admin] The token cache is being invalidate because project 12a138e95d884fdaa8bb8653b151a546 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88113) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 10:09:52.955079 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.notifications [None req-828f4700-97a9-4107-8427-8fc70351ce33 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 10:09:52.955719 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.notifications [None req-828f4700-97a9-4107-8427-8fc70351ce33 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 10:09:52.957210 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 1469/2941] 10.222.0.13 () {58 vars in 1233 bytes} [Wed Aug 31 10:09:52 2022] DELETE /identity/v3/projects/12a138e95d884fdaa8bb8653b151a546 => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 10:09:52.961877 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-020c2436-8a81-4a4a-9ebc-7bbd1ad6597b admin admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:09:52.963317 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.fernet_utils [None req-020c2436-8a81-4a4a-9ebc-7bbd1ad6597b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88114) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:09:52.977141 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-020c2436-8a81-4a4a-9ebc-7bbd1ad6597b admin admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:09:53.027507 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:09:53.033687 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc098ad-2e33-4cce-9858-559558465f44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:09:53.034586 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc098ad-2e33-4cce-9858-559558465f44 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:09:53.034884 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc098ad-2e33-4cce-9858-559558465f44 admin admin] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:09:53.035339 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc098ad-2e33-4cce-9858-559558465f44 admin admin] PATH_INFO: `/v3/users/66d1d908cf3c4a46a3350f60be3bc555` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:09:53.036852 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dc098ad-2e33-4cce-9858-559558465f44 admin admin] RBAC: Authorizing `identity:delete_user(user_id=66d1d908cf3c4a46a3350f60be3bc555)` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:09:53.039665 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dc098ad-2e33-4cce-9858-559558465f44 admin admin] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:09:53.086553 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.notifications [None req-2dc098ad-2e33-4cce-9858-559558465f44 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '66d1d908cf3c4a46a3350f60be3bc555'} {{(pid=88114) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 10:09:53.093215 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.notifications [None req-2dc098ad-2e33-4cce-9858-559558465f44 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '66d1d908cf3c4a46a3350f60be3bc555'} {{(pid=88114) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 10:09:53.100493 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.notifications [None req-2dc098ad-2e33-4cce-9858-559558465f44 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '66d1d908cf3c4a46a3350f60be3bc555'} {{(pid=88114) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 10:09:53.101124 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.notifications [None req-2dc098ad-2e33-4cce-9858-559558465f44 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '66d1d908cf3c4a46a3350f60be3bc555'} {{(pid=88114) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 10:09:53.114524 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.notifications [None req-2dc098ad-2e33-4cce-9858-559558465f44 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '66d1d908cf3c4a46a3350f60be3bc555'} {{(pid=88114) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 10:09:53.120516 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 1473/2942] 10.222.0.13 () {58 vars in 1224 bytes} [Wed Aug 31 10:09:52 2022] DELETE /identity/v3/users/66d1d908cf3c4a46a3350f60be3bc555 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 10:09:53.301874 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-828f4700-97a9-4107-8427-8fc70351ce33 admin admin] Authenticating user token {{(pid=88113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:09:53.303344 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.fernet_utils [None req-828f4700-97a9-4107-8427-8fc70351ce33 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:09:53.316412 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-828f4700-97a9-4107-8427-8fc70351ce33 admin admin] Validating token access rules against request {{(pid=88113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:09:53.355629 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:09:53.382834 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d99af7-429e-497e-b16b-2246b5b03265 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd836cf590a145df9acf19cf3ae2ac56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f187a2ac46b4c229b4eeead01522f05', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:09:53.383602 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d99af7-429e-497e-b16b-2246b5b03265 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:09:53.383878 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d99af7-429e-497e-b16b-2246b5b03265 admin admin] SCRIPT_NAME: `/identity` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:09:53.384148 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d99af7-429e-497e-b16b-2246b5b03265 admin admin] PATH_INFO: `/v3/projects/ca314df23321471bab248ce29b9d1427` {{(pid=88113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:09:53.385251 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84d99af7-429e-497e-b16b-2246b5b03265 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ca314df23321471bab248ce29b9d1427)` {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:09:53.387179 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84d99af7-429e-497e-b16b-2246b5b03265 admin admin] RBAC: Authorization granted {{(pid=88113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:09:53.433864 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.notifications [None req-84d99af7-429e-497e-b16b-2246b5b03265 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ca314df23321471bab248ce29b9d1427', 'request_id': 'req-84d99af7-429e-497e-b16b-2246b5b03265'} {{(pid=88113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 10:09:53.439349 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.notifications [None req-84d99af7-429e-497e-b16b-2246b5b03265 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ca314df23321471bab248ce29b9d1427', 'request_id': 'req-84d99af7-429e-497e-b16b-2246b5b03265'} {{(pid=88113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 10:09:53.447150 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.notifications [None req-84d99af7-429e-497e-b16b-2246b5b03265 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ca314df23321471bab248ce29b9d1427', 'request_id': 'req-84d99af7-429e-497e-b16b-2246b5b03265'} {{(pid=88113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 10:09:53.458113 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.notifications [None req-84d99af7-429e-497e-b16b-2246b5b03265 admin admin] The token cache is being invalidate because project ca314df23321471bab248ce29b9d1427 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88113) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 10:09:53.458395 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.notifications [None req-84d99af7-429e-497e-b16b-2246b5b03265 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 10:09:53.459014 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG keystone.notifications [None req-84d99af7-429e-497e-b16b-2246b5b03265 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 10:09:53.460485 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: [pid: 88113|app: 0|req: 1470/2943] 10.222.0.13 () {58 vars in 1233 bytes} [Wed Aug 31 10:09:53 2022] DELETE /identity/v3/projects/ca314df23321471bab248ce29b9d1427 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 10:10:03.117573 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=27,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:10:03.455081 dv-cinder-844961-2-iscsi devstack@keystone.service[88113]: DEBUG dbcounter [-] [88113] Writing DB stats keystone:SELECT=34,keystone:DELETE=7,keystone:INSERT=1 {{(pid=88113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:12:11.895453 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc098ad-2e33-4cce-9858-559558465f44 admin admin] Authenticating user token {{(pid=88114) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:12:11.896856 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.fernet_utils [None req-2dc098ad-2e33-4cce-9858-559558465f44 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88114) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:12:11.967164 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc098ad-2e33-4cce-9858-559558465f44 admin admin] Validating token access rules against request {{(pid=88114) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:12:12.028639 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60246d11-b729-4075-a185-f1a8e23615df service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ff3aff5ef444910be6dbb989d0e3194', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '49548eb44d62415582af820ab66decee', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88114) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:12:12.029239 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60246d11-b729-4075-a185-f1a8e23615df service placement] REQUEST_METHOD: `GET` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:12:12.029543 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60246d11-b729-4075-a185-f1a8e23615df service placement] SCRIPT_NAME: `/identity` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:12:12.029812 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60246d11-b729-4075-a185-f1a8e23615df service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=88114) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:12:12.031523 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.fernet_utils [None req-60246d11-b729-4075-a185-f1a8e23615df service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88114) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:12:12.043220 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60246d11-b729-4075-a185-f1a8e23615df service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:12:12.044790 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60246d11-b729-4075-a185-f1a8e23615df service placement] RBAC: Authorization granted {{(pid=88114) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:12:12.054515 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: [pid: 88114|app: 0|req: 1474/2944] 10.222.0.13 () {60 vars in 1365 bytes} [Wed Aug 31 10:12:11 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 160 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 10:12:22.050316 dv-cinder-844961-2-iscsi devstack@keystone.service[88114]: DEBUG dbcounter [-] [88114] Writing DB stats keystone:SELECT=58 {{(pid=88114) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}